Skip to main content

Experian Pandora uncovers what lies beneath your data


Derek Munro 2 minute read Data quality

As part of a customer software evaluation process we analysed some of their data live as they watched.

So I chose three of my favourite questions:

  1. What is wrong with the data everyone thinks can’t be wrong?
  2. What is lurking in the “comments” fields?
  3. Where does the word “test” appear in the various application systems?

The customer had almost 2,000 tables in a single application system, and I chose to start with a small one, the table of users defined to the application – it couldn’t possibly have issues.

Although the application only had 120 users, the outlier report based on Experian Pandora’s automatic data profiling immediately noticed that one of the user ids was duplicated and highlighted this as a broken key.

Then, I asked Experian Pandora relationship discovery about the comments fields. It instantly showed common values between the contract-type reference table and a “customer comments” field. A few minutes of interactive investigation showed that there was lots of other interesting information in the same comments field, and someone from the business area was brought into the meeting room to help us understand.

When talking to customers about their contracts, staff needed to have various pieces of information in front of them, and unfortunately the application system spread this over three separate screens. Rather than suffer the constant paging back and forth, the users had copied some relevant contractual information from the third screen to the comments field on the first screen. Of course with Experian Pandora we quickly established that when the contractual data was changed, no-one thought to modify the comments field which they relied on when talking to customers; the inconsistencies had potentially costly consequences.

I looked for “test” and “do not use” and found them in several places of course, but by then no-one was surprised.

How much could undiscovered data issues like this be costing your business? I’m sure you could come up with validation rules to find the data quality problems you know about or suspect, however Experian Pandora finds the problems you aren’t even looking for.