Skip to main content

Experian Pandora: who should validate the validation rules?


Derek Munro 2 minute read Data quality

In my experience there are two ways you find out that business rules have been incorrectly specified:

A technical colleague builds and supplies the results of the rules, which you then realise are wrong. A common response at this point is to claim your colleague misunderstood the rule….

During a discussion with a colleague at the coffee machine you discover there’s a problem with a rule, and you rush to issue a revised specification before anyone notices the mistake.

So, who should validate the validation rules, and how should they do it?

Ideally, the subject matter experts should examine all their actual data and specify rules which (at least) take account of all the variations in that data. But experience shows that just doesn’t happen – there isn't enough time, there isn't easy access to the data, there's too much data, and so on.

Last week I was given some data validation rules which had been specified and validated by a team of subject matter experts based on their business knowledge and experience, as well as what they believed about the data. I used Experian Pandora software to trial the first rule, only to find that the rule itself was wrong.

I then worked with one of the subject matter experts and used Experian Pandora data prototyping to do exactly what was required, defining, refining and validating the rule interactively using all the existing data.

In two hours we were able to redefine, prototype and implement the rule to achieve its objective, and automatically generate the corresponding documentation showing what had been done. This is a common example of how Experian Pandora's data prototyping capabilities is reducing by 95% the effort needed to validate and transform data.

Experian Pandora data management software enables organisations to understand, transform and manage the quality of their data more easily and quickly than any other software product or manual approach. Experian Pandora is used on projects such as data integration, compliance, DWH, fraud detection and data governance.