Nearly everyone in the world of information technology today is in agreement that data quality is of the utmost importance. Modern firms are looking to do everything they can to make strategic decisions based on business intelligence, and big data is a big part of that movement. If companies aren't able to verify the accuracy of their data, however, the movement goes nowhere.
Quality is a must - that much is agreed. But how can companies ensure this quality? That question is tougher to answer, and it's been the source of much disagreement among IT business leaders in recent years.
Information Management recently published a long feature on this debate. Dan Myers, who manages enterprise data management initiatives for Farmers Insurance, argued that because there is dissension among IT professionals about what "data quality" means and how firms should ensure it, it's difficult for the industry to move forward.
"Now is the right time for the data quality industry to finalize a set of standards, much like the accounting field has done with the Generally Accepted Accounting Principles," Myers wrote. "Every organization needs to have a defined set of measures of quality, which should be composed of industry standard dimensions. Each organization should then identify its unique needs for measurement."
Before industries can agree on data quality standards, they must answer a few questions about their data initiatives.
What necessitates data quality initiatives?
Companies must agree on why data quality is important. There are several reasons - one big one is human error, as often, data is collected from forms filled out by users, many of whom make typographical errors or deliberately mislead companies with false information. Also, data can become outdated over time, meaning companies must invest time and effort correcting it.
What concepts define "quality?"
Companies must know what "quality" is before they can guarantee it. One criterion is using address management solutions to make sure people's contact information is correct. Another is looking at demographic data, verifying people's ages, races and other information about them. There's also financial information, checking numbers for any irregularities. Different kinds of data have different quality problems.
How accurate is accurate enough?
Companies must also agree on a standard baseline for how accurate their data must be. Is 80 percent quality good enough? Is 90? Striving for perfection is nice, but it can become impractical to invest too much time and money in the process. Companies must know how far is too far.