Skip to main content

Quality has become the 'ugly duckling' of big data

Richard Jones Archive

Every company that peddles in large volumes of customer information has a need to maintain high levels of data quality. Without good data, businesses risk sending mailings to the wrong addresses or making marketing decisions based on faulty assumptions. Even small mistakes, like typographical errors or duplicate database entries, can lead to major slip-ups.

Unfortunately, though, a large swathe of the business world remains in the dark about the importance of data quality or how best to ensure accurate information in their operations. According to TechRepublic, quality is becoming the "ugly duckling" of big data - no one wants to talk about it, but the problem is a reality for businesses everywhere.

It's doubtful that companies can keep sweeping this under the rug for much longer. Bad data is becoming a massive issue. According to a recent survey from TEKsystems, a subsidiary of private talent management firm Allegis Group, 60 percent of IT leaders currently believe their organizations lack accountability for data quality. Furthermore, more than 50 percent of these professionals question the validity of their data.

Mary Shacklett, president of tech research and market development firm Transworld Data, believes that inaccuracies in corporate data, and the arduous ensuing process of cleansing it, represent a serious concern.

"Dirty data is also a challenge organizationally for companies," Shacklett stated. "You can 'job out' [extract, transform, load] tasks to data cleaners, but nobody knows your own 'dirt' better than you do. This is because it's hard to get data squeaky clean without knowing what it really should look like within the context of your business. It is here where data cleaning can become a painstakingly manual task."

It's important that companies take action sooner rather than later where data quality is concerned. Doing so can have a tremendously beneficial effect on overall operations.