Increasing data volumes lead to new thinking

The process of data management has changed in recent years due to the nature of information itself changing. The usage cases behind information have changed. Storage volumes have increased and information once considered unusable due to its format has become an important source of insight.

According to Forrester's Michele Goetz, companies need to create data quality guidelines in the big data era with fresh thinking, "rebooting" their strategies and tackling the problems in ways that would have been considered strange a few years ago. She recently offered tips for keeping quality up at companies bent on increasing their reserves.

New view of acceptable data

Goetz stated that many of the processes developed for structured information will simply have no effect when applied to a large archive. By its nature, big data often accumulates at a rapid rate. Companies are eager to use the information while it is "fresh," to get the most accurate insights possible. This means there might not be enough time for a conventional analysis of the figures.

While standard cleansing may have no effect on big data archives, that does not mean they cannot be used with confidence. Goetz noted that companies may simply have to change their reference points. She suggested that firms can use the rules for third party rather than primary data. Those rules are typically more relaxed, and less focused on making the information fit a rigid structure.

The classic cleansing process alters data to make it fit the rules governing information. Goetz noted that firms can now forego that type of basic change, instead resorting to matching the information to a usable profile. She stated that companies should apply their data quality restrictions in a specific way specially tailored to fit the company and situation's particular requirements.

Big data on the rise

A recent overview of executives at Fortune 1000 companies and government agencies by NewVantage Partners found executives eager to bend big data to a variety of ends. The study uncovered 17 different types of projects inspiring new development. Overall, 85 percent of respondents stated that they have either a big data plan or an analytics program already active. The survey found widespread enthusiasm for both expanding data volumes and increased variety of formats.