Skip to main content

The Big Data Revolution


David Lopez Serrano 3 minute read Data quality

At this point you have probably heard about big data a million times. It's a bit of a meme and a marketing term. It probably won't predict wars before they happen1, I personally doubt that it will cure cancer2, and I’m not sure if Netflix really needed big data to create their hit House of Cards3. Although I have to admit, Kevin Spacey was pretty good.

But think about this.

  • 500 million tweets are sent every day
  • 100 hours of videos are uploaded to YouTube every minute
  • 1.8 billion photos are posted each day on Facebook, Flickr, Snapchat, and WhatsApp
  • Every day Spotify generates more than 4 terabytes of data (they currently have 28 petabytes of storage, spread out over 4 data centres across the world)
  • Everyday 2.5 quintillion bytes of data are created

In other words: we have created more data within the last 2 years than in the 2000 years before.

This staggering amount of new data being created and all the new opportunities associated with it will have a big impact on a wide range of fields as varied as science, sports, advertising and public health, just to name a few.

Let’s for example take the Public Sector. Extraordinary quantities of data are processed on a daily basis, from managing welfare payments to issuing passports and driving licences.

“It’s a revolution,” says Gary King, Director of Harvard’s Institute for Quantitative Social Science. “We’re really just getting under way. But the march of quantification, made possible by enormous new sources of data, will sweep through academia, business and government. There is no area that is going to be untouched.”

Organisations are preparing to invest in big data

It’s not a surprise that according to a recent A.T. Kerney study more than 45% percent of companies have invested in business intelligence or big data solutions during the last two years. Moreover the study estimates that more than 90% of fortune 500 companies are planning to invest in at least one big data initiative this year.

On the other hand, recent research we carried out here at Experian revealed that almost 63% of organisations lack a coherent, centralised approach to data quality.4 Data quality organisations need to provide guidance to  companies on their (big) data journey, because it is not the amount of data that is revolutionary, it's that actually we can do something with it.




4 Experian Data Quality Global Research 2015