Part One: Move Over Big Data. Say Hello to Big Knowledge.

Over the past few years, much has been made of the explosion in the quantity of data being produced by humanity. The statistics are absolutely staggering:

• We currently produce about 50,000 gigabytes of data per second.
• 294 billion emails are sent and over a billion Google searches are conducted every day..
• By 2020, the amount of data stored on all servers globally will exceed 44 zettabytes (44 trillion gigabytes). For those of us old enough to remember 10MB hard drives, this is almost incomprehensible.
  

Incomprehensible, but nothing new. Back on 2011 I wrote in this same space about this very same topic. At the time we were producing about 1.8 zettabytes – enough data to store over 200 billion high definition movies – per year, but as you can see above, today the numbers are even crazier.

The growth in data production continues to vastly outpace population growth, so it should come as no surprise that investment in “Big Data” companies and technologies continues at a fever pace. In 2015 alone there were 462 venture capital investments in big data totaling over $5.6 billion. Even with some modest pullback over the past couple of years, VCs continue to dump exorbitant amounts of money into the sector. After all, conventional wisdom goes, what good is all of this data if we can’t organize, retrieve, and make sense of it?

What might surprise you is how much “data” is actually floating around in human heads instead of servers. Those numbers, in some ways, are even more astounding. For example, did you know that the storage capacity of the average human brain exceeds that of THE ENTIRE INTERNET? It’s true: the average brain can store as much as a quadrillion bytes of information.

With the world population close to 7.5 billion souls, humanity’s total data storage potential is something on the order of 7.5 yottabytes (a yottabyte is 10^24 bytes).

By the way, much is made of the fact that humans only use ten percent of their brain capacity. Unfortunately, that’s an urban legend. In reality, we use virtually all of that capacity on a daily basis. Turns out you need all of those brain cells (or, more specifically, neurons) after all.

Although clearly not everybody has the same brain capacity and the global population includes lots of incomplete or partially-formed brains (regardless of what your teenager tells you), the fact remains that there is far more information stored within us than in all of the servers everywhere in the world. Even if digital data production continues to accelerate at the same rate as it currently is, computers won’t “know” more than humans for at least another couple decades.

I call this Big Knowledge, and it’s time we start paying more attention to it.