The Big Data EconomyWhy and how our future with data is cleaner, leaner, and smarter.

2The Big Data Economy: Our Future With Data

3...6...9...10...12...16...

17...19...

30...31...32...33...34...

Table of ContentsPart IIntroduction to the Big Data EconomyPower of InformationThe Challenge of Understanding Big DataUsing Data to Create ValueDifferent Types of DataThe Complexity of Big Data

Part IICatching Up to Data - A Brief HistoryRational Thinking Over Time

Part IIIThe Future is ClearWe Have to be Bold Like GalileoLearn to Think at the Scale of DataThe Data Will Self IdentifyWe Will See Interdisciplinary Data Layering Creating Value

3The Big Data Economy: Our Future With Data

So much of our American identity is tied up in the iconic image: the ability to move freely, to experience a wider array of goods than when we were regionally locked by limited infrastructure, and the opportunity to pursue wildly different dreams than what were possible before these paths to prosperity were laid. For example, there’s a 76.6% chance that you or a loved one uses a car as your primary source of transportation to and from work. Considering that nearly 130 million people were employed in 2010, that’s a lot of cars buzzing around every day. Just a century ago this number was impossible. Cars were expensive, roads were spotty, and the morning commutes were much shorter than those seen in modern American cities today.

The 20th century witnessed the shift from a rail-centric transportation system to one dominated by the automotive centric industry—one that opened up the path for people and goods to travel wider, more distant paths on extensive public roads projects.

Welcome to the Big Data Economy

4The Big Data Economy: Our Future With Data

The advent of the Internet in the latter half of the 20th Century as an accessible, user-friendly apparatus created a new highway: the Digital Highway. Whereas the Interstate Highway System exists as an enabling infrastructure for moving people and physical goods from one point to another along varying stretches of road, the Digital Highway has allowed for the proliferation of unlimited amounts of data—data that travels with far fewer physical limits than what it takes for your car to get from point A to B.

Since highways were built, American consumerism has been fed like a wild beast that drove its size to proportions previously unseen, but there are always barriers and limits to what can be created and consumed. There are only so many resources in the world, so many items that can be dreamed up, so many people who are able to obtain and use them, and only so much room to store these items while not in use or once they have become waste. When it comes to the Data Economy, there are no such limits.

The Internet and all the amazing, mundane, useful, and incredibly useless things we do with it has created an invisible world where data can grow at rates virtually unconfined by the laws of physics. Sure, there’s only so much storage for all that data. As storage options grow cheaper and sensors and other data collectors create increasingly large amounts of data, there’s no stopping the snowball. The potential for data to grow is presumably limitless, and that prospect is not always easy to grasp. The passage of time and history has proven that humans fear of what we don’t know—what we can’t see, what we can’t wrap our minds around, and what can’t be sensed. The best news is that we don’t need to fear the data-driven Digital Highway.

The Big Data Economy: Our Future With Data

Share this eBook:

5

Data is a precious thing and will last longer than the systems themselves.”

Tim Berners-LeeInventor of the World Wide Web

6The Big Data Economy: Our Future With Data

If you have ever had to drive through the massive Dallas-Fort Worth metropolitan area, you’ve probably been on one of the five stacked bridge decks of the massive High Five Interchange. It’s a tangled mass of controlled chaos. This major infrastructure achievement replaced a clover-leaf design left over from the Eisenhower period and was finished in 2005 because there was simply too much demand from many streams of traffic for the previous system to hold. The current world of data is not unlike the High Five Interchange. It is a massive undertaking necessitated by demand. Since humanity began to record the world around us and strive to make sense of it, we’ve used words and numbers to collect our memory and track numbers pertinent to our lives. This road has been, for the most part, a fairly linear one. Until Gutenberg introduced the first modern printing press in 1450, knowledge had to be written or carved into stone by hand and preserved from the elements. The book allowed collected knowledge both to be protected from the elements and disseminated with less error than the previous expensive, manually intensive copying manuscripts.

In centuries since, literacy rates had to catch up to the slow and steady growth of recorded knowledge that came with easier means of disseminating it.

Power of Information

7The Big Data Economy: Our Future With Data

History does not see literacy rates catch up to affordable access to information until the early 20th Century—the same century that witnessed a massive growth of available goods, services, and transformative new technologies like the Internet. In a world that constantly demands more needs and wants, including disparate types of consumable information, the Internet has become a multi-layered deck of services that are constantly being used to create more and more data. While the highest point of the High Five interchange might be scary to approach, mounting this artificial hill is necessary for a lot of traffic to travel efficiently where they need to go. As data’s size grows we will have to place more trust in the infrastructures that we have built for ourselves to be efficient, well-engineered pieces of an oiled machine.

There’s no denying that the world has seen enormous change in the last 100 years. The years since cars and planes were first invented have been momentous and the growth of data that has come with the introduction of a consumer-based Internet has caused what a lot of people and businesses picture to be a daunting mountain of insurmountable information.

The current world of data is forcing businesses, governments, and other organizations to reassess how they are functioning and how they can use the data they have collected so rapidly. As with most shifts in thought and function, there ends up being more good than harm done, even if the world spins in transition.

The essential rule to remember about data is this: don’t panic.

The Big Data Economy: Our Future With Data

Share this eBook:

8

Data are becoming the new raw material of business.”

Craig MundieHead of Research and Strategy at Microsoft

9The Big Data Economy: Our Future With Data

Just how much data does there have to be to be considered “massive?” Studies suggest that nearly all of the data that has been created in human history has materialized in the last two years, and the speed at which it will continue to grow is staggering. The amount of information swirling around in those unseen wires is measured in units that dazzle the layperson. While most regular computer users think of hundreds of gigabytes as being a lot of storage space, the average supervisor of large data sets encounters measurements like petabytes and zettabytes: 1,048,576 or 1,099,511,627,776 gigabytes, respectively. With big, scary sounding words and numbers like petabyte and zettabyte, the knee-jerk way to talk about and understand this type of data is to conflate it with hype and fear.

The Challenge of Understanding Big DataThe challenge in our relationship with data comes into play because it’s an idea that lives in the realm of the imagination, and we tend not to trust things that we cannot easily see and understand. With the exception of the computer screen you use to access specific information that you have queried or created, the Internet and its infrastructure make up an invisible landscape of data. The veins of the Internet’s body are fiber cables buried underground. They run along power lines that most of us forget exist. These cables run from homes to large buildings full of servers that are like brains that store and manage the flow of massive amounts of data every second. While that infrastructure is very much a thing of the physical world, you cannot walk up to these buildings and see the actual data housed there.

It’s invisible. This is where the challenge of understanding massive data sets comes from.

10The Big Data Economy: Our Future With Data

With any new infrastructure or system come doubt, fear, confusion, excitement, anxiety, frustrations and awe. The thing to accept is that this magnitude of data will continue to be mostly invisible and we will be required to press our ability to think in the abstract. This presents a new opportunity to realize that while the magnitude is huge, most of the data that exists in this invisible landscape is worthless on its own. Once you understand that something is both invisible and mostly worthless without some addition, it’s easy to move past the shock and awe of it and come to grips with how to use the data to create value.

The best way to move past the misunderstanding of this invisible force is to picture what it is made up of and where it’s coming from. Since the invention of the Internet, a number of different data streams have developed that are augmented by businesses’ data and the data coming from mobile devices. If the Internet is an information superhighway, there are multiple lanes that represent different types of data. • Website data: the collection of information from and about websites that includes the information on the website, web

traffic data, rankings, where the website is linked in, and so on.• Social Media data: this data set encompasses all posts, messages, and emails, how they’re interconnected and

shared, the geolocational data around them, and how often they’re shared.• Mobile data: comes from your device’s interactions—usage times, data use, where the device has been, and any other

actions that you have performed on your device.• Machine data: generated by computers and other sensors.

Using Data to Create Value

11The Big Data Economy: Our Future With Data

Despite the caveats, there seems to be no turning back. Data is in the driver’s seat. It’s there, it’s useful and it’s valuable, even hip.”

Steve LohrTechnology Reporter at The New York Times

12The Big Data Economy: Our Future With Data

There are also many other types of data that are not mentioned above, primarily because they are not nearly as big an instigator in the recent boom of data’s invisible cloud or have not yet been digitized. A macro-level list of these types of data includes, but certainly is not limited to, the following: • Medical data: the information collected about your health during medical visits, whether they’re for preventative health

or treatment of a condition.• Governmental data: this information encompasses the various branches of governmental entities and the recorded

information about the places they are governing and their constituency.• Business data: businesses have collected data about purchases, inventory, and other transaction information in

varying forms for quite some time. This data can include sales lead information, pertinent customer data, inventory information, and purchasing patterns. This is an incredibly varied set of data.

• Scientificdata: one of the largest creators of new data, physicists, geographers, ecologists, biologists, and chemists are generating profound amounts of data as science moves towards the more ambitious experiments and investigations that modern technology allow for.

• Historical data: the annals of human history largely reside in books or other forms of record, a number of which are either currently digitized or will become so as electronic archives grow in importance and funding.

Different Types of Data

13The Big Data Economy: Our Future With Data

All of these varying types of data populate the invisible landscape of the data superhighway, whether they are currently digitized or not. The amount of digitized data will only grow as these varying forms of data make their way into this ether. As these varying types of data are considered, something becomes very evident about them: whereas most of human existence was spent dealing with one type of data, numerical, we have moved into a brave new realm of data creation that exists around so much more than strings of numbers. • Structured data: the image most of us recall when we think of data is a spreadsheet populated with numbers that

reflect certain information. This is one type of structured data. Structured data is data that is stored methodologically and in an orderly fashion.

• Unstructured data: one of the most relevant examples of unstructured data is Twitter. The stream of tweets really has no rhyme or reason to it, other than the fact that it is a stream of words. Unstructured data is composed of photos, text, documents, videos, and emails, to name a few types.

• Numerical: this type of data is exactly what it sounds like—data in numbers. This type of data is what we have analyzed for thousands of years with mathematics.

• Non-numeric: yet another part of an oppositional pair with a name that does a lot of the explaining itself, non-numeric data is any other data that is not a number—words, pictures, images, etc.

14The Big Data Economy: Our Future With Data

Consider these lists in the context of your life or the business that you do every day. The data in your day might consist of the geolocational information your cell phone gives out on the way to work, the tweets of articles sent out that are relevant to your industry (or not), the financial data created by the bills paid, and could end with the information your Netflix account collected on the shows you watched and rated. This ballooning world of invisible data probably means many things to many people, but how do we create some kind of value from it all? The answer lies in collaboration.

The Big Data Economy: Our Future With Data

Share this eBook:

15

Big Data is the Holy Grail: It promises to unearth the mathematical laws that govern society at large.”

Albert-László BarabásiPhysicist and author

16The Big Data Economy: Our Future With Data

Would you ever pay for Facebook? How about Twitter? Google is virtually a free utility to most of us these days, and whether we like it or not, the engine has become a cultural icon of the connected age of free web utilities. Because so many of these data-creating forces in our lives are free, they have allowed for the monstrous growth of data discussed earlier in this chapter, and all generally without any potential for valuation as a stand-alone resource. What happens, though, when data types are crossed? What do you get when you mix business data and social data? What about structured governmental data with unstructured tweets from suspicious persons? And consider geolocated photo data mixed with scientific data collected about a recent earthquake? The dimensionality of combined data is what creates the opportunity to find real value in disparate, massive stores of information. Just as the value of the Interstate Highway System grew when goods were transported more frequently along its thoroughfares, the combination of two seemingly unrelated types of data frequently results in insights that did not exist within its own context. Have you ever noticed the way your trips to the supermarket tend to work? When you subtract the marketing tactics used to get you to pass things you don’t need, more often than not shoppers will walk out of the grocery store with a thing or two more that was needed that was not on their list. Combining more data allows for broader, richer results and insights that might be surprising. Just as the Highway system and the Internet were born of military defense projects that became incredibly more valuable with civilian use, so will data increase in value as it’s allocated for more uses. Possibilities abound across industries because of the interdimensionality of disparate data pairings—possibilities that are already driving economic incentive and will continue to do so more and more as organizations of all kinds begin collecting their data and sharing it. The next chapter explains the history of where analytics comes from, why we’re in for a new order of analysis, and why analyzing data efficiently will set us all on the fast-track to a robust data economy.

The Complexity of Big Data

17The Big Data Economy: Our Future With Data

Catching Up to Data - A Brief History.

18The Big Data Economy: Our Future With Data

Hypothesis has been the choice for thinkers and researchers for the length of modern thought. Formalized processes of logic and hypothesis emerged independently around the world as a means to explain ideas and opinions, from the supposition of planetary motion that Copernicus posited to CERN’s search for the Higgs-Boson particle. The approaches and arguments for and against logic and hypothesis as a framework to understand the world have evolved as we have explored, and we’re beginning to approach a time when this type of thought must evolve to solve new magnitudes of problems.

Since we’ve created data over a period that extends all the way back into our history as intelligent modern beings, we’ve created a lot, but the realm of data we could analyze systematically has been limited. Compared to the amount of data that is building up in the world, much of the data from human history is an incomplete picture of our world has existed for some time, but we’ve had the tools to understand the information we have had access to within context, so we’ve made do. Now that we have more and varied kinds of data to understand, the mathematical engine that has been established and built off of for thousands of years is not quite the sufficient logical framework to understand what we are collecting, and hypotheses have become less of a strength and more of a weakness. Theoretical mathematics has to catch up to this new framework of data, and it has to do so faster than the human mind can really catch up. While we are intelligent creatures, our minds have been cast into the massive amounts of data and we’re scrambling to understand it with little conceptual context in place to guide us.

In order to understand the Herculean task of catching up to the data around us, we must consider the history of where our mathematical mechanics came from, the type of challenges that thinkers have faced, and that slow and steady was able to win the race up until now.

19The Big Data Economy: Our Future With Data

The history of mathematics is long, and since we began counting, measuring, and recording the world around us, we’ve sought ways to develop systems that quantify and track everything from the passage of the stars through the sky to what our hunting trips looked like on cave walls. The earliest evidence of any suspected human recording of time, number, or complex expression of numbers happened through bone carvings.

Between 20,000 and 35,000 years ago, humans on separate continents chose to carve representations of what scientists’ best guesses are mathematical expressions. Due to our own current limitation of knowledge about their world and what might have driven these people to create these markings, we can’t be sure what the ideas expressed were, but they demonstrate the earliest of human curiosity regarding what the world could show, and how to record things for more complex understanding later.

The great ancient civilizations display a vast amount of knowledge based off the observable world, and what they developed through hypothesis and proof is an established foundation of mathematics as we know it today. Hellenistic thinkers borrowed ideas from Egyptian and Babylonian tradition and knowledge, developed their own theories and hypotheses, and proved the theories with deductive reasoning patterns that paid more vigorous attention to sound proof than any other system seen before. Pythagoras formed a school of thought that actually gave mathematics its name and embraced the idea of math for math’s sake--not to mention solidifying the Pythagorean theorem, everyone’s favorite high school geometry proof. Two hundred years later the Platonic Academy reinforced the importance of Pythagoras’ work and allowed for the development of the hypothesis as a means for investigating and proving presumptions about the physical world through mathematical reasoning.

Rational Thinking Over Time

20The Big Data Economy: Our Future With Data

It’s a revolution. We’re really just getting under way. But the march of quantification, made possible by enormous new sources of data, will sweep through academia, business and government. There is no area that is going to be untouched.”

Gary KingDirector of Harvard’s Institute for Quantitative Social Science

21The Big Data Economy: Our Future With Data

As time progressed and ideas spread this system grew more refined and complex across civilizations, but the information available for analysis was still mostly limited by the observable world and the speed by which the human mind could postulate a hypothesis and prove it. Developments in Arabic mathematics between the 8th and 15th centuries A.D. included algebra the cornerstone of modern applied mathematics. Other traditions granted their knowledge as trade and exploration expanded what we could observe around us.

As citizens of a fast-paced world where time consistently feels crunched into smaller and smaller proportions, this staggeringly slow process of obtaining more and more enlightenment seems virtually unimaginable. The truth is that we have been grasping at whatever is around us to understand the world, and until now this set of information was far more limited than it currently is.

There was, however, a historical turning point—one that most people around the world learn about as a turning point that fueled our tumbling run towards the age of modernity. The collection of mathematical concepts and other observations based on hypothesis allowed eventually allowed human thought to move beyond superstition into digging even deeper to prove facts, a development that paved the way for one of the most productive periods of thought that we have ever seen: the Renaissance.

22The Big Data Economy: Our Future With Data

Many of the advances made up until the Renaissance were made in the name of particular biases meant to prove the existence or importance of philosophical ideas that served as many people’s understanding of life, the universe, and their sense of morality. Plato approached the world around him not simply to observe, learn, and present an unbiased image of his most recent point of enlightenment.

Instead, those in the Platonic school of thought approached their hypotheses with a larger end goal in mind: what is the Truth? History proves that other mathematicians and scientists approached questions about the observable universe with the end-goal of using what was found to prove their own philosophical framework.

Before Copernicus, astronomy was a melting pot of ideas that placed the earth is at the absolute center of the known universe and painted the oddly modern, science fiction-like picture that there was a so-called counter-earth balancing the cosmos.

While Copernicus was a religious man, he made the bold move to present a hypothesis based around what he could actually observe. Instead of forcing dogma that evolved over hundreds of years into the shaping of his theory, Copernicus developed a theory based on mathematics and left the interpretation of his heliocentric idea open, regarding it neither as truth or fact.

Copernican heliocentrism challenged Ptolemy’s theory of the earth being at the center of the universe. Placing the sun near the center of the universe versus the earth would maintain a mathematically elegant explanation for why a year was the length that it was, all the while retaining the idea of an ordered cosmos that was essential to appeasing skeptics of his time. Copernicus did his best to function within the system of observation and thought that he was working in. He’d seen some light, but was not ready to completely compromise everything he knew and believed regardless of observable fact.

23The Big Data Economy: Our Future With Data

Big data means much more than a change in technology, it represents a structural transformation in how we will manage our enterprises.”

Greg SatellTechnology Reporter at Forbes

24The Big Data Economy: Our Future With Data

Galileo Galilei, the man regarded as the father of modern science, was perhaps a greater champion of Copernican heliocentrism than the reluctant Copernicus himself was. Galileo was one of the Renaissance’s foremost astronomers, the creator of the modern telescope, and one of the first in history to separate scientific thought from religious or philosophical prescriptions or presumptions. Because Galileo believed mathematics to be the basis for the laws of nature, an idea that laid the path for Newton’s Laws, his observations were freer of preconceived notions than any other scientist and mathematician before him. This revolutionary freedom allowed Galileo to prove Copernicus’ theory by applying an understanding of ellipses to the orbital system that Copernicus hypothesized—and the consequence was his pursuit by the Church under charges of heresy.

Instead of approaching information assuming that the outcome should support a specific philosophy or dogma, Galileo observed what he could of the solar system for what it was, with the tools he had or invented, and attempted to understand the recordings of what he found within the least biased framework he knew: applied mathematics.

Galileo’s collected data was by no means small, but in the age of the Renaissance, the amount of data collected was still within the reach of human understanding. This was an age of collecting, watching, and patiently noting information while attempting to understand the meaning of larger questions about the world around them through math, science, and other means. The tools developed from the ancient civilizations that were rediscovered, reexamined, and refined during the Renaissance were a fantastic mechanism for this set of thinkers to develop the groundwork for the speed of advancement that we have seen since.

25The Big Data Economy: Our Future With Data

Since the great thinkers of the Renaissance were brave enough to sacrifice the convenience of previous assumptions about the existence of everything around us, science and technology have bloomed—sometimes slowly like the develop-ment of energy technologies that fuel our world, sometimes with the kind of brute speed that we’ve seen since computing became a practical prospect. These Renaissance thinkers were like the first to step out of the cave and see what they could of everything around them that was real, that was observable, that was reality freed from superstition.

They had their chance to shake the rest of humanity into realization that there are so many things to observe and under-stand as their own set of facts, and they did just that with a number of other great thinkers that pressed humanity into the modern age.

There is, however, some trouble with what has developed since The Great Rebirth. The establishment of the modern scientific method of developing a hypothesis, performing experiments to prove it, recording the results of the experiments, and coming to a conclusion led to the ever-increasing growth of data available about the physical world.

This process allowed the world in the centuries after the Renaissance to move into the age of modernity—into the In-dustrial age of machines, the age that commonly enable people to do more, move more, and make more. The growing complexity of the world required better ways to understand what was happening than the traditional methods of scientific and mathematical hypotheses, a requirement that led to the development of many offshoots of applied mathematics.

Since the Renaissance, the world of mathematics has seen the development of many applications including statistical anal-ysis, probability modeling, engineering mathematics, computational mathematics, and so on. Applied mathematics enabled the growth of a number of theories and practices, including physics, chemistry, biology, engineering, and computing.

26The Big Data Economy: Our Future With Data

Applied mathematics and the scientific method have allowed us to build flying machines that take us into space, printers that build three dimensional objects, have populated the earth with over 7 billion people, and have created hand-held computing devices that possess the same order of computing power as the Saturn V rocket.

We are able to solve incredibly complex equations, use math to develop theories of quantum physics, and can understandtheinnerworkingsofourever-fluctuatingglobal economy.

The method of hypothesis has been a useful tool in helping us understand the complexity of the world around us. Until now, though, the realm of data available to us to understand things have been somewhat limited to small, numerical, mostly struc-tured sets of information.

Hypothesis and mathematics allowed us to create the most sophisti-cated inventions that have been conceived in all of our history. What happens, though, when our frame of thought can’t keep up with the data created around our creations?

27The Big Data Economy: Our Future With Data

Just as it was important for Galileo to know that Platonic thought was not always the best answer to understanding the world around us, and for modern scientists like Einstein to challenge his contemporaries’ theories, it is very important for those currently up against the invisible force of data challenges to question whether or not the formation of their thought process is the right one to tackle this magnitude of data.

While we might have thousands of years of mathematical thought to guide us to this point, as well as the scientific method of hypothesis, massive amounts of data require a new framework of thought that does not include approaching the data with any thought of what one might want to come from it.

By attempting to apply the principles of hypothesis to the order of data that we now have, we’re attempting to fit a square peg through a round hole. We’re like the order of scientists and mathematicians that Copernicus and Galileo were bold enough to look past, only the order of data that we’re dealing with leave us helpless. Why?

The previous generations were dealing with data sets that were manageable by human brains, and the urgency of the data that was collected was not so important that time was of the essence. The greatest challenge facing the data economy is that the demands of time and the urgency of understanding the data collected leaves no room to hypoth-esize, test, and come to conclusions about new thought frameworks and solutions in the manner that we’ve devised them since ancient philosophers began seeking the enlightenment of systematic understanding of the world.

In the coming age of the Data Economy, we must force ourselves to consider where we have been, wash our hands of what tools brought us here, and be bold enough to approach the challenge with a fresh, minimal perspective.

28The Big Data Economy: Our Future With Data

The volume of Big Data demands a change in the human relationship to data. The algorithms have to do the work, not the humans. The role of the analysts will be to select the best algorithms and approve the quality of results based on speed, quality and economics.”

Radhika SubramanianChief Executive Officer at Emcien

29The Big Data Economy: Our Future With Data

4 Ways the Future of Data is Cleaner, Leaner, and Smarter

than its storied past.

30The Big Data Economy: Our Future With Data

The key interruption in the way we think about the universe right now is the fact that our systems were built with the assumption that we’d have to work with what little data and knowledge we had. Building a hypothesis is based around the notion of considering what you know, what’s possible from what you know, measuring based on experiments, and finding if you learned anything new based on what you already knew. Now that so much data is readily available from so many places, we have to step back and consider that thinking about outcomes within the scope of this previously limited approach simply does not work. The sheer scope of data available to us indicates that many of the assumptions we’ve made about the world based on limited data might, in fact, be entirely wrong.

While we may not yet know everything about everything, data is out there, just waiting to be collected or generating, and the capacity to store it is very much in place. It’s hard to conceive of thinking like Galileo when all he had was a rudimentary telescope to see beyond the veil of the earth’s atmosphere when we have telescopes that are cataloguing billions of stars, their billions of attributes, and the billions of moons and satellites within their systems. As we approach larger and larger data sets, the tools we use to analyze them cannot, by necessity, be the tools of the past. They have no biases.

We cannot know what direction data will drive us in over the coming decades, but we can understand the types of thinking that must and must not be retained or creating to move forward in understanding this invisible force that’s changing our lives.

Here’s a list of how and why the future of data is cleaner, leaner, and smarter than its long, storied past.

The Future is Clear

31The Big Data Economy: Our Future With Data

One of the biggest challenges that analytics project leaders across industries are having right now is one of general innocence, but is shrouded in the veil of being wrong. Because we’ve always been taught to move into a research-based project with an end-goal in mind by the thousands of years of tradition that inform our thought process and education, we approach new information and projects with biased objectives. Data of the magnitude that we’re currently encountering begs the user to allow its insights to come independently of any presumptions or framework.

There must be an end to the hypothesis when approaching data sets this large. If we do not end this thought pattern, we could potentially make disastrous decisions based on data that we assumed was unbiased, but that was, in fact, dangerously biased due to human preconception. Imagine the trajectory of history if no one like Galileo ever made the bold choice to stand back and let the reality of what he was observing speak for itself, rather than impose a perspective on it. While it’s unlikely that we’d still believe the earth was at the center of the universe, it might have taken us much longer to come to that conclusion.

We Have to be Bold Like Galileo1

32The Big Data Economy: Our Future With Data

One of the greatest limitations on human thought around our quickly-growing technology frenzy is it’s growing faster than we can think of ways to keep up with it. Machines are generating large amounts of data each moment of the day, let alone the data generated by humans through their use of the machines. The human brain simply can’t conceptualize new solutions fast enough to keep pace with these developments.

That’s why we must start doing our best human thinking to give our able-minded computers the tools needed to create their own solutions. After all, the computer is engineered to function in ways that our mind, or the collective minds of many people could never function alone. Why not assume that computers can create equations of their own if they can quickly sift through trivial human history and tell a studio audience that Galileo was accused of heresy on an order of magnitude faster than a human can?

Learn to think at the scale of data2

33The Big Data Economy: Our Future With Data

The Data Will Self Identify3 With the development of machines that can piece together the best ways to understand their own data, we’ll also come to the point where the important data will self identify. Once we know what data is useless, and a lot of it will be, the machines will analyze the relevant information and we can use the results to creatively solve problems in an order of magnitude faster than in any other point in history.

34The Big Data Economy: Our Future With Data

We’ll also be able to tell if more data that has already been deemed valuable needs to be layered onto another set to create value. As we begin to approach large data sets, we must not allow ourselves to make presumptions when first analyzing the data, and we must accept that data isn’t telling us anything when it isn’t.

If we find that data doesn’t tell us something, we can toss it out or decide to layer more data onto it, and move on to finding whatever value the layering might offer us.

We Will See Interdisciplinary Data Layering Creating Value4

The Big Data Economy: Our Future With Data

Share this eBook:

35

The magnitude of data that we’ve come across is almost so dazzling that we don’t know what direction to move in. While we might have been out of the cave and into the light for quite some time, the light of reality just got much brighter. As we remember the history of how thought carried us to where we are now, we have to consider the fact that trail blazers are who moved us forward into a world that’s closer to the clearest picture of reality.

As we move forward, we’ll look at the tools that we currently have to handle the magnitude of data that exists, which are best, which might function too close to the old order, and where they will take us in the Data Economy.

Looking Forward to Life with Big Data

Click here to see data analytics tools in action!