How long does it take to read Big Data?

The reading time for Big Data varies depending on the reader's speed. However, the Blinkist summary can be read in just 15 minutes.

Is Big Data a good book? Is it worth reading?

Big Data is worth reading as it highlights the significance of data in today's society, revealing its implications and opportunities.

Technology & the Future

Big Data Book Summary - Big Data Book explained in key points

Listen to the Intro

00:00

Big Data summary

Name: Big Data
Rating: 4.4 (118 reviews)
Author: Viktor Mayer-Schönberger and Kenneth Cukier

Viktor Mayer-Schönberger and Kenneth Cukier

A Revolution That Will Transform How We Live, Work and Think

4.4 (118 ratings)

24 mins

Brief summary

'Big Data' by Viktor Mayer-Schönberger & Kenneth Cukier is a thought-provoking book that explores how data can change how we see our world, and how its use can impact our lives. It poses important questions about privacy, ownership and the decision-making power of algorithms.

Topics

Artificial Intelligence Big Data Mathematics

Table of Contents

Big Data
Summary of 12 key ideas

Audio & text in the Blinkist app

Key idea 1 of 12

Big data provides insights we could not discover by analyzing data on a smaller scale.

Prior to the advent of computers, collecting and recording information was an arduous and time-consuming task. To put this in context, consider the information needed to complete a census of the population. Under the US Constitution, a census is required every decade, yet the 1880 census took over eight years to complete and publish. This meant the information had become obsolete before it was even made available.

But that was then. Now – with the invention of computers, digitization and the Internet – the picture has changed considerably. Information can be collected passively (or with much less effort) and at greater speeds, and the cost of storage is increasingly economical. This has brought us to the advent of the big-data era.

Although there is no formal definition, “big data” refers to both the data being captured on a much greater scale than previously possible, and the opportunities that data-sets of this size offer in terms of valuable insights discovered through analysis.

In 2009, Google provided a great example of the possibilities of big data when they published a research paper showing how they could analyze users’ search terms to predict the outbreak of flu and monitor its spread. They compared historical search-term data with data on the spread of flu in time and space from 2007 and 2008, and discovered 45 search terms that could be used in a formula to predict the spread of flu – a prediction which correlated strongly with official figures.

Only weeks after the paper was published, the outbreak of the new deadly strain of flu, H1N1, hit the headlines. Google’s system was pressed into action and provided indicators that proved to be more useful and timely than government statistics in delivering valuable information to public health officials.

Big data provides insights we could not discover by analyzing data on a smaller scale.

Data is increasingly being collected and put to use in all aspects of our lives, from the size of our bums to the way we walk.

With the rise of Internet companies such as Facebook and Twitter, and the popularity of smart devices, we have become familiar with things such as our relationship statuses, comments, preferences and location being stored as data that can then be analyzed. This trend is part of the process of datafication – capturing information about the world in the form of data.

Because we can discover valuable insights from such data, we are likely to see the trend continue, with innovations in capturing data from sources we had not previously thought of as information.

An example of this trend can be seen at Japan’s Advanced Institute of Industrial Technology, where pressure sensors are used to measure the distribution of weight our backsides put on a car seat. The research has revealed that individuals can be so accurately identified by this information that weight distribution can be used as a security device, with the car starting only for drivers it “recognizes.”

Other companies too have realized the potential in datafication. Apple applied for a patent in 2009 to passively measure the blood oxygenation, heart rate and body temperature of users through the company’s earbuds. In a similar move, IBM was awarded a patent in 2012 for touch-sensitive floor surfaces, which have the potential to identify where and how different people are moving across it.

As these examples show, researchers are already harnessing sources of information we hadn’t previously considered as data. They aim to discover valuable insights into the ways we interact and behave, with an eye on creating innovative new products.

Data is increasingly being collected and put to use in all aspects of our lives, from the size of our bums to the way we walk.

Big data frees us from the limitations of using small samples of data to represent whole populations.

Before our current technological age of the Internet and computing, information was much harder to collect and record. Accordingly, we could collect only very limited amounts of information and then try to interpret them as best we could.

For example, say you wanted to conduct a telephone survey of voters for an upcoming local election. Clearly, it would be impossible to contact the entire population, so you call a few hundred people and assume that their answers reflect the whole population’s opinions. This approach is called sampling: you take a sample of all the data, and hope it is representative of the whole.

But what if a journalist approached you after you had conducted the survey and asked you to predict the votes of a specific segment of the population, for example, the public servants?

On looking through your data, you find that you have surveyed only ten such people, and therefore can’t make very reliable predictions.

You are then asked about an even more specific subgroup, say, public servants under the age of 30. This time you have queried only one such person, and hence you can’t make any predictions at all.

This is the inherent problem with sampling: when you begin to examine smaller and smaller subgroups of data, you will quickly find that you have insufficient observations to draw any meaningful conclusions.

In a big-data world, information is much easier to collect because we have access to much more of it, or in some cases all of it. This is why in a big-data version of your election survey, you would probably have information on the voting preferences of tens of thousands of people, possibly even everyone in your town. This would make it possible to “zoom in” on subgroups in the data almost endlessly.

Big data frees us from the limitations of using small samples of data to represent whole populations.

Vast sets of messier data can be more useful than smaller, more accurate ones.

While trying to develop a language-translation program in the 1980s, the engineers at IBM had a novel idea. They decided to dispense with the standard method of using grammar rules and dictionaries and instead allowed the computer to rely on statistical probabilities to calculate which word or phrase was called for, based on samples of translated text they fed into it.

IBM’s engineers decided to use a large but limited sample of high-quality data, using three million sentence pairs from official translations of Canadian parliamentary documents. Despite promising early results, the project failed. Although the system could provide reliable translations for the most frequently used words and phrases, it was less reliable for those that occur infrequently. The system failed, despite the quality of the data. The problem was the quantity – there was simply not enough of it.

When we have only a small proportion of the data, inaccuracies can be a big problem, especially when we want to look at results that occur infrequently. But as we move to having significantly higher proportions of data, inaccuracies have a much smaller effect on the results.

Less than a decade after IBM’s failed attempt, Google decided to tackle the translation issue with a slightly different approach. They decided to use a much bigger data-set of questionable quality: the entire global Internet. Their system scoured the web and used any translation it could find, amounting to billions of pages of text. Despite the dubious quality of the input, the sheer volume of data made the system’s translations more accurate than those of any rival system.

The size of the data-sets we can have with big data allows us to be more forgiving in terms of inaccuracies in the data; having such a large proportion of the available data minimizes the effect of any inaccuracies.

Vast sets of messier data can be more useful than smaller, more accurate ones.

Big data does not tell us why two things are related, just that they are, but even this is often good enough.

When buying a used car, what criteria do you look for to ensure you don’t end up buying a clunker? You might consider things such as age, mileage, country of origin, and the make and model – these criteria all seem pretty logical. But would you take the color of the car’s paintwork into account?

In 2012, contestants in a data-analysis competition were given a similar task, and their correlation analysis revealed a surprising finding: cars painted orange were half as likely to have defects as the average car.

No doubt you are asking yourself why this would be so. Being curious about the reasons behind a relationship and developing theories to explain it is human nature. But one of the implications of big data is that we don’t need to develop our own theories about cause and effect and then test them out. Automatic analyses of all the data can deliver correlations we never even thought of looking for.

In the used-car example, the causes behind the relationship may remain invisible to us, but allowing the data to speak for itself can at least deliver correlations. And finding such correlations can already be put to practical use.

Consider research by IBM and the University of Ontario to help doctors make better decisions when caring for premature babies. By analyzing data on babies’ vital signs, they hoped to identify subtle changes that could signal the onset of infection even before the symptoms become visible. Counter-intuitively, the study revealed that babies’ vital signs became very stable prior to a serious infection – a kind of calm before the storm. Before the research, doctors would have been unconcerned with stable vital signs, but armed with this finding they are now able to provide better treatment when it is most needed.

Big data does not tell us why two things are related, just that they are, but even this is often good enough.

Although data is generally collected for a specific purpose, there are often secondary applications that hold even greater value.

When companies collect data, they generally have a specific purpose in mind: stores collect sales data for their financial accounting, factories monitor their output to track productivity, and websites track mouse movements over their pages to optimize their customers’ user experience. Consider, too, the interbank payment system Swift, which collects data on the billions of financial transactions it processes across the globe in order to provide accurate customer records.

But, increasingly, companies are finding secondary uses for the data they have collected that are sometimes even more valuable than the original use. For example, Swift discovered that its payment data correlates well with global economic activity. As a result, the company now offers highly accurate GDP forecasts derived from their transaction data.

People’s old Internet search terms are another great example of how data can find secondary uses. At face value, the information seems of little use after it has performed its primary function – returning search results to the user – but companies such as Experian allow clients to mine this data to learn about their potential customers’ tastes and market trends, a veritable gold mine to any retail company.

Similarly, mobile phone companies amass real-time location data from their users as part of routing calls. This data has numerous potential uses, from monitoring traffic-flows to delivering personal location-based advertising.

This trend has not gone unnoticed. Big-data-savvy companies and individuals, aware of the value of such data, are already designing products and systems to capitalize on the potential secondary uses of the data they and others collect.

Although data is generally collected for a specific purpose, there are often secondary applications that hold even greater value.

Anyone can spot new opportunities to create value from the data around them – you just need the right mindset.

Owning vast amounts of data is not much use if you don’t know what to do with it. Equally, having the skills and tools to analyze data is of little use if you don’t own any data or don’t know where to get it.

Nevertheless, there are people who have neither of these things, yet still manage to find a niche for themselves in the big-data world.

The key to these individuals’ success is having a big-data mindset: an ability to recognize where available data can be mined for information of value to many people. Although people with such a mindset may not necessarily have data or data analysis skills, they are adept at spotting opportunities and capitalizing on them before others do.

One such individual is Bradford Cross. In his mid-twenties, he started the website FlightCaster with a group of friends. They combined publicly available data on flight times and historical weather records in order to predict delays in flights across the US. Their predictions became so accurate that even airline employees began referring to their site to check on their own scheduled flights.

Decide.com is another company built from these principles. Their computer system records twenty-five billion price quotes for over four million products from e-commerce sites across the web. By analyzing this information, they don’t just provide users with the cheapest price but also advise them on the best time to buy a product, predicting if and when prices are likely to rise or fall.

It’s clear that as economies are starting to form around data, more and more people are beginning to recognize the potential value of data and attempt to extract it. Individuals and companies that have a big-data mindset are best placed to capitalize on this data gold rush.

Anyone can spot new opportunities to create value from the data around them – you just need the right mindset.

Combining sets of data can create greater value than the individual parts.

As anyone who has ever played the board game Clue (i.e. Cluedo) will know, pieces of information may have little value alone but when they are combined with others they can tell you so much more. This is also true for data-sets: sometimes their value becomes apparent only when combined with other data-sets. Trends can then be found in the newly combined data that were not discoverable from the individual data-sets alone.

For example, in 2011 a Danish research group demonstrated this phenomenon. In one of the largest studies of its kind, they combined mobile phone user data with cancer patient records. This meant that they were able to check not only for a link between mobile phone use and cancer, but also whether greater mobile phone use increased the risk.

Critically, they used not merely a sample of the data but records of almost all cancer cases in the country, which allowed them to control for factors such as education and income without the data becoming unreliable. Despite the comprehensive nature of the study, the published results did not receive much media attention because no evidence of a link was found.

Although the above example involves combining different sets of data, similar effects can also be achieved by combining multiple sets of the same type of data that then provides greater value in the aggregate.

Inrix, a traffic-analysis company based in Seattle, is based on this principle. They gather real-time location data from car manufacturers, commercial fleets and their own smartphone app. Piecemeal, this info is not much use to the original data holders, but by combining it, Inrix can create timely data on traffic flows and jams for its users in return for a fee for their service.

Combining sets of data can create greater value than the individual parts.

Online services such as Facebook record everything we do on their sites and use this data to enhance the service they offer.

Most businesses use some form of customer feedback to improve the products and services they deliver. Traditionally though, it has been both time-consuming and difficult to collect feedback in large enough volumes to be meaningful.

In the age of big data and the Internet, information can be collected instantly and with much less effort than before, often even completely passively. Smart companies are already tracking everything we do online, including where we move the mouse and how long we hover over items. This information is referred to as data exhaust and is used to optimize the fine details of products, such as the size and placement of buttons.

Google is the undisputed leader of recycling data exhaust: users’ search queries and even their typos have been used to create a spell-checker and an autocomplete system, both of which are used across all Google services.

The more we interact with a website, the greater the trail of data exhaust we provide. Facebook used their rich source of data exhaust to discover that users were more likely to post content or reply to posts if they had just seen a friend do so. The layout was then amended to make friends’ interactions more visible.

Online gaming is also an area where this data trail can be pressed into service. Zynga’s online games are refined depending on how users play them: if a lot of players give up at a certain point in the game, Zynga adjusts the game to improve the players’ experience.

The examples show that companies who have both grasped the art of recycling user data and implemented it into their systems can enhance the service they deliver.

Online services such as Facebook record everything we do on their sites and use this data to enhance the service they offer.

Current privacy laws and anonymization methods are ineffective and inefficient when applied to Big Data.

It’s hard to spend any amount of time online these days without being presented with a lengthy user agreement at some point. But, be honest, do you actually read through them before agreeing to the terms?

Current privacy laws require that we are informed about what information is being collected and for what purpose, and that we then give consent, which is why we are bombarded with such requests. If the company then wants to share the data it collects, it uses anonymization – the stripping out of any personal details to preserve the privacy of the individuals – before publishing the data.

Although these methods have worked up to a point, the acceleration in the collection and use of data has meant that they are rapidly becoming obsolete.

First of all, the privacy laws prevent companies from realizing secondary uses for data. Imagine that your company has collected user data and later discovers a new and valuable use for it. Under the current system, your company would need to seek approval from every user before adopting the data for this new purpose. While the intent of the legislation is sensible, its application in a big-data world may greatly hinder the benefits that could be realized.

Second, the greater details of big data allow users to be re-identified from anonymized data, potentially revealing sensitive information in the process. For example, in 2006, AOL released a mountain of old, anonymized search terms in the hope that researchers could find interesting insights from the data. Within days, the New York Times had successfully identified one of the users as Thelma Arnold, a 62-year-old widow from Lilburn, Georgia.

The current tools, either legal or technical, are already proving to be ineffective, and as we move further down the big-data road they may become obsolete. More suitable alternatives need to be considered.

Current privacy laws and anonymization methods are ineffective and inefficient when applied to Big Data.

Big data facilitates the prediction of criminal behavior, but we must never judge someone before they have actually committed a crime.

The movie Minority Report depicts a society where predictions have become so accurate that the police arrest the would-be criminal before he or she has a chance to commit the crime. People are imprisoned not for what they have done but for what they are foreseen to do.

Although the movie is science fiction, predictions of human behavior are already used to guide certain decisions in society. As an example, parole boards in more than half of all US states use data-analysis-based predictions of a prisoner’s chance of re-offending when deciding a prisoner’s fate.

The police department in the United States is increasingly turning to “predictive policing” in order to allocate sparse resources. They use profiling – selecting individuals, groups and neighborhoods for additional scrutiny – based on characteristics seen as predictors of crime; for example, poverty, unemployment and drug-usage. Similar profiling measures are employed heavily in national security.

Yet, if misused, such methods can lead to problems of discrimination and “guilt by association.” How would you feel about being arrested on suspicion of terrorism based purely on your ethnicity, acquaintances and background?

While the additional level of detail available through big data may allow us to minimize these problems by targeting individuals rather than groups, this profiling trend is dangerous. Following this trend to its natural conclusion leads us to a world where we deny people their free will – where suspects are apprehended, patients are denied treatment or employees are dismissed – because of what they are predicted to do, not what they have done.

We have already taken tentative steps down the road of using predictions to inform decisions in the realm of law and order. If we take this trend to its extreme, we deny individuals the possibility of moral choice, something we need to guard against.

Big data facilitates the prediction of criminal behavior, but we must never judge someone before they have actually committed a crime.

Being overly data-driven can be perilous: we may be measuring the wrong thing, incentivizing the wrong behavior or relying on inaccurate data.

As our ability to collect and analyze data has developed, we have increasingly tried to use data to improve many aspects of life. However, this ability does come with certain potential pitfalls.

First of all, quantifying life can lead us to measure something that does not really capture the information we intended it to. Consider the introduction of standardized tests in education. Do a student’s standardized test scores truly reflect the range of qualities we expect education to provide?

Second, misuse of data can lead us to incentivize behavior we never intended to. Standardized tests also demonstrate this effect, as their importance has made teachers and students focus on improving test scores and not on the overall quality of education.

Finally, being overly data-driven can be problematic because we run the risk of allowing data that is biased or unreliable to shape our actions.

Consider the experience of Robert McNamara, who became the United States’ Secretary of Defense during the escalation of the Vietnam War. He became completely fixated on measuring the enemy body count as an indicator of progress, and shaped the military’s strategy around it, a decision that would later come to haunt him.

Data can be hard to verify in the chaotic conditions of war, and it later became clear that officers had reported unreliable figures. Ironically, they had done so to impress superiors such as McNamara.

With the wealth of detail and insight that big data offers, there is a risk that we could lose perspective and become so fixated on data that we fail to acknowledge its limitations or verify its quality, allowing the data to govern us in ways that create more harm than good.

Being overly data-driven can be perilous: we may be measuring the wrong thing, incentivizing the wrong behavior or relying on inaccurate data.

Want more?
Read or listen to the key ideas
from 7,000+ titles

Key ideas in Big Data

More knowledge in less time

Read or listen

Get the key ideas from nonfiction bestsellers in minutes, not hours.

Find your next read

Get book lists curated by experts and personalized recommendations.

Shortcasts ^New

We’ve teamed up with podcast creators to bring you key insights from podcasts.

What is Big Data about?

Big Data provides an insightful look at why a change to “big data” is a major shift in how we collect, use and think about the data around us. It provides great explanations and examples of how individuals and companies already ahead of the curve are using the tools of big data to create value and profit. Casting an eye forward, the book also outlines the future implications for a big-data society in terms of the risks, opportunities and legal implications.

Big Data Review

Big Data (2013) explores the vast impact of data in our modern world and why it's crucial to understand its implications. Here's why this book is worth reading:

Filled with insightful examples, it demonstrates how big data is shaping industries, politics, and our daily lives.
Combining in-depth research and engaging storytelling, the book presents complex concepts in a relatable and accessible manner.
By examining the potential benefits and risks of big data, it stimulates critical thinking and helps readers navigate the data-driven era we live in.

Who should read Big Data?

Anyone who is interested in learning more about what “big data” is and what it means for society
Anyone who is looking to make a career using big data
Anyone from a company who is looking for opportunities to use the data the company collects

About the Author

Viktor Mayer-Schönberger was on the faculty of Harvard’s Kennedy School for over ten years before taking up the position of professor of Internet Governance and Regulation at Oxford University. He is also the author of Delete: The Virtue of Forgetting in the Digital Age.

Kenneth Cukier is the data editor of the Economist, and writes widely about what is happening in the world of big data. His articles, covering technology, business and economics, have appeared in the New York Times, Foreign Affairs and the Financial Times.

Categories with Big Data

Technology & the Future

Book summaries like Big Data

Weapons of Math Destruction

Cathy O’Neil

How Big Data Increases Inequality and Threatens Democracy

14 min

4.1

Streaming, Sharing, Stealing

Michael D. Smith and Rahul Telang

Big Data and the Future of Entertainment

17 min

3.5

Everybody Lies

Seth Stephens-Davidowitz

Big Data, New Data and What the Internet Can Tell Us About Who We Really Are

18 min

4.0

Architects of Intelligence

Martin Ford

The truth about AI from the people building it

The Fall of Big Data and the Rise of the Blockchain Economy

23 min

3.9

7 Strategies for Wealth & Happiness

Jim Rohn

Power Ideas from America's Foremost Business Philosopher

How Today's Biggest Trends Will Collide and Reshape the Future of Everything

Why We Can’t Live Without Deceit

16 min

4.2

Creativity, Inc.

Ed Catmull with Amy Wallace

Overcoming the Unseen Forces That Stand in the Way of True Inspiration

The Disruptive Economics of Artificial Intelligence

Ten Reasons We’re Wrong About the World – and Why Things Are Better Than You Think

18 min

4.3

Million Dollar Habits

Brian Tracy

Proven Power Practices to Double and Triple Your Income

The New Industrial Revolution

12 min

3.4

Algorithms to Live By

Brian Christian & Tom Griffiths

The Computer Science of Human Decisions

22 min

3.9

The Six Disciplines of Strategic Thinking

Michael D. Watkins

Leading Your Organization Into the Future

20 min

4.2

The Art of Statistics

Get Focused, Stop Procrastinating and Upgrade Your Life

17 min

4.2

Find Out Anything From Anyone, Anytime

James O. Pyle and Maryann Karinch

Secrets of Calculated Questioning From a Veteran Interrogator

How to Find the Work You Were Meant to Do

What the Rich Teach Their Kids about Money – That the Poor and the Middle Class Do Not!

A Common Sense Guide to the Economy

20 min

4.4

Understanding Artificial Intelligence

Nicolas Sabouret

A Straightforward Explanation of AI and Its Possibilities

The Power to Predict Who Will Click, Buy, Lie, Or Die

The 33 Laws of Business and Life

17 min

4.3

Building a Second Brain

Tiago Forte

A Proven Method to Organize Your Digital Life and Unlock Your Creative Potential

A Guide to Healing Intergenerational Trauma

Little Things That Can Change Your Life… And Maybe The World

Successful Habits of Visionary Companies

A Guide to What Never Changes

17 min

4.2

The Intelligence Trap

David Robson

Why Smart People Make Dumb Mistakes

9 min

3.7

Nudge

Richard H. Thaler and Cass R. Sunstein

Improving Decisions about Health, Wealth and Happiness

14 min

4.1

Built to Move

Kelly Starrett & Juliet Starrett

The Ten Essential Habits to Help You Move Freely and Live Fully

The Power of Knowing What You Don't Know

What We Should Know about the People We Don’t Know

21 min

3.9

When the Body Says No

Gabor Maté

The Cost of Hidden Stress

The Bulletproof Plan to Age Backward and Maybe Even Live Forever

The 12 Levers of Success

Unleash the Power of Your Microbiome to Reverse Disease and Transform Your Mental, Physical, and Emotional Health

5 min

3.2

The Purpose Driven Life

Rick Warren

What on Earth Am I Here For?

11 min

3.6

The Four Agreements

Don Miguel Ruiz and Janet Mills

A Practical Guide to Personal Freedom. A Toltec Wisdom Book

Explore the Evolution of Inequality and a Path to a Better Future

9 min

3.1

Leadership Two Words at a Time

Bill Treasurer

Simple Truths for Leading Complicated People

Ten Easy Rules to Make Sense of Statistics

Cybersecurity Advice from the Best Hackers in the World

Fire Your Boss, Do What You Love and Work Better to Live More

Your Guide to Superior Management Effectiveness

An uncommon guide to rapid fat-loss, incredible sex and becoming superhuman

19 min

3.3

Walmart

Natalie Berg and Bryan Roberts

Key Insights and Practical Lessons from the World's Largest Retailer

17 min

4.5

People ❤️ Blinkist

Sven O.

It's highly addictive to get core insights on personally relevant topics without repetition or triviality. Added to that the apps ability to suggest kindred interests opens up a foundation of knowledge.

Thi Viet Quynh N.

Great app. Good selection of book summaries you can read or listen to while commuting. Instead of scrolling through your social media news feed, this is a much better way to spend your spare time in my opinion.

Jonathan A.

Life changing. The concept of being able to grasp a book's main point in such a short time truly opens multiple opportunities to grow every area of your life at a faster rate.

Renee D.

Great app. Addicting. Perfect for wait times, morning coffee, evening before bed. Extremely well written, thorough, easy to use.