Big Data Book Summary - Big Data Book explained in key points

Listen to the Intro

00:00

Big Data summary

Name: Big Data
Rating: 4.4 (103 reviews)
Author: Viktor Mayer-Schönberger and Kenneth Cukier

Viktor Mayer-Schönberger and Kenneth Cukier

A Revolution That Will Transform How We Live, Work and Think

4.4 (103 ratings)

24 mins

Topics

Artificial Intelligence Big Data Mathematics

Table of Contents

Big Data
Summary of 12 key ideas

Audio & text in the Blinkist app

Key idea 1 of 12

Big data provides insights we could not discover by analyzing data on a smaller scale.

Prior to the advent of computers, collecting and recording information was an arduous and time-consuming task. To put this in context, consider the information needed to complete a census of the population. Under the US Constitution, a census is required every decade, yet the 1880 census took over eight years to complete and publish. This meant the information had become obsolete before it was even made available.

But that was then. Now – with the invention of computers, digitization and the Internet – the picture has changed considerably. Information can be collected passively (or with much less effort) and at greater speeds, and the cost of storage is increasingly economical. This has brought us to the advent of the big-data era.

Although there is no formal definition, “big data” refers to both the data being captured on a much greater scale than previously possible, and the opportunities that data-sets of this size offer in terms of valuable insights discovered through analysis.

In 2009, Google provided a great example of the possibilities of big data when they published a research paper showing how they could analyze users’ search terms to predict the outbreak of flu and monitor its spread. They compared historical search-term data with data on the spread of flu in time and space from 2007 and 2008, and discovered 45 search terms that could be used in a formula to predict the spread of flu – a prediction which correlated strongly with official figures.

Only weeks after the paper was published, the outbreak of the new deadly strain of flu, H1N1, hit the headlines. Google’s system was pressed into action and provided indicators that proved to be more useful and timely than government statistics in delivering valuable information to public health officials.

Big data provides insights we could not discover by analyzing data on a smaller scale.

Data is increasingly being collected and put to use in all aspects of our lives, from the size of our bums to the way we walk.

With the rise of Internet companies such as Facebook and Twitter, and the popularity of smart devices, we have become familiar with things such as our relationship statuses, comments, preferences and location being stored as data that can then be analyzed. This trend is part of the process of datafication – capturing information about the world in the form of data.

Because we can discover valuable insights from such data, we are likely to see the trend continue, with innovations in capturing data from sources we had not previously thought of as information.

Big data frees us from the limitations of using small samples of data to represent whole populations.

Before our current technological age of the Internet and computing, information was much harder to collect and record. Accordingly, we could collect only very limited amounts of information and then try to interpret them as best we could.

For example, say you wanted to conduct a telephone survey of voters for an upcoming local election. Clearly, it would be impossible to contact the entire population, so you call a few hundred people and assume that their answers reflect the whole population’s opinions. This approach is called sampling: you take a sample of all the data, and hope it is representative of the whole.

Vast sets of messier data can be more useful than smaller, more accurate ones.

While trying to develop a language-translation program in the 1980s, the engineers at IBM had a novel idea. They decided to dispense with the standard method of using grammar rules and dictionaries and instead allowed the computer to rely on statistical probabilities to calculate which word or phrase was called for, based on samples of translated text they fed into it.

IBM’s engineers decided to use a large but limited sample of high-quality data, using three million sentence pairs from official translations of Canadian parliamentary documents. Despite promising early results, the project failed. Although the system could provide reliable translations for the most frequently used words and phrases, it was less reliable for those that occur infrequently. The system failed, despite the quality of the data. The problem was the quantity – there was simply not enough of it.

Big data does not tell us why two things are related, just that they are, but even this is often good enough.

When buying a used car, what criteria do you look for to ensure you don’t end up buying a clunker? You might consider things such as age, mileage, country of origin, and the make and model – these criteria all seem pretty logical. But would you take the color of the car’s paintwork into account?

In 2012, contestants in a data-analysis competition were given a similar task, and their correlation analysis revealed a surprising finding: cars painted orange were half as likely to have defects as the average car.

Although data is generally collected for a specific purpose, there are often secondary applications that hold even greater value.

When companies collect data, they generally have a specific purpose in mind: stores collect sales data for their financial accounting, factories monitor their output to track productivity, and websites track mouse movements over their pages to optimize their customers’ user experience. Consider, too, the interbank payment system Swift, which collects data on the billions of financial transactions it processes across the globe in order to provide accurate customer records.

But, increasingly, companies are finding secondary uses for the data they have collected that are sometimes even more valuable than the original use. For example, Swift discovered that its payment data correlates well with global economic activity. As a result, the company now offers highly accurate GDP forecasts derived from their transaction data.

Anyone can spot new opportunities to create value from the data around them – you just need the right mindset.

Owning vast amounts of data is not much use if you don’t know what to do with it. Equally, having the skills and tools to analyze data is of little use if you don’t own any data or don’t know where to get it.

Nevertheless, there are people who have neither of these things, yet still manage to find a niche for themselves in the big-data world.

Combining sets of data can create greater value than the individual parts.

As anyone who has ever played the board game Clue (i.e. Cluedo) will know, pieces of information may have little value alone but when they are combined with others they can tell you so much more. This is also true for data-sets: sometimes their value becomes apparent only when combined with other data-sets. Trends can then be found in the newly combined data that were not discoverable from the individual data-sets alone.

For example, in 2011 a Danish research group demonstrated this phenomenon. In one of the largest studies of its kind, they combined mobile phone user data with cancer patient records. This meant that they were able to check not only for a link between mobile phone use and cancer, but also whether greater mobile phone use increased the risk.

Online services such as Facebook record everything we do on their sites and use this data to enhance the service they offer.

Most businesses use some form of customer feedback to improve the products and services they deliver. Traditionally though, it has been both time-consuming and difficult to collect feedback in large enough volumes to be meaningful.

In the age of big data and the Internet, information can be collected instantly and with much less effort than before, often even completely passively. Smart companies are already tracking everything we do online, including where we move the mouse and how long we hover over items. This information is referred to as data exhaust and is used to optimize the fine details of products, such as the size and placement of buttons.

Current privacy laws and anonymization methods are ineffective and inefficient when applied to Big Data.

It’s hard to spend any amount of time online these days without being presented with a lengthy user agreement at some point. But, be honest, do you actually read through them before agreeing to the terms?

Current privacy laws require that we are informed about what information is being collected and for what purpose, and that we then give consent, which is why we are bombarded with such requests. If the company then wants to share the data it collects, it uses anonymization – the stripping out of any personal details to preserve the privacy of the individuals – before publishing the data.

Big data facilitates the prediction of criminal behavior, but we must never judge someone before they have actually committed a crime.

The movie Minority Report depicts a society where predictions have become so accurate that the police arrest the would-be criminal before he or she has a chance to commit the crime. People are imprisoned not for what they have done but for what they are foreseen to do.

Although the movie is science fiction, predictions of human behavior are already used to guide certain decisions in society. As an example, parole boards in more than half of all US states use data-analysis-based predictions of a prisoner’s chance of re-offending when deciding a prisoner’s fate.

Being overly data-driven can be perilous: we may be measuring the wrong thing, incentivizing the wrong behavior or relying on inaccurate data.

As our ability to collect and analyze data has developed, we have increasingly tried to use data to improve many aspects of life. However, this ability does come with certain potential pitfalls.

First of all, quantifying life can lead us to measure something that does not really capture the information we intended it to. Consider the introduction of standardized tests in education. Do a student’s standardized test scores truly reflect the range of qualities we expect education to provide?

Want to see all full key ideas from Big Data?

Key ideas in Big Data

More knowledge in less time

Read or listen

Get the key ideas from nonfiction bestsellers in minutes, not hours.

Find your next read

Get book lists curated by experts and personalized recommendations.

Shortcasts ^New

We’ve teamed up with podcast creators to bring you key insights from podcasts.

What is Big Data about?

Big Data provides an insightful look at why a change to “big data” is a major shift in how we collect, use and think about the data around us. It provides great explanations and examples of how individuals and companies already ahead of the curve are using the tools of big data to create value and profit. Casting an eye forward, the book also outlines the future implications for a big-data society in terms of the risks, opportunities and legal implications.

Who should read Big Data?

Anyone who is interested in learning more about what “big data” is and what it means for society
Anyone who is looking to make a career using big data
Anyone from a company who is looking for opportunities to use the data the company collects

About the Author

Viktor Mayer-Schönberger was on the faculty of Harvard’s Kennedy School for over ten years before taking up the position of professor of Internet Governance and Regulation at Oxford University. He is also the author of Delete: The Virtue of Forgetting in the Digital Age.

Kenneth Cukier is the data editor of the Economist, and writes widely about what is happening in the world of big data. His articles, covering technology, business and economics, have appeared in the New York Times, Foreign Affairs and the Financial Times.

Categories with Big Data

Technology & the Future

Books like Big Data

Everybody Lies

Seth Stephens-Davidowitz

Big Data, New Data and What the Internet Can Tell Us About Who We Really Are

18 min

4.1

Streaming, Sharing, Stealing

Michael D. Smith and Rahul Telang

Big Data and the Future of Entertainment

17 min

3.6

Weapons of Math Destruction

Cathy O’Neil

How Big Data Increases Inequality and Threatens Democracy

14 min

4.1

The Fourth Industrial Revolution

Klaus Schwab

A new transformation in technology and industry

Ten Easy Rules to Make Sense of Statistics

Quit Your Day Job, Start Your Own Business, and Live Happily Ever After

19 min

3.0

Thinking, Fast and Slow

Daniel Kahneman

Intuition or deliberation? Where you can (and can't) trust your brain

28 min

4.5

Emotional Intelligence

The Patient Will See You Now

Eric Topol

The Future of Medicine is in Your Hands

15 min

4.1

Impromptu

Reid Hoffman with GPT-4

Amplifying Our Humanity Through AI

The New Industrial Revolution

12 min

3.9

Numbers Rule Your World

Kaiser Fung

The Hidden Influence of Probability and Statistics on Everything You Do

The New Psychology of Success

A Brief History of Tomorrow

The Hidden History of the Billionaires Behind the Rise of the Radical Right

19 min

3.9

The Art of Statistics

What We Should Know about the People We Don’t Know

The Fall of Big Data and the Rise of the Blockchain Economy

23 min

4.0

It’s Not the Size of the Data

A Simple Plan to Stop People-Pleasing, Reclaim Your Boundaries, and Say Yes to the Life You Want

Work, Wealth and Welfare in the Robot Age

22 min

4.0

American Prometheus

Kai Bird & Martin J Sherwin

The Triumph and Tragedy of J. Robert Oppenheimer

16 min

4.2

The Future of Humanity

Michio Kaku

Terraforming Mars, Interstellar Travel, Immortality, and Our Destiny Beyond Earth

20 min

4.4

Understanding Artificial Intelligence

Nicolas Sabouret

A Straightforward Explanation of AI and Its Possibilities

The Tiny Clues That Uncover Huge Trends

Listening to Ignite the Human Mind

How Google, Bono, and the Gates Foundation Rock the World with OKRs

14 Management Principles from the World's Greatest Manufacturer

15 min

4.5

Capital in the Twenty-First Century

Thomas Piketty

Groundbreaking Research That Unravels Economic Disparity in Our World Today

An Essential Guide to Life

Understanding the 12 Technological Forces That Will Shape Our Future

A Financial History of the World

366 Meditations on Power, Seduction, Mastery, Strategy, and Human Nature

Why You Can't Pay Attention – and How to Think Deeply Again

25 min

4.5

21 Lessons for the 21st Century

Yuval Noah Harari

Future proof yourself against the 21st Century

How the Quantum Computer Revolution Will Change Everything

18 min

4.4

Tools and Weapons

Brad Smith and Carol Ann Browne

The Promise and the Peril of the Digital Age

26 min

4.1

Rewired

Eric Lamarre & Kate Smaje & Rodney Zemmel

The McKinsey Guide to Outcompeting in the Age of Digital and AI

The Art of Skepticism in a Data-Driven World

Achieving Success at Work and in Life One Conversation at a Time

17 min

3.6

Logistics and Supply Chain Innovation

John Manners-Bell

A Practical Guide to Disruptive Technologies and New Business Models

Being Human in the Age of Artificial Intelligence

101 Simple Truths for a Better Life

Escape 9–5, Live Anywhere, and Join the New Rich

28 min

4.2

Make Time

Jake Knapp and John Zeratsky

How to Focus on What Matters Every Day

35 min

4.5

NFTs Are a Scam / NFTs are the Future

Bobby Hundreds

The Early Years: 2020-2023

15 min

3.9

Human/Machine

Daniel Newman and Olivier Blanchard

The Future of Our Partnership With Machines

22 min

4.1

The Age of Surveillance Capitalism

Shoshana Zuboff

The Fight for a Human Future at the New Frontier of Power

Developing Mental Agility and Resilience to Thrive in Uncertainty

16 min

4.4

The Coaching Habit

Michael Bungay Stanier

Say Less, Ask More & Change the Way You Lead Forever

13 min

4.5

People ❤️ Blinkist

Sven O.

It's highly addictive to get core insights on personally relevant topics without repetition or triviality. Added to that the apps ability to suggest kindred interests opens up a foundation of knowledge.

Thi Viet Quynh N.

Great app. Good selection of book summaries you can read or listen to while commuting. Instead of scrolling through your social media news feed, this is a much better way to spend your spare time in my opinion.

Jonathan A.

Life changing. The concept of being able to grasp a book's main point in such a short time truly opens multiple opportunities to grow every area of your life at a faster rate.

Renee D.

Great app. Addicting. Perfect for wait times, morning coffee, evening before bed. Extremely well written, thorough, easy to use.