The Data Detective summary

When the author got his job as a presenter on a BBC radio show, he immediately fell in love with the work. But he was less fond of the morning commute from East to West London. He had to ride a crowded bus and then the tube – a subway train – that was practically bursting at the seams.

Because of those miserable mornings, the author was interested in learning more about just how busy London’s public transport system actually was. He was shocked to find that the average occupancy of a London bus was just 12 people, and on the tube, it was less than 130.

In the late-2010s, the UK seemed to be in the midst of an infant mortality crisis. Rates of early death varied substantially across the country – and at first, it wasn’t clear why.

It turned out that the difference in mortality rates ultimately resulted from differences in definition – specifically, whether a baby born at 22 or 23 weeks should be recorded as a miscarriage or a live birth followed by an early death. In London, those pregnancies were recorded as miscarriages. In the English Midlands, by contrast, they were instead considered live births. The difference was enough to explain the gap in mortality rate at London versus Midlands hospitals.

Alarming headlines blared out from London’s newspapers in April 2018. They all read some variation of: “London’s Murder Rate is Higher Than New York’s for the First Time Ever!”

Setting aside the fact that the definition of “murder” is different in each city, the claim was technically true. There were 14 murders in New York City in February 2018, while in London, there were 15.

Have you heard about the famous jam-tasting experiment run by psychologists Sheena Iyengar and Mark Lepper? In it, the researchers set up a jam-tasting stall that sometimes offered 24 varieties of jam and other times offered six. After customers tasted the jam, they were given a voucher to buy it at a discount. Ultimately, the bigger display attracted more customers – but only three percent of them ended up buying the jam. Meanwhile, 30 percent of customers bought the jam at the smaller display. The psychologists concluded that people respond better to fewer choices and worse to more choices.

Since its publication, the study has become a sensation. You can find the results plastered everywhere, from pop-psychology articles to TED Talks. But can we trust them?

How much pressure do people feel to conform to their peers? There’s a large amount of research suggesting that the answer is: a lot.

In the 1950s, psychologist Solomon Asch conducted a study in which he showed subjects two images. One depicted three lines of different lengths, and the other depicted a “reference line.” Their task was simply to identify which of the three lines was the same length as the reference line.

Upon its release in 2009, Google Flu Trends was touted as a revolutionary tool in tracking the spread of seasonal influenza. By counting searches for “flu symptoms” and “pharmacies near me,” Google could accurately estimate new daily flu cases faster than the CDC.

Google Flu Trends was in many ways the herald of a new age: that of “big data” and algorithms. Big data refers to the information we produce when we surf the web, pay with credit cards, or use mobile phones. Algorithms are computer programs often used to find patterns in datasets.

The Congressional Budget Office was established in the US in 1974 to provide Congress with reports on the budgetary costs of policy proposals. As one CBO official described it, the process was like dropping a bill down into a manhole and then having the cost estimates sent back up 20 minutes later –⁠ objective and noncontroversial.

But not every president accepted the CBO’s estimates with grace. The first president to complain was Jimmy Carter, who wanted to improve America’s energy efficiency. But the CBO evaluated Carter’s proposals for doing so and found they wouldn’t work as well as planned. The Carter administration was unhappy because the CBO “wasn’t helping.” But that was precisely the point –⁠ the best government organizations will present statistics accurately, whether or not they make politicians happy.

David McCandless, author of Information is Beautiful, once produced a striking and unforgettable animation called Debtris. Just like the classic computer game Tetris, Debtris showed large colored blocks falling to the bottom of a screen. The size of each block represented the cost of various items, like the UN budget, the estimated cost of the 2003 Iraq war, and Walmart’s revenue.

The catchy music, colorful graphics, and slow presentation of the various comparisons are beautiful to look at. But unfortunately, those elements obscure the many problems in the data used to make the graphic.

Philip Tetlock was a young Canadian-born psychologist who, along with a group of other social scientists, had been given a huge task: prevent nuclear war between the US and the Soviet Union. To do that, Tetlock interviewed countless experts to get their take on all the possibilities about what might happen next and why.

However, Tetlock became frustrated when he found that experts of all stripes were extremely stubborn, constantly refusing to change their minds when given contradictory evidence. Many of them were also relentless in trying to justify incorrect forecasts they’d made in the past. So Tetlock created a devious study that vividly showed just how bad the forecasters were at forecasting.