Superforecasting summary

As we hinted at in the previous chapter, it’s not because superforecasters have genius-level IQs that they’re able to outperform experts. It’s the techniques that they use, including specific evidence-based reasoning methods.

Superforecasters break down difficult questions logically rather than rely solely on intuition. For example, in 2012, when superforecasters were asked to predict whether evidence of poisoning would be found in Yasser Arafat’s remains, they avoided intuition-based guesses about Israel’s role. Instead, they broke the question into parts. First, they researched whether polonium, the suspected poison, could be detected so many years after his death. It was only after confirming the technical feasibility that they considered political motivation – including other political actors such as Palestinian rivals, or factions wanting to cast suspicion on Israel. By decomposing the prediction into sub-questions, the superforecasters avoided bias and built an evidence-based framework for objective analysis.

In our age of Big Data, extracting meaningful insights can feel like a magical art. Lionel Levine, a mathematics professor at Cornell, stands as a “magician” at the forefront of this domain. Surprisingly, though, Levine and many other superforecasters don't just lean on intricate math. They blend this with keen judgment, proving that forecasting isn't merely a game of numbers.

This interplay between judgment and numbers was put to the test in the high-stakes hunt for Osama bin Laden. The film Zero Dark Thirty depicts officials grappling with varying probability estimates on bin Laden's location. While the movie portrays a desire for absolute certainty, real-life CIA Director Leon Panetta navigated the uncertainty, appreciating the spectrum of judgments rather than seeking absolute assurance. This real-world scenario contrasts with the movie, emphasizing the value of a holistic approach to prediction.

Superforecasting is about making highly precise predictions regarding future events. It's not a step-by-step method, but does involve a systematic approach. First, forecasters break down questions into smaller parts. They then distinguish between what they already know and what remains uncertain. This means they don't just accept things at face value; they really dig into the details. They also take two main viewpoints – one that sees the issue as part of a broader category and another that considers its unique aspects. After gathering different perspectives, they merge them into one clear prediction. The key here is that once they make a prediction, they don't just forget about it. They revisit and revise it as new information comes in.

Remember our superforecaster from earlier, Bill Flack? He made predictions about whether polonium would be found in Yasser Arafat’s remains. As time went on and more news came out, he kept refining his prediction to be more accurate. Another forecaster, Devyn Duffy, is exceptionally good at this. He constantly monitors the news and tweaks his forecasts based on what he learns. What's interesting is that these superforecasters adjust their predictions more often than the average person, which seems to be a big part of their success.

Mary Simpson, despite her extensive background in economics and a successful career, missed the signs leading up to the 2007 financial crisis. This personal failure, which significantly affected her retirement savings, motivated her to refine her forecasting abilities. Joining the Good Judgment Project, she became a superforecaster, exemplifying a “growth mindset.” This concept, first articulated by psychologist Carol Dweck, suggests that abilities are developed through effort. Contrarily, many possess a “fixed mindset,” believing that their capacities are unchangeable. Dweck's research shows that those with a growth mindset are more adaptive and learn from challenges, while fixed-mindset individuals often avoid difficulties.

John Maynard Keynes, known for his macroeconomic theories, is another example of this growth mindset in action. As an investor, Keynes experienced significant losses, but continuously adapted and learned from his failures. Rather than seeing failures as insurmountable, he viewed them as opportunities to enhance his strategies. This mindset, the essence of learning from your mistakes and persistently striving for improvement, is universal. From children learning basic skills to adults acquiring new ones, the cycle of “try, fail, analyze, adjust, try again” is crucial to human growth and advancement across various fields, from piloting jets to performing surgery.