Superforecasting summary

Measuring forecasts might seem like a no-brainer: collect the forecasts, judge their accuracy, do the calculations, et voilà! But it isn’t that easy at all.

To ascertain the accuracy of a forecast, you must first understand the meaning of the original forecast. For example, in April, 2007, media outlets reported that Microsoft’s CEO, Steve Ballmer, had made a prediction: the iPhone would fail to win a significant market share. Considering Apple’s size, Ballmer’s forecast seemed ridiculous, and people literally laughed at him. Others highlighted the fact that Apple controlled 42 percent of the US smartphone market, an undeniably significant number.

So how do we avoid horrendous errors like what happened with the WMDs? Clearly, more accuracy is needed in our forecasts. Let’s look at some ways of attaining this.

The best way is to keep score.

Are all superforecasters gifted thinkers with access to top secret information? Not really. So how do they make such accurate predictions about the future?

A superforecaster tackles a question by breaking down seemingly impossible problems into bite-sized sub-problems. This is known as Fermi-style thinking.

As every situation is unique, you should avoid jumping the gun and judging a case too quickly. The best way to approach any question is to take an outside view, which means finding out what the base rate is. But what exactly does that mean?

To offer an example, picture an Italian family who lives in a modest house in the United States. The father works as a bookkeeper and the mother has a part-time position at a daycare center. Their household consists of them, their child and the child’s grandmother.

We’ve seen how superforecasters get the process going, but once you’ve made your initial forecast, you can’t just wait and see if you were right. You’ve got to update and modify your judgment based on any new piece of information.

Remember Bill Flack? After he forecasted a 60-percent chance of polonium being detected in Yasser Arafat’s corpse, he kept an eye on the news and updated his forecast whenever he deemed fit.

Perhaps you’ve heard of the term groupthink. Psychologist Irving Janis, who coined the term, hypothesized that people in small groups build team spirit by unconsciously creating shared illusions that interfere with critical thinking. This interference comes from people shying away from confrontation and simply agreeing with one another.

But not conforming is a source of real value. Independent expression and thought are tremendous assets in any team.