Fooled by Randomness summary

The basis of all empirical science is a process called induction: we infer things about the nature of the world based on our observations. Thus from seeing hundreds of white swans, we might infer (mistakenly) that all swans are white.

Unfortunately, this approach carries an inherent problem, illustrated by the famous example of black swans as stated by philosopher John Stuart Mill: “No amount of observations of white swans can allow the inference that all swans are white, but the observation of a single black swan is sufficient to refute that conclusion”.

The popular belief is that evolution always means survival of the fittest. In fact it merely means that on average, the fittest organisms will survive. A few lucky unfit organisms will usually also endure, at least in the short term.

The same is true for many things in life. Consider the common keyboard: how did this bizarre layout of keys (known as QWERTY) end up being the near-universal standard for typing?

Humans are ill-equipped to handle the probabilistic reasoning required by today’s high-information environment. Despite what we may believe, our mind is not a sophisticated thinking-machine, but rather a patchwork of rules and shortcuts called heuristics.

These heuristics have evolved to help us make quick decisions when needed instead of contemplating needlessly: If you encounter a tiger in the jungle, it may be wise just to run and not ponder the details of the situation.

Some researchers believe emotions are the true shortcuts in our decision-making process, the “lubricants of reason”. Without emotions to give us that little irrational nudge, we would agonize endlessly over the slightest decisions.

Consider the example known as Buridan’s donkey: A donkey that is equally hungry and thirsty stands equidistant between food and water. If it were purely rationally optimizing what to do, in theory it would die of both hunger and thirst, unable to decide which to pursue first. A little randomness helps it make up its mind, just like you might use the flip of a coin to help resolve an impasse. Emotions are fundamentally irrational precisely to stop us from temporizing.

Learning from history does not come naturally to humans.

Even after multiple “completely unexpected” stock market crashes, somehow many traders still believe the next crash will not happen or will be spotted by them in advance. This is due to hindsight bias: in retrospect, earlier events always seem more predictable than they really were at the time.

When hedge funds report losses, they often refer to large and “unexpected” events, factors that their risk management models were completely unprepared for. These statements ignore the fact that things which have never happened before actually happen all the time, and are always unexpected.

An unexpected event can dwarf expected events in magnitude, which is why they often show up as outliers in data and are ignored when conducting risk analysis.

Although being fooled by randomness in the stock market is often lethal for your portfolio, there are instances where randomness can truly be enjoyable.

While generally it is sensible to be hyper-rational when dealing with science and finances, one can gladly be fooled by randomness when it comes to art and poetry. A scientist using elaborate prose as noise to disguise the fact that he has nothing of value to say is infuriating. A poet on the other hand, using similarly elaborate prose can be sublimely beautiful. Aesthetic forms appeal to our brains whether they are formed by randomness or not.

People who obsessively read the Wall Street Journal every day expend a lot of effort for very little reward.

Today’s information environment is so cluttered with useless news, the cost of wading through all of them by far exceeds the cost of missing those few truly valuable items. It is akin to searching for a needle in a haystack for some 30 hours each month.