How to Measure Anything summary

In the business world, it’s common to have uncertainties about certain values. Be it the time spent on customer complaints or potential sales increase with a new campaign, predicting these numbers precisely can be challenging.

One way to express these uncertainties is to consider them as a range of possible values –⁠ a concept in statistics known as a confidence interval, or CI. For instance, when you predict that the number of prospects turning into customers will be between three and seven this quarter, with a 90 percent confidence level, this is considered a 90 percent CI. It doesn’t matter whether these estimates are computed through statistical inference or personal experience –⁠ they are still a reflection of your uncertainty.

Calibration methods can be utilized to assess both uncertainties and risks. Regrettably, many organizations use oversimplified risk measurements such as “high,” “medium,” or “low” ratings, which can lead to confusion. Instead, using specific figures, like the percentage chance of a certain loss, is far more effective.

Now let’s talk about a problem many organizations face: How to perform calculations when the available data are in ranges rather than precise points? The solution lies in Monte Carlo simulations. This method, devised by Enrico Fermi, uses a computer to generate large numbers of scenarios based on the probability ranges of different variables. Each scenario randomly generates a specific value for each unknown variable, which is then used to calculate a specific output.

The art of decomposition is a powerful tool in our measurement arsenal, even if it’s not a conventional “measurement” since it doesn’t require new observations. It’s like cracking open a complex question. Here’s the idea in a nutshell: define the problem, break it into measurable elements, quantify these elements, and recombine them to provide a comprehensive measure of the original problem.

Imagine you’re trying to measure a potential productivity improvement. You might be given an estimate that a process or technology could improve productivity by 5 to 40 percent for a set of employees. “5 to 40 percent” – that’s quite a wide range! Technically speaking, the uncertainty stems from approximating several unknown variables. But this is where the magic of decomposition comes in. 

You might have heard of Bayesian statistics. It’s a branch of statistics based on Bayes’ theorem, named after Thomas Bayes, an 18th-century English statistician and minister. This theorem provides a mathematical framework for updating probabilities based on new data. At the heart of Bayesian statistics is the concept of a probability distribution for a parameter of interest. For example, if you’re trying to estimate the average height of men in a certain city, a Bayesian approach would not just give you a single best guess but rather a distribution of plausible values, each assigned a probability.

Now, in the uncertain world of business, the ability to adjust your strategies based on new information is a vital skill. This is where Bayesian analysis comes in. It embodies this principle of adaptability and continuous learning, refining its probability estimates as fresh data enters the picture.

Surveys are also a powerful tool for gathering data about people’s preferences. There are several ways of designing surveys. One example is the Likert scale, where respondents say how they feel about something on a scale from “strongly disagree or dislike” to “strongly agree or like.” Another example is an ordinal survey, where respondents rank several things on a scale –⁠ say, from 1, least preferred, to 8, most preferred.

In what scenarios might quantifying sentiments or subjective assessments come into play? Consider survey respondents stating they strongly agree with opinions like “Retail stores tend to put up Christmas decorations too early.” You might determine what percentage of shoppers agree with this sentiment and how this perception might impact your business decisions.