Why We’re Imperfect Judges

Two people, convicted of identical crimes, might receive identical sentences. But their punishments may also differ significantly—from probation for one person to years in jail for the other—and might be influenced by factors like the judge’s mood, the weather, or even how the local football team fared in its most recent game.

A justice system that doles out scattered sentences for similar crimes is one example of “noise,” or unwanted variability in judgments that should, in theory, be identical. In Noise: A Flaw in Human Judgment, researchers Daniel Kahneman, Olivier Sibony, and Cass Sunstein explore the consequences of noise—and argue that reducing it is an oft-overlooked step toward fairer decisions in myriad domains. PT asked Kahneman what we know about noise and what can be done to dampen it.

Noise is everywhere.

Noise isn’t present only in the legal system, Kahneman emphasizes. It also arises when doctors make diagnoses, when social workers decide whether to place children in foster care, and in countless other decisions in which there is often no “right” answer. “Wherever there is judgment, there is noise,” he says. “And more of it than you think.”

Noise has many sources.

Few are immune to cognitive biases, from confirmation bias to the halo effect. Taken systematically, such biases are a key source of noise. Since they can pull each individual decision in a different direction, the end results are often scattered. But noise has causes other than bias. Individual differences in values, for example, may steer some judges toward leniency and others toward harshness.

Noise is costly.

The human cost of a noisy justice system is perhaps incalculable. But for organizations, noise often comes with a measurable economic cost. To illustrate the point, the authors asked underwriters at an insurance company to assign premiums to the same case. Their answers fell across a wide range, varying five times more than executives predicted. Some agents set low premiums, potentially leaving money on the table; others aimed high, which could push business to a competitor. Either way, Kahneman notes, the company loses. “With that much noise, there’s a serious question as to whether the organization needs such employees at all.”

Noise is fixable.

One way to reduce noise is to break complex cases into smaller, fact-based analyses—for example, assessing job candidates on their skill set, problem-solving ability, and interpersonal strengths separately—before making an overarching decision. “An intuitive judgment made after evaluating each element separately is often more valid than one made at the beginning,” Kahneman says. “The point is not to avoid intuition altogether, but to delay it.”