D·10
Decisions & Judgment
Judgments Vary More Than Anyone Believes
Two qualified people judging the same case — or the same person on two days — disagree wildly. That scatter costs more than bias does.
- Evidence
- solid
- Topic
- Decisions
- Things to do
- 3
- Read
- 1 min
solidHow strong is the evidence?
Good evidence behind it, but fewer people have checked it independently.
- Kahneman, Sibony & Sunstein, Noise (2021)
- structured-interview literature, Schmidt & Hunter
Why it works
Bias is a consistent offset you can correct for. Noise is random and invisible from inside, so nobody corrects for it.
What to do about it
- Collect independent judgments before anyone discusses. Discussion first destroys independence and creates false confidence.
- Average the independent estimates. This one move beats almost any individual improvement.
- Break judgments into scored components — structured beats holistic in hiring, lending and assessment every time.
Where people go wrong
Going straight to a group discussion because it feels collaborative. The first confident voice sets the answer.
Read next
- Start From the Base RateAsk what usually happens to cases like this one before you consider what is special about this one.proven
- The Planning FallacyYour estimate of how long this will take is an estimate of the best case, and you know it is not the best case.proven
- The Pre-MortemBefore committing, assume it is eighteen months later and the project has failed. Write the story of why.solid
- Sunk Costs Are Not CostsMoney and years already spent are gone whatever you now decide. The only question is forward value.proven