10 Forecasting & Overconfidence

Meikle et al. (2016) Overconfidence at Work

Forecasting

Predicting the probability of future events

Overconfidence

Confidence exceeds what objective evidence supports

Overestimation

Overestimating own ability or performance

Overplacement

Believing you perform better than others

Overprecision

Being too certain that your judgments are correct

Hard-easy effect

Overestimating performance on difficult tasks but sometimes underestimating it on easy tasks

Hindsight bias

Overestimating how well you predicted an event after it happened

Dunning-Kruger effect

Overestimation tends to be highest among lower performers

Confidence interval

Range expected to contain the true value with a stated probability

Calibration

Confidence matches actual accuracy

Overconfidence vs. underconfidence

Confidence is higher vs. lower than actual accuracy

Brier score

Forecasting accuracy measure; ranges from 0 (best) to 2 (worst)

Epistemic humility

Recognising the limits of one's knowledge

Known unknowns

Information gaps you are aware of

Illusion of explanatory depth

Overestimating how well you understand how something works

Pre-mortem

Assuming a prediction failed and identifying possible reasons

Prospective hindsight

Imagining a future failure to generate possible explanations in advance

Reference class forecasting

Predicting outcomes using results from similar past cases

Inside vs. outside view

Inside: focus on the specific case; outside: use outcomes of similar cases

Reference class problem

Difficulty choosing the appropriate group of past cases

Equivalent-bet test

Comparing a forecast with an equivalent gamble to improve probability estimates

Superforecasters

Highly accurate forecasters with strong reflection, open-mindedness, and motivation to update

Feedback

Repeated accuracy feedback can improve forecasting calibration

Organizational overconfidence

Organizational structures can reduce or reinforce overconfidence through feedback, incentives, and accountability

Overconfidence remedies

Use independent monitoring, safety margins, calibrated forecasts, and reference class data