Scott Alexander, curated
← Back to curation

2018 Predictions: Calibration Results

Quality
58
Solid
Claude Shift
35
Slight
RWI
2
of 10

Follows up on

Predictions For 2018 — Predictions · Feb 2018

Summary

The scoring of Scott's 100 calibrated predictions for 2018, with a calibration chart and a breakdown by probability bucket -- and, unlike the bare predictions list, a genuinely useful reflective payload. His 50% predictions came out 6-right/16-wrong ('more wrong than should be statistically possible'), and his 95% bucket was poorly calibrated at 75%. The real insight is about CORRELATED predictions: two black swans (the crypto crash and an unexpected breakup that forced a move) each knocked out a whole cluster of correlated bets at once -- e.g. spending his savings on the move made the planned 'financial planning' prediction moot -- so a few correlated shocks dominated his miss rate, with the lesson 'don't make too many correlated predictions.' A clean, honest demonstration of personal calibration and a good practical point about correlation risk in forecasting.

Why this score

Quality 58 · Solid. Solid (58): a competent, honest calibration-results post elevated above a bare data dump by a real reflective insight (correlated predictions amplifying black-swan misses). High-Solid rather than Strong because it is primarily a self-scoring exercise with limited reach beyond the practice itself.

Claude’s paradigm shift 35 · Slight. Slight (35): models the valuable public-calibration practice and adds the correlated-predictions observation, but the practice and the insight are instances of established forecasting/calibration ideas rather than novel ones.

Real-world impact 2 · Minor. A calibration-results post elevated by a real reflective insight (correlated predictions amplify black-swan misses); a self-scoring exercise within the forecasting discourse, no material-world reach → RWI 2.