Why I Am Not (As Much Of) A Doomer (As Some People)
Read the original on Astral Codex Ten →
Summary
Scott's canonical statement of his ~33% p(doom) and the optimist case. Core framing: will a world-killer AI (smart enough to deploy superweapons under hostile conditions) arrive before or after pseudo-aligned AIs (smart enough to stop it under ideal conditions)? Then the sleeper-agent / USSR-fifth-columnist analogy for motivational bugs, and six optimist/pessimist cruxes (coherence, AI-AI cooperation/acausal bargaining, solve-vs-check asymmetry, superweapon difficulty, slow-vs-fast takeoff, catching a sleeper agent). Clear, influential, Excellent-tier.
Why this score
Quality 76 · Excellent. An unusually lucid synthesis of the AI-risk debate from Scott's vantage - the 'before or after' framing crystallizes the whole optimist case, the sleeper-agent/USSR-fifth-columnist analogy is a memorable intuition pump (prescient re the later deceptive-alignment 'Sleeper Agents' work), and the six-crux decomposition genuinely clarifies where optimists and pessimists diverge. A reference people point to for 'Scott's case for ~33%.' Excellent floor.
Claude’s paradigm shift 54 · Moderate. Moderate-high - crystallizes rather than originates: the cruxes build on existing alignment discourse (Christiano, Yudkowsky, mesa-optimizers, deceptive alignment), but the 'before/after' framing and sleeper-agent packaging meaningfully sharpened how informed readers hold the debate.
Real-world impact 3 · Moderate. 3 - influential within the AI-safety subculture; the optimist-case framing and sleeper-agent intuition pump are vocabulary the alignment community uses.