Notes From The Asilomar Conference On Beneficial AI

Scott's report from the 2017 Asilomar Conference on Beneficial AI. Ten impressions: (1) AI-safety's 'coming-out party' / breaking the taboo; (2) economists newly convinced technological unemployment is real (Acemoglu's robots paper, ~7 jobs lost per robot); (3) the 'teach coal miners to code' optimism; (4) inverse reinforcement learning for value alignment; (5) reward-uncertainty AIs that wouldn't resist being shut off (corrigibility); (6) AlphaGo showing human Go masters were 'completely wrong', as a test of how close human communities get to optimal; (7) PETRL; (8) the interpretability/transparency problem and the 'treacherous turn'; (9) AI arms races needing international coordination (the missing China/Russia contacts); (10) geniuses hilariously debating generic ethics platitudes over lunch.

Quality 69 · Strong. 69 — high-Strong. A substantive, engaging snapshot of AI safety circa 2017 — the corrigibility/reward-uncertainty idea, the AlphaGo-as-test-of-human-rationality point, and the arms-race coordination problem are real and well-explained. A notes/report format rather than a developed argument, which holds it high-Strong.

Claude’s paradigm shift 46 · Moderate. 46 — Moderate. Reports others' ideas from the conference; Scott's framing adds value but the substance isn't his novel frame.

Real-world impact 3 · Moderate. 3 — a notable AI-safety-community document popularizing the Asilomar themes; influential within that sphere.