Should The Future Be Human?
Read the original on Astral Codex Ten →
Summary
Responds to the Page-vs-Musk 'speciesist' clash and e/acc's 'no affinity for biological humans': sketches an Optimistic (recognizable AI successors that make real art/science) vs Pessimistic (non-conscious paperclip maximizer) future, then lists what would have to go right to cede the future gladly - consciousness (maybe a rare subset of useful computation, cf. Blindsight), individuation, art/science/philosophy as non-guaranteed (music as spandrel), and a firm 'humans won't merge with AI by default' argument (we never merged with prior tools; the chess human+engine window has already closed). Closes with the alien-fleet thought experiment: basic rights (life/liberty/property) trump speciesism worries, and the work to build a trustworthy successor is the same work needed to trust it.
Why this score
Quality 72 · Strong. Strong - original, memorable, useful, widely-cited statement of the 'successorship' question, with several portable framings (the never-merged-with-our-tools argument, the closed cyborg-window, the alien-fleet rights move). Held below Excellent because it's short and largely sharpens/positions Scott's own prior material (Ascended Economy) and existing alignment/consciousness discourse rather than breaking new ground.
Claude’s paradigm shift 50 · Moderate. Moderate - recombines and crisply restates ideas already circulating in 2024 AI discourse (consciousness loss, paperclip maximizers, successor species, e/acc vs EA); the value is clarity and stance, not a new paradigm.
Real-world impact 2 · Minor. Influential-within-discourse AI essay; shaped how some readers frame 'should AI replace us' but no material/institutional effect.