Failure To Replicate Anti-Vaccine Poll
Read the original on Astral Codex Ten →
Summary
Investigates Steve Kirsch's viral anti-vaccine polls (Pollfish/Rasmussen: ~8.5% say a household member died FROM the COVID vaccine -- more than died of COVID). Scott runs the same question on the ACX survey and gets 0.6% (vs Kirsch's 8.5%), then does the legwork: emails the 38 'yes' respondents, 9 reply, 3 retract as misreads, and the 6 real stories are mostly very elderly (4 of 6 are 80+) dying days-to-weeks post-vaccination -- which a base-rate calculation (a 10%/yr death rate in the 80+ group, ~12,000 such vaccinations in the sample) shows is roughly what coincidence predicts. Trump supporters were ~20x more likely than opponents to report a vaccine death. Conclusion: any poll whose result swings by an order of magnitude with respondents' politics or numeracy is useless as a guide to real-world frequencies -- stop using this methodology, defer to peer-reviewed studies.
Why this score
Quality 68 · Strong. Strong. A clean, honest empirical debunk that actually does the work -- replicates the question, chases down the individual cases, and runs the base rates -- reaching a calibrated null-plus-tiny-effect conclusion. Bounded (a single focused investigation), so mid-Strong.
Claude’s paradigm shift 44 · Moderate. Moderate. Mostly legwork; the modest fresh point is that politics-sensitive 'do you know someone killed by X' polls are structurally invalid guides to event frequency.
Real-world impact 2 · Minor. Minor. Counters a piece of viral vaccine misinformation, a real-world concern, but at the discourse level. Within-niche.