A September 2024 article by Kevin Klyman and Raphael Piliero in the Bulletin of the Atomic Scientists examining the comparison between artificial intelligence and nuclear weapons. The authors argue the analogy offers some lessons for risk mitigation but that the differences are more significant: AI development is diffuse across tens of firms and countries, making it far harder to contain than nuclear technology. Free to read.
Why I recommend it: The nuclear analogy drives a lot of AI policy proposals, from IAEA-style agencies to licensing regimes. This piece is the clearest short explanation of where that analogy helps and where it breaks.
A 39-page history by SJ Beard (University of Cambridge) and Émile P. Torres of how existential risk became an academic field, tracing three successive waves: an explicitly transhumanist and techno-utopian first wave, a second built on longtermism and closely tied to Effective Altruism, and a third formed where the field met disaster studies, environmental science and public policy. Posted to SSRN January 2021, revised March 2021.
Why I recommend it: Free — the full PDF downloads without an account. This is the piece to read before anyone quotes an extinction probability at you, because it shows the field's assumptions were inherited from a particular movement rather than discovered. It is a working paper posted by the authors, not journal peer-reviewed, and the authors are participants in the debate they are describing, so it is a history written from inside.
Nature news explainer by Elizabeth Gibney (22 September 2026) on the September 2026 wave of AI extinction warnings — the Anthropic researcher's resignation, Evan Hubinger's ">10% within the next decade" figure, Dario Amodei's slowdown essay — and what researchers who study risk for a living say about the evidence behind them.
Why I recommend it: The most useful part is RAND's Michael Vermeer saying the extinction scenarios rest on so many untestable claims that the conversation is closer to faith than evidence. Read it before repeating any percentage you see on social media — those numbers are personal estimates, not measurements.
A statement jointly signed by a historic coalition of AI experts: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
From the site: A statement jointly signed by a historic coalition of experts: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
Project from the Foresight Institute collecting expert interviews, essays and worldbuilding on plausible good long-term futures rather than only the risks.
Why I recommend it: A deliberate counterweight if the safety reading has left you gloomy. It is advocacy for optimism, so read it as a position, not a finding.
Cambridge research centre studying risks that could threaten humanity's long-term future, with open papers, seminars and policy submissions.
From the site: We study existential and global catastrophic risks & foster a worldwide community of academics, technologists and policy-makers working to mitigate these risks.
Why I recommend it: Academic and careful. Their reading lists and seminar recordings are the fastest way into the field's actual literature.
#academic#ai ethics#ai-safety
CSER - Centre for the Study of Existential RiskAdded Sep 17, 20260 opens
Non-profit working on risks from advanced technology, publishing policy work, the AI Safety Index, open letters and a large free podcast and newsletter archive.
From the site: FLI works on reducing extreme risks from transformative technologies. We are best known for developing the Asilomar AI governance principles.
Why I recommend it: Their AI Safety Index is the most readable scorecard of what the big labs actually do about safety. They campaign, so read the policy pages as arguments.
Personal site of Andrew Critch, mathematician and AI safety researcher, collecting his papers, talks and writing on multi-agent risk and existential safety.
Why I recommend it: Denser than most safety writing and worth the effort. Note the site refuses automated visits, so the picture here may be a screenshot.
A free explainer of Roko's Basilisk, the AI thought experiment about a hypothetical future superintelligence that would punish people who knew about it but did not help bring it into existence.
Stanford economist Charles I. Jones works out, in plain economic terms, how much money it would be worth spending to lower catastrophic risks from advanced AI — comparing it to the roughly 4 percent of GDP the U.S. effectively spent during Covid-19.
LessWrong wiki article explaining the canonical AI safety thought experiment: how an artificial general intelligence with an innocuous goal could pose an existential risk by pursuing it single-mindedly. Covers the orthogonality thesis and instrumental convergence.
