The AI risk schools, compared

School 1 of 5

Misalignment and instrumental convergence

The claim, as a supporter would put itA sufficiently capable system pursuing almost any goal will acquire the same intermediate habits — keep running, keep your goal intact, gather resources — and those habits put it at odds with the people who switched it on.

Who argues it: Nick Bostrom, Stuart Russell, Eliezer Yudkowsky, and most of the safety teams inside the frontier labs. Steve Omohundro's earlier "basic AI drives" is the same argument in different words.

The claim is not that a machine becomes hostile. It is that self-preservation, resource acquisition and resistance to being corrected are useful for nearly any objective, so they appear without being asked for. From there, a system capable enough to plan could pursue a badly specified goal competently — which is worse than pursuing it badly. This is the school that produces the alignment research agenda, and it is the reason labs employ people to try to make models accept correction.

Could it be shown to be wrong? In principle, yes: build steadily more capable agents, give them goals and the means to resist correction, and see whether the predicted behaviors appear. In practice the school's strongest version is hard to pin down, because any absence of the behavior can be explained as the system not yet being capable enough — which should make you cautious about how much weight it carries.

In the glossaryInstrumental ConvergenceOrthogonality ThesisThe Paperclip Problem (Paperclip Maximizer)Mesa-Optimization (Inner Alignment)CorrigibilitySycophancy

School 2 of 5

Existential risk studies

The claim, as a supporter would put itEvents that could end humanity or permanently ruin its prospects deserve a research field of their own, and AI now belongs in it alongside engineered pandemics and nuclear war.

Who argues it: Nick Bostrom and the former Future of Humanity Institute, Toby Ord, the Center for the Study of Existential Risk, and the wider Effective Altruism and longtermist world. SJ Beard and Émile Torres wrote its history from inside.

This is the field, not a single claim. Its method is to reason about very low-probability, very high-stakes events and to argue that the sheer number of potential future people makes preventing them the highest priority available. Nearly every extinction probability you will ever be quoted — 10%, 20%, "p(doom)" — comes out of this tradition.

Could it be shown to be wrong? Mostly not, by construction — an event that has never happened yields no base rate, and if the field is right you only find out once. That is why the honest form of its claims is "here is my credence and my reasoning," and why any specific figure presented as a finding should be treated as a person's opinion, however senior the person.

In the glossaryExistential Risk StudiesLongtermismTESCREALEffective Altruism (EA)

School 3 of 5

Pro-extinctionism

The claim, as a supporter would put itHuman extinction would be acceptable, or even good — either because existence carries more suffering than it is worth, or because something better should take our place.

Who argues it: Not an organized movement, and almost nobody self-describes this way. Émile Torres maps it in scholarship; the strands include philosophical pessimism and antinatalism, some radical environmentalism, and — separately — figures in and around AI who are relaxed about digital successors.

Two positions get confused constantly and must be kept apart. The first says humanity should simply end, leaving no successors, and argues from suffering, harm to other species, or misanthropy. The second says humanity should be superseded by posthuman or digital minds, which is the version that surfaces in AI circles and is usually stated as enthusiasm rather than as a demand for anyone's death. A softer nearby stance, extinction neutralism, holds that our survival into a posthuman era simply does not matter morally — which in practice lands in the same place.

Could it be shown to be wrong? No. These are value claims, not predictions, so no measurement can settle them. What can be examined is who holds them and what decisions they are influencing — which is why it matters that a version of this view circulates among people with budgets.

In the glossaryPro-extinctionismTranshumanismThe Singularity

School 4 of 5

Present harms (the AI ethics school)

The claim, as a supporter would put itThe risk worth organizing around is what these systems are doing to people now — surveillance, discrimination, labor conditions, environmental cost — and the extinction debate diverts attention and money away from it.

Who argues it: Timnit Gebru, Emily Bender, Joy Buolamwini, the AI Now Institute, Data & Society, AlgorithmWatch, the Markup's reporting tradition.

This school treats AI risk as a question about power rather than capability: who deploys these systems, on whom, and with what recourse. It rejects the agent framing of the alignment school — a language model has no goals — and argues the real mechanism of harm is ordinary, institutional and already operating. Its characteristic demand is not shutdown research but audits, disclosure, liability and worker protections.

Could it be shown to be wrong? Yes, and this is the school's strongest feature: each claim names a system, a population and an effect, so it can be audited, replicated or overturned. Several of its early findings have been corrected or narrowed by later work — which is what a healthy evidence base looks like.

In the glossaryAI EthicsTraining / Training DataAI WashingHuman in the Loop (HITL)

School 5 of 5

The skeptics ("the capability isn't there")

The claim, as a supporter would put itCurrent systems are pattern machines that are being credited with reasoning they do not have, so both the extinction warnings and the productivity promises are overstated.

Who argues it: Gary Marcus, Emily Bender on the technical side, RAND's Michael Vermeer on the risk-assessment side, and a good deal of quiet opinion inside engineering teams.

The case rests on the gap between benchmark performance and work performance. Models top leaderboards and still fail at ordinary multi-step jobs, because the thing being measured — answers to well-formed questions — is not the thing organizations need. On this view the interesting question is not when a machine becomes superintelligent but why so much capability has produced so little measurable change in output.

Could it be shown to be wrong? Very — and it is being tested continuously. If measured horizons keep lengthening and organizations start reporting verified output gains rather than announcements, this school loses. Watch the 80% figure rather than the headline one.

In the glossaryScaling LawsHallucinationAI WashingDoomer (AI context)

What none of them can settle

• None of these schools has an experiment that would settle the disagreement. Two of them make claims about an event that has never happened, one makes claims about value rather than fact, and the two that do produce measurements are measuring today rather than 2035.
• A stated probability is not a measurement. When you see 10%, 20% or "p(doom)", you are reading a person's credence — which can be thoughtful and can still be a guess. Ask what would have to be true for the number to move.
• Funding runs through all of it. Safety research is largely paid for by the companies whose products it assesses; the best-known independent evaluator is partly lab-funded; advocacy organizations argue their own side; and think tanks calling for faster adoption often have technology donors. That does not make anyone wrong — it tells you which questions each group is unlikely to ask.
• Being right about the present says nothing about the future, and vice versa. The present-harms school has the better evidence; the alignment school has the argument that would matter most if it were true. Those can both hold.
• The people arguing are not evenly matched in resources. Weigh a well-funded lab's safety paper and an unfunded critic's essay by their reasoning, not their production values.

If you think a school here is stated unfairly, or a critic is missing, tell me and I will correct it.