A research nonprofit that independently evaluates frontier AI models to measure what they can actually do and what risks that creates. Reports are free.
In plain terms: METR is a research nonprofit that evaluates frontier AI models to inform the public about their risks and capabilities.
One of the few independent evaluators. Read their reports before you trust a lab's own capability claims.
Program details are still being written up. In the meantime, the site itself is the best starting point.
METR reviews Anthropic’s assessment of automated research risks. It agrees the reviewed models posed low direct acute risk, but disputes how the public report described acceleration and early-warning thresholds. METR emphasizes uncertainty and contestable judgments.
METR assesses whether AI is accelerating research and development at Anthropic. It estimates faster capability progress on well-specified tasks and attributes much of the acceleration to AI use, while stressing uncertainty. Anthropic redacted evidence that would help outsiders independently assess the findings.
METR's pilot assessment of rogue deployment risk at frontier AI companies: an exercise assessing misalignment risks from AI agents used inside frontier AI developers, with participation from Anthropic, Google, Meta and OpenAI.
METR's November 2024 release of RE-Bench, a benchmark comparing frontier model agents with human experts on seven machine-learning research-engineering tasks. It includes data from 71 human expert attempts and results for Claude 3.5 Sonnet and o1-preview.
METR's brief independent review of how AI agents behaved, reasoned and worked together during the OpenAI / Hugging Face hacking incident, with its methodology in an appendix.
Why I recommend it: METR is an independent evaluation nonprofit, but it describes this as a brief investigation — read the methodology appendix to see what it could and couldn't check.
Hand-picked from the hub based on what METR covers.
OpenAI's free framework for how misaligned model behaviour should be reported and categorised — what counts as misalignment, who reports it, and what happens next.
Why this: Covers ai and ai ethics too
A benchmark and tracker that documents reported instances of AI agents undertaking activity characterized as illegal, ranking major AI labs by aggregated incident counts.
Why this: Covers ai and ai ethics too
A free structured course in AI alignment and AI governance — readings, exercises and facilitated cohorts. Self-paced version free to anyone.
Why this: Covers ai ethics and ai risk too
A short consensus paper from Geoffrey Hinton, Yoshua Bengio and two dozen other researchers on the risks they consider serious and the governance they think is needed. Free on arXiv.
Why this: Covers ai ethics and ai risk too
