Melanie Mitchell
Santa Fe Institute professor; author of a leading plain-language AI guide
Complexity researcher who writes about what current AI systems can and cannot do, arguing that benchmark performance is routinely mistaken for understanding.
Key arguments & positions
- Argues benchmark scores are regularly mistaken for understanding, and that models fail at the abstraction and analogy humans use easily.
- Warns against both dismissing and over-crediting current systems, and pushes for evaluation designed to resist memorization.
- Skeptical of near-term AGI timelines while treating present-day harms as the more tractable problem.
Accomplishments
- Davis Professor of Complexity at the Santa Fe Institute; PhD under Douglas Hofstadter.
- Author of Artificial Intelligence: A Guide for Thinking Humans (2019) and Complexity: A Guided Tour.
- Writes the AI: A Guide for Thinking Humans newsletter and co-leads work on abstraction-and-reasoning evaluation.
Papers & key writings
Links
In the library
Nothing of theirs is filed in the hub yet. Browse the full library.
Recommended next
Hand-picked from the hub based on what Melanie Mitchell covers.
Stop Automating Peer Review Without Rigorous Evaluation
Peer-reviewed ICML 2026 position paper by Joachim Baumann, Jiaxin Pei, Sanmi Koyejo, and Dirk Hovy, accepted for an oral presentation. Compares human and AI ICLR 2026 reviews and tests automated paper rewriting, reporting excessive reviewer agreement and susceptibility to stylistic changes. The authors argue for rigorous evaluation before automating peer review.
Why this: Also about AI evaluation
Hamel Husain
Personal site of machine-learning engineer Hamel Husain, with writing and teaching on AI evaluation, debugging, analysis, and measurement, drawing on prior work at Airbnb and GitHub.
Why this: Also about AI evaluation
AI Is Entering Its 'Slurp Juice' Era
Gizmodo article by Webb Wright (October 7, 2026) comparing the current AI boom with the 2021–2022 NFT and crypto hype cycle, including celebrity promotion and companies appointing metaverse executives.
Why this: Also about AI hype
Mercor Experts — Paid AI Training Roles
A marketplace matching professionals with paid contract work training and evaluating AI systems. Lists expert roles across software, data science, finance, medicine and more, with posted hourly rates; free to join as an expert.
Why this: Also about AI evaluation