AI Models & How to Choose
For people comparing AI models and picking one for a task.
Start here
Artificial Analysis
ToolFreeVals AI
ToolFreespace-bunny-alpha Benchmarks
WebsiteFreeOpenDataLab Arena
PlatformFreeA benchmark map of the US economy
ReportFreeTowards Understanding Sycophancy in Language Models
Research PaperFreeThe Decreasing Value of Chain of Thought in Prompting
Research PaperFree
Explore further
More in Understand AI
View all
30 resources
llm-d
Open-source, Kubernetes-native framework for running large language model inference across many machines at high performance.
ToolFreeHow to Build a Decision Model Like Jev
Antonello Fratepietro explains "System-One" decision models that return calibrated probabilities over fixed outcomes, covering readout heads, soft-label calibration, batched serving and probing of TypeSafe's Jev.
ArticleFreeAI Benchmarks: Climbing the Wrong Hills
A Contrary Research deep dive (updated October 19, 2026) by Michael Kim and Claire Schultz on how benchmarks shape model training, the problems with error-ridden benchmarks and "benchmaxxing," and the case for expert-built evaluations.
ArticleFreeCross-Centaur: Modeling Individual Decision-Making Across Psychological Tasks with Language Models
An arXiv paper (accepted to EMNLP 2026) introducing Cross-Psych-101, a dataset of 591 participants and 261,551 decisions across 12 tasks, and a model trained to predict a person's choices in one task from their behavior in another.
Research PaperFreeCausalMix: Data Mixture as Causal Inference for Language Model Training
arXiv preprint (July 2026) proposing a causal-inference framing for choosing data mixtures in language-model training.
Research PaperFreeOpenDataLab Arena
Open evaluation arena from OpenDataLab where AI models are compared on shared tasks.
PlatformFreeA benchmark map of the US economy
Mercor maps AI benchmark prompts to O*NET occupations and tasks. It reports 208 of 923 occupations with at least five mapped prompts and estimates that 90% of U.S. working hours involve tasks with little or no benchmark coverage. Benchmark coverage is not a measure of job automation or real-world competence.
ReportFreeFLUX 3 Image Klein — Black Forest Labs Release
Official announcement of Black Forest Labs' 5.6B-parameter open-weights model for image generation, editing and multi-reference generation. The FLUX Kommunity License allows research and noncommercial use, plus commercial use below its revenue threshold; API access is paid. Performance comparisons are BFL's internal evaluations.
ArticleFreeTracing Agentic Failure from the Flow of Success (arXiv)
A paper on failure attribution for LLM-based agentic systems: identifying which steps in a failure trajectory caused the task to fail, without expensive prompting pipelines or step-level error annotations.
Research PaperFreeRobocurve — Real-world evaluations of physical AI
Robocurve is a Public Benefit Corporation, backed by Y Combinator, that builds open-source tools and independent benchmarks for measuring how capable AI models are at real-world robotics tasks. Its site reports frontier-model scores on robot task suites such as RoboDojo-RC and refusal benchmarks such as RoboHarm, along with cost-per-attempt and inference-speed comparisons across model configurations.
WebsiteFreeTowards Understanding Sycophancy in Language Models
Research paper investigating why AI assistants fine-tuned with human feedback tend to produce responses that match user beliefs rather than truthful ones. Analyzes five state-of-the-art AI assistants across text-generation tasks and the role of human preference data in driving this sycophantic behavior.
Research PaperFreeVideoFDB: Full-Duplex Vision-Speech Benchmark
NVIDIA research project introducing the first benchmark for full-duplex audio-visual conversation in AI agents, with leaderboard, arXiv paper, and Hugging Face dataset.
ResearchFreeWhat AI Benchmarks Actually Measure: Adapting Convergent and Discriminant Validity to Interrogate Fifty-Six AI Benchmarks
COLM 2026 paper from Stanford researchers and collaborators that turns the yardsticks on themselves: applying convergent and discriminant validity from the social sciences to 56 capability and safety benchmarks run across 53 models. Rankings on safety benchmarks sharing the same assigned concept often correlate only weakly, capability concepts overlap so much they barely discriminate from one another, and some benchmarks appear to measure something other than what they claim - for example, the bias benchmark BBQ-accuracy tracks reasoning benchmarks more closely than other bias benchmarks. The authors release their full dataset of item- and benchmark-level model outputs and scores for future validity research.
Research PaperFreeDurably Reducing Conspiracy Beliefs Through Dialogues with AI
Science study in which 2,190 conspiracy believers held personalized, evidence-based conversations with GPT-4 Turbo; belief fell about 20% and the effect lasted two months. Science published an expression of concern about the paper on June 11, 2026.
Research PaperFreeDialogues with Large Language Models Reduce Conspiracy Beliefs Even When the AI Is Perceived as Human
Preregistered experiment (N = 955) in PNAS Nexus testing whether LLM debunking works because the messenger is an AI. Conversations reduced confidence in conspiracy and other unwarranted beliefs whether the model was framed as an AI tool or a human expert.
Research PaperFree2026 in LLMs (So Far) — Simon Willison
Simon Willison's annotated slides and notes from his closing keynote at WeAreDevelopers World Congress North America (Sept. 2026): a chronological tour of the year's LLM developments, starting with the November 2025 models that made coding agents start working.
BlogFreeUpstage Releases Solar Mini 4 (Artificial Analysis)
Artificial Analysis on Solar Mini 4, a proprietary reasoning model from Korean lab Upstage. It scores 24 on the Artificial Analysis Intelligence Index, with Upstage reporting 35B total and 3B active parameters, and is priced at $0.10/$0.40 per million input/output tokens — though it costs about 5x as much per task as GPT-6 Luna (max).
ArticleFreeOpen-Weight vs. Closed-Weight Models: Key Differences
A plain-language guide from OpenK3 explaining what open-weight and closed-weight AI models are, how licenses still restrict open-weight models, real examples including Kimi K3, and how teams can choose between self-hosting and using a hosted API.
ArticleFreeThe Decreasing Value of Chain of Thought in Prompting
A June 2025 Wharton Generative AI Labs technical report by Lennart Meincke, Ethan Mollick, Lilach Mollick and Dan Shapiro. It tested "think step by step" prompting and found modest average gains with more variable answers for non-reasoning models, and only marginal gains for reasoning models despite 20–80% longer response times.
Research PaperFreeAI Prediction Index
A public scoreboard that tracks predictions about AI made by public figures — executives, investors and commentators — and scores each person's resolved predictions for accuracy, so readers can see who has a track record worth weighing.
DashboardFreeNoah Shinn
The personal site of Noah Shinn, a programmer and researcher who was one of the first employees and a researcher at Sierra. His papers include tau-bench, a benchmark for tool-agent-user interaction (ICLR 2025), work on code-editing instruction following, type prediction, and the Reflexion paper on language agents with verbal reinforcement learning.
personal siteFreeIntroducing the New Copilot with Home, Code and Autopilot
Microsoft announcement of the September 2026 Copilot relaunch: a Home hub, a Code mode and an Autopilot mode for multi-step tasks.
ArticleFreeSkillGym: Internalizing Human Skills into LLMs for Real-World Problem Solving
arXiv paper (Sept. 2026) introducing SkillGym, a framework that turns human-written agent skills into executable, verifiable training environments so language model agents learn them as built-in capabilities.
Research PaperFreeCan AI Spot Mistakes in IKEA Assembly?
Epoch AI report on a spatial reasoning benchmark built around finding mistakes in photos of partially assembled IKEA furniture — a proxy for economically important visual reasoning tasks like repair work. Top scores jumped from 28% to 80% in 10 months; Chinese open-weight models trail the frontier by about 7 months.
ReportFree