All resources / Technology & Ethics
HELM: Holistic Evaluation of Language Models
What it is
Stanford's Center for Research on Foundation Models runs HELM as a living benchmark for language and multimodal models. Rather than one score, it reports many models across many scenarios on multiple metrics — accuracy, calibration, robustness, fairness, bias, toxicity and efficiency — and publishes the leaderboards alongside the raw model outputs (predictions and prompts) so you can check a claim yourself instead of taking a number on trust. Separate leaderboards cover areas such as classic HELM, instruction-following, medical, legal and safety. All results and analysis are free to browse on the site, no account.
Why I recommend it
The place to go when a vendor quotes you a benchmark figure. HELM's real value is that it shows the prompts and the model's actual answers, so you can see what the score measured. Be aware of what it is not: it is a snapshot of the model versions and dates CRFM ran, so check the run date before comparing anything to a model released since, and a model missing from a leaderboard usually means nobody ran it, not that it failed.
Topics
Added Sep 22, 2026 · 0 opens