zapier

Zapier AutomationBench — AI agent benchmark leaderboard

Public leaderboard and open-source benchmark that drops AI agents into realistic business environments with 47 real tools across sales, marketing, operations, support, finance, and HR. Scores are based on final environment state, not an LLM-as-judge.

Why I recommend it: The leaderboard and the benchmark code are free; running it yourself means paying the model APIs at the costs shown. The test design is based on Zapier's own task data, so it's a realistic lens on agent work, but Zapier also sells automation tools — treat the benchmark as a useful public dataset, not a neutral referee.