SRE-Bench: Agentic Reverse Engineering Benchmark

What it is

A Columbia University-hosted benchmark that tests AI agents on reverse engineering complex binary software. The page reports results from evaluations run with OpenAI, stating that GPT-6 Astra solves nearly all tasks while other evaluated models stay below 60%, and describes pilot evaluations of harder settings.

Topics