Model Evaluation for Extreme Risks

Abstract

Current approaches to building general-purpose AI systems tend to produce systems with both beneficial and harmful capabilities. Further progress in AI development could lead to capabilities that pose extreme risks, such as offensive cyber capabilities or strong manipulation skills. We explain why model evaluation is critical for addressing extreme risks. Developers must be able to identify dangerous capabilities (through "dangerous capability evaluations") and the propensity of models to apply their capabilities for harm (through "alignment evaluations"). These evaluations will become critical for keeping policymakers and other stakeholders informed, and for making responsible decisions about model training, deployment, and security.

The authors' own abstract, copied from arxiv.org.

How to cite it

Built from the details listed on this page. Check them against the paper before you submit.

Shevlane, T., Farquhar, S., Garfinkel, B., Phuong, M., Whittlestone, J., et al. (2023). Model Evaluation for Extreme Risks. Preprint, not yet peer-reviewed. https://arxiv.org/abs/2305.15324

Shevlane, Toby, et al. "Model Evaluation for Extreme Risks." Preprint, not yet peer-reviewed, 2023, https://arxiv.org/abs/2305.15324.

@article{shevlane2023model,
  title = {Model Evaluation for Extreme Risks},
  author = {Toby Shevlane and Sebastian Farquhar and Ben Garfinkel and Mary Phuong and Jess Whittlestone and others},
  journal = {Preprint, not yet peer-reviewed},
  year = {2023},
  url = {https://arxiv.org/abs/2305.15324},
}

What it is

Argues that before release, labs should test models for dangerous capabilities and for whether they will apply them — and sets out what responsible release decisions would look like. Free on arXiv.

Topics