Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing

Abstract

Rising concern for the societal implications of artificial intelligence systems has inspired a wave of academic and journalistic literature in which deployed systems are audited for harm by investigators from outside the organizations deploying the algorithms. However, it remains challenging for practitioners to identify the harmful repercussions of their own systems prior to deployment, and, once deployed, emergent issues can become difficult or impossible to trace back to their source. In this paper, we introduce a framework for algorithmic auditing that supports artificial intelligence system development end-to-end, to be applied throughout the internal organization development lifecycle. Each stage of the audit yields a set of documents that together form an overall audit report, drawing on an organization's values or principles to assess the fit of decisions made throughout the process. The proposed auditing framework is intended to contribute to closing the accountability gap in the development and deployment of large-scale artificial intelligence systems by embedding a robust process to ensure audit integrity.

The authors' own abstract, copied from arxiv.org.

How to cite it

Built from the details listed on this page. Check them against the paper before you submit.

Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020). Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing. Preprint, not yet peer-reviewed. https://arxiv.org/abs/2001.00973

Raji, Inioluwa Deborah, et al. "Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing." Preprint, not yet peer-reviewed, 2020, https://arxiv.org/abs/2001.00973.

@article{raji2020closing,
  title = {Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing},
  author = {Inioluwa Deborah Raji and Andrew Smart and Rebecca N. White and Margaret Mitchell and Timnit Gebru and Ben Hutchinson and Jamila Smith-Loud and Daniel Theron and Parker Barnes},
  journal = {Preprint, not yet peer-reviewed},
  year = {2020},
  url = {https://arxiv.org/abs/2001.00973},
}

What it is

Deborah Raji and co-authors set out a practical, stage-by-stage internal audit process for AI systems, from scoping through to post-deployment review.

Topics