Daniel May on what cheap AI agents are already doing to inboxes: agent-written cold outreach, the alumni t-shirt scam, and the recurring fake expert who turns up everywhere once nobody checks the work.
Why I recommend it: Free to read. Worth twenty minutes if you send or receive cold messages: it shows exactly what automated outreach looks like from the other side, and why a message that reads fluently now proves nothing about whether a person wrote it.
Raji and colleagues examine the ethics of the audits themselves: who is photographed, who consented, and what an auditor owes the people in the test set.
From the site: Although essential to revealing biased performance, well intentioned attempts at algorithmic auditing can have effects that may harm the very populations these measures are meant to protect. This concern is even more salient while auditing biometric systems such as facial recognition, where the data is sensitive and t…
Why I recommend it: A rare paper about the ethics of doing ethics work. Free on arXiv.
Vinay Prabhu and Abeba Birhane examine widely used image datasets and find non-consensual photos of real people, offensive labels and no realistic route to consent.
From the site: In this paper we investigate problematic practices and consequences of large scale vision datasets. We examine broad issues such as the question of consent and justice as well as specific concerns such as the inclusion of verifiably pornographic images in datasets. Taking the ImageNet-ILSVRC-2012 dataset as an example…
Why I recommend it: This is the audit that got a major benchmark dataset withdrawn. Short, readable, and free.
Birhane and colleagues show that training on a larger scrape makes hateful content and racist misclassification worse, not better — direct evidence against "more data fixes it".
From the site: `Scale the model, scale the data, scale the GPU-farms' is the reigning sentiment in the world of generative AI today. While model scaling has been extensively studied, data scaling and its downstream impacts remain under explored. This is especially of critical importance in the context of visio-linguistic datasets wh…
Why I recommend it: Useful whenever someone argues scale solves bias. Free in full on arXiv.
Abeba Birhane, Vinay Prabhu and Emmanuel Kahembwe audit the LAION-400M dataset used to train popular image models and document the racist, misogynistic and non-consensual material inside it.
From the site: We have now entered the era of trillion parameter machine learning models trained on billion-sized datasets scraped from the internet. The rise of these gargantuan datasets has given rise to formidable bodies of critical work that has called for caution while generating these large datasets. These address concerns sur…
Why I recommend it: The paper to read before anyone tells you a model is fine because the data was "publicly available". Free in full on arXiv.
Raji and colleagues argue many deployed AI systems fail on their own stated terms — they simply do not work — and that this belongs in the harm conversation alongside bias.
From the site: Deployed AI systems often do not work. They can be constructed haphazardly, deployed indiscriminately, and promoted deceptively. However, despite this reality, scholars, the press, and policymakers pay too little attention to functionality. This leads to technical and policy solutions focused on "ethical" or value-ali…
Why I recommend it: The first question is not "is it fair" but "does it work at all". Free in full on arXiv.
Raji and co-authors show that benchmarks claiming to measure general ability measure something much narrower, and that the gap is how overclaiming happens.
From the site: There is a tendency across different subfields in AI to valorize a small collection of influential benchmarks. These benchmarks operate as stand-ins for a range of anointed common problems that are frequently framed as foundational milestones on the path towards flexible and generalizable AI systems. State-of-the-art…
Why I recommend it: Read this before you trust a benchmark chart in a launch post. Free on arXiv.
Deborah Raji and co-authors set out a practical, stage-by-stage internal audit process for AI systems, from scoping through to post-deployment review.
From the site: Rising concern for the societal implications of artificial intelligence systems has inspired a wave of academic and journalistic literature in which deployed systems are audited for harm by investigators from outside the organizations deploying the algorithms. However, it remains challenging for practitioners to ident…
Why I recommend it: The closest thing to a step-by-step audit template you can use inside an organization. Free on arXiv.
Five plain-language protections people should expect from automated systems — safe systems, protection from discrimination, data privacy, notice and explanation, and a human alternative — each with a section on what organisations should do.
From the site: Among the great challenges posed to democracy today is the use of technology, data, and automated systems in ways that threaten the rights of the American public. Too often, these tools are used to limit our opportunities and prevent our access to critical resources or services. These problems are well documented. In…
Why I recommend it: Led by Alondra Nelson at the White House science office in 2022. It was never binding and has since been removed from whitehouse.gov, so this link goes to the official archive — still the clearest short statement of what people are owed.
Arvind Narayanan and Sayash Kapoor's newsletter separating AI that works from AI that is oversold, with close readings of specific product and research claims.
From the site: Analyzing AI as transformative but normal technology, not superintelligence. Click to read AI as Normal Technology, a Substack publication with tens of thousands of subscribers.
Why I recommend it: The archive is free to read and the best regular antidote to launch-post hype. The book of the same name is a paid purchase — you do not need it to follow the newsletter.
Proposes a short standard document to ship with every trained model: what it is for, who it was tested on, where it performs worse, and what it should not be used for. Most model documentation you see today descends from this. Free on arXiv.
From the site: Trained machine learning models are increasingly used to perform high-impact tasks in areas such as law enforcement, medicine, education, and employment. In order to clarify the intended use cases of machine learning models and minimize their usage in contexts for which they are not well suited, we recommend that rele…
Why I recommend it: This is the practical end of AI ethics — a template, not an argument. Useful if you ever have to evaluate a vendor's model.
Separates the technical question (how do you make a system pursue a goal) from the normative one (whose values, chosen how). Argues no single person's values are a legitimate target and looks at fair-process alternatives. Free on arXiv.
From the site: This paper looks at philosophical questions that arise in the context of AI alignment. It defends three propositions. First, normative and technical aspects of the AI alignment problem are interrelated, creating space for productive engagement between people working in both domains. Second, it is important to be clear…
Why I recommend it: The clearest philosophical treatment of “aligned to what?” I have found. Skip the lab blog posts and read this instead.
A long report on what changes when AI acts on your behalf rather than answering questions: manipulation, anthropomorphism, misaligned delegation, and what happens when everyone has an assistant at once. Free on arXiv.
From the site: This paper focuses on the opportunities and the ethical and societal risks posed by advanced AI assistants. We define advanced AI assistants as artificial agents with natural language interfaces, whose function is to plan and execute sequences of actions on behalf of a user, across one or more domains, in line with th…
Why I recommend it: Dense but the most thorough thing published on agent ethics. Use the section headings to read only the parts you need.
Argues that before release, labs should test models for dangerous capabilities and for whether they will apply them — and sets out what responsible release decisions would look like. Free on arXiv.
From the site: Current approaches to building general-purpose AI systems tend to produce systems with both beneficial and harmful capabilities. Further progress in AI development could lead to capabilities that pose extreme risks, such as offensive cyber capabilities or strong manipulation skills. We explain why model evaluation is…
Why I recommend it: This is where today's “frontier safety framework” language comes from. Written largely by the labs it would govern, which is worth holding in mind.
Argues that ever-larger language models carry costs that scale with them: environmental cost, unauditable training data, encoded bias, and the illusion of understanding. The source of the phrase “stochastic parrot.” Free to read on the ACM site.
Why I recommend it: The paper that got two of its authors pushed out of Google. Worth reading before you accept either the hype or the dismissal of it.
Borrows the electronics-industry datasheet idea for training data: how it was collected, who is in it, who consented, and what it should not be used for. Free on arXiv.
From the site: The machine learning community currently has no standardized process for documenting datasets, which can lead to severe consequences in high-stakes domains. To address this gap, we propose datasheets for datasets. In the electronics industry, every component, no matter how simple or complex, is accompanied with a data…
Why I recommend it: Pairs directly with Model Cards. Together they are the closest thing the field has to a documentation standard.
Reviewed 146 papers on bias in language technology and found most never say who is harmed or how. Argues bias work has to start from real-world power relations, not just from a metric. Free on arXiv.
From the site: We survey 146 papers analyzing "bias" in NLP systems, finding that their motivations are often vague, inconsistent, and lacking in normative reasoning, despite the fact that analyzing "bias" is an inherently normative process. We further find that these papers' proposed quantitative techniques for measuring or mitigat…
Why I recommend it: Read this if “bias” has started to sound like a box to tick. It is a careful takedown of shallow fairness work by people who do the work.
The founding technical paper on algorithmic fairness: defines fairness as treating similar individuals similarly, and shows why blindness to a protected attribute does not deliver it. Mathematical. Free on arXiv.
From the site: We study fairness in classification, where individuals are classified, e.g., admitted to a university, and the goal is to prevent discrimination against individuals based on their membership in some group, while maintaining utility for the classifier (the university). The main conceptual contribution of this paper is…
Why I recommend it: The math is heavy, but the first few pages explain why “we just don't collect race” is not a fairness strategy.
Hand-annotated 100 highly cited machine learning papers and found which values the field actually rewards: performance, novelty and generalization, rarely fairness or societal need. Also traces the funding behind the work. Free on arXiv.
From the site: Machine learning currently exerts an outsized influence on the world, increasingly affecting institutional practices and impacted communities. It is therefore critical that we question vague conceptions of the field as value-neutral or universally beneficial, and investigate what specific values the field is advancing…
Why I recommend it: Turns “the field has blind spots” into countable evidence. Good antidote to the idea that research priorities are neutral.
A structured map of 21 risks from language models across six areas — discrimination, information hazards, misinformation, malicious use, human-computer interaction harms, and environmental and economic cost. Free on arXiv.
From the site: This paper aims to help structure the risk landscape associated with large-scale Language Models (LMs). In order to foster advances in responsible innovation, an in-depth understanding of the potential risks posed by these models is needed. A wide range of established and anticipated risks are analysed in detail, draw…
Why I recommend it: The best single reference if you need vocabulary for a specific harm rather than a general argument. Written by a lab, so read it as a lab's own framing.
Tested three commercial face-classification products and found error rates of up to 34.7% for darker-skinned women against 0.8% for lighter-skinned men. The study that turned algorithmic bias from a theory into a measured, published fact. Free to read in full.
From the site: Recent studies demonstrate that machine learning algorithms can discriminate based on classes like race and gender. In this work, we present an approach to evaluate bias present in automated facial...
Why I recommend it: If you read one AI ethics paper, read this one. It is short, the method is easy to follow, and it is the reason facial recognition audits exist at all.
Shows that “interpretable” is used to mean several incompatible things, and that simpler models are not automatically more honest about what they do. Free on arXiv.
From the site: Supervised machine learning models boast remarkable predictive capabilities. But can you trust your model? Will it work in deployment? What else can it tell you about the world? We want models to be not only good, but interpretable. And yet the task of interpretation appears underspecified. Papers provide diverse and…
Why I recommend it: Useful skepticism to carry into any conversation about explainable AI, especially a vendor's.
The US government's voluntary framework for identifying and managing AI risk, plus its playbook of concrete practices. Free.
Why I recommend it: The one your employer's legal team is most likely already citing. Useful vocabulary if you want to raise AI risk at work and be taken seriously.
Microsoft's open-source toolkit for red-teaming AI systems: automated attack prompts, scoring of the responses, and repeatable runs. Free.
From the site: The Python Risk Identification Tool for generative AI (PyRIT) is an open source framework built to empower security professionals and engineers to proactively identify risks in generative AI system...
Why I recommend it: Built by the team that red-teams Microsoft's own AI products, and released as-is. Best paired with a written idea of what you are testing for.
The 2016 paper that framed AI safety as a set of specific engineering problems — side effects, reward hacking, unsafe exploration — rather than a philosophical worry. Free on arXiv.
From the site: Rapid progress in machine learning and artificial intelligence (AI) has brought increasing attention to the potential impacts of AI technologies on society. In this paper we discuss one such potential impact: the problem of accidents in machine learning systems, defined as unintended and harmful behavior that may emer…
Why I recommend it: Start here if the safety conversation sounds abstract. It is plain about what can go wrong and why, and almost everything since cites it.
A free structured course in AI alignment and AI governance — readings, exercises and facilitated cohorts. Self-paced version free to anyone.
From the site: Free online courses, grants, and intensive in-person programs from the leading talent accelerator for beneficial AI and societal resilience. Join 10,000+ alumni and start today.
Why I recommend it: The usual route in for people trying to move into safety work. The reading list alone is worth the visit even if you never join a cohort.
A short consensus paper from Geoffrey Hinton, Yoshua Bengio and two dozen other researchers on the risks they consider serious and the governance they think is needed. Free on arXiv.
From the site: Artificial Intelligence (AI) is progressing rapidly, and companies are shifting their focus to developing generalist AI systems that can autonomously act and pursue goals. Increases in capabilities and autonomy may soon massively amplify AI's impact, with risks that include large-scale social harms, malicious uses, an…
Why I recommend it: The clearest statement of what the safety-concerned researchers actually agree on, signed rather than paraphrased.
A structured survey of the risks — malicious use, competitive pressure, organizational failure, and systems pursuing goals of their own — with the evidence for each. Free to read.
From the site: There are many potential risks from AI. CAIS focusses on mitigating risks that could lead to catastrophic outcomes for society, such as bioterrorism or loss of control over military AI systems.
Why I recommend it: The best single map of the different worries, which are usually mashed together into one. Written by a safety organization, so read it as advocacy with citations.
An open-source framework from the UK's AI Security Institute for evaluating models — writing tests, scoring answers and logging what happened. Free.
From the site: Open-source framework for large language model evaluations
Why I recommend it: What a government safety institute actually uses to test models. Technical, but the docs explain the thinking behind each kind of test.
Yann LeCun's position paper arguing that today's language models are the wrong architecture, and sketching what he thinks should replace them. Free to read.
Why I recommend it: The serious technical case against scaling language models further. Dense, but it is the argument itself rather than a summary of it.
An open-source scanner that probes a language model for weaknesses — prompt injection, data leakage, jailbreaks, toxic output — and reports what it found. Free.
From the site: the LLM vulnerability scanner. Contribute to NVIDIA/garak development by creating an account on GitHub.
Why I recommend it: Point it at a model you are about to rely on and see how it fails before your users do.
An open-source library of metrics and algorithms for finding and reducing unwanted bias in datasets and models, in Python and R. Free.
From the site: A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models. - Trusted-AI/AIF360
Why I recommend it: For the harm that shows up in ordinary systems long before anything dramatic does — hiring screens, lending, scoring. Measuring bias is the easy half; deciding what fair means is yours.
A detailed scenario for how AI might develop through 2027, written by former OpenAI researcher Daniel Kokotajlo and colleagues, with the reasoning and uncertainties spelled out. Free to read in full.
From the site: A research-backed AI scenario forecast.
Why I recommend it: The forecast everyone in this field argued about. Read it as one carefully argued scenario, not a prediction — the authors say as much themselves.
An open-source tool for testing and red-teaming prompts and AI apps — run the same prompts across models, compare answers, and catch regressions. Free and self-hosted.
From the site: The AI Security Platform that catches vulnerabilities in development. Trusted by 156 of the Fortune 500 and 300,000+ developers worldwide.
Why I recommend it: The practical one: if you have built anything on top of a model, this is how you check a prompt change did not quietly make it worse.
Anthropic's paper describing how Claude is trained against a written set of principles instead of relying only on human ratings. Free on arXiv.
From the site: As AI systems become more capable, we would like to enlist their help to supervise other AIs. We experiment with methods for training a harmless AI assistant through self-improvement, without any human labels identifying harmful outputs. The only human oversight is provided through a list of rules or principles, and s…
Why I recommend it: Worth reading to see what "aligned" means in practice at one lab — and note it comes from the company selling the model.
