Paul Christiano
Executive director, Alignment Research Center; technical advisor, NIST Center for AI Standards and Innovation
An alignment researcher who led the work introducing reinforcement learning from human feedback, the training method behind today’s chat assistants, and who shaped how the field talks about alignment and the alignment tax.
In the library
Nothing of theirs is filed in the hub yet. Browse the full library.
Recommended next
Hand-picked from the hub based on what Paul Christiano covers.
What Is RLCD? Reinforcement Learning from Contrast Distillation
A plain-language explainer on RLCD, a way of aligning language models by learning from contrasting outputs rather than human ratings alone.
Why this: Covers RLHF and AI alignment too
The Basic AI Drives
Paper by Stephen Omohundro (2008) arguing that sufficiently advanced goal-directed AI systems would tend to develop basic drives such as self-preservation, resource acquisition, and goal preservation, and discussing what those drives imply for designing safe AI. Hosted on the Self-Aware Systems site.
Why this: Also about AI alignment
"Alignment Engineering" vs. "Misalignment Science"
LessWrong post that separates AI alignment research into two categories, engineering work to make systems behave and scientific work to understand misalignment, in the context of debate over whether some alignment research is net negative.
Why this: Also about AI alignment
Towards Understanding Sycophancy in Language Models
Research paper investigating why AI assistants fine-tuned with human feedback tend to produce responses that match user beliefs rather than truthful ones. Analyzes five state-of-the-art AI assistants across text-generation tasks and the role of human preference data in driving this sycophantic behavior.
Why this: Also about RLHF