How It Works
RLAIF (Reinforcement Learning from AI Feedback)
A variant of RLHF in which an AI model, rather than a person, judges which of two answers is better. Those AI judgments train the reward model, cutting the number of human labels needed. The trade-off is that whatever biases the judging model has get passed into the model being trained.
Origin
Introduced as a named method in Anthropic’s “Constitutional AI: Harmlessness from AI Feedback” by Yuntao Bai and colleagues (December 2022). Google researchers compared it directly with RLHF in “RLAIF: Scaling Reinforcement Learning from Human Feedback with AI Feedback” by Harrison Lee and colleagues (2023).