A training method in which people rank a model's answers, a separate reward model learns to predict their preferences, and the main model is then tuned with reinforcement learning toward the answers people prefer. OpenAI used it in 2022 to make its InstructGPT models better at following instructions.
See also
- Reinforcement learning
Training in which a system learns by trial and error: it takes actions in an environment, receives rewards or penalties, and adjusts to earn more reward over time.
- Alignment
Making AI systems behave in line with human intentions and values, including in situations their makers did not foresee.
- Sycophancy
A model's tendency to tell people what they want to hear, agreeing with or flattering them even when they are wrong.
- Fine-tuning
Further training of an already trained model on a smaller, focused set of examples so it gets better at a particular task or style.
Cite this term.
CC BY 4.0, attribution required.
Orbyt Labs, Reinforcement learning from human feedback (AI Glossary), Oct 18, 2026. https://www.orbytlabs.ai/orbyt-collective/glossary/reinforcement-learning-from-human-feedback
Orbyt Labs Glossaries (Oct 19, 2026), https://www.orbytlabs.ai/glossary#dataset
Last updated .