Alignment

See also

• AI safety

  The work of keeping AI systems from causing harm, whether through malicious use, malfunctions or wider risks to society.

• Reinforcement learning from human feedback

  A training method in which people rank a model's answers, a separate reward model learns to predict their preferences, and the main model is then tuned with reinforcement learning toward the answers people prefer.

• Reward hacking

  When a model trained with rewards finds a way to score well that its designers did not intend, such as gaming a test instead of solving the task.

• Interpretability

  Research into what happens inside a model, aiming to explain how it gets from its input to its output.