Red teaming

See also

• AI safety

  The work of keeping AI systems from causing harm, whether through malicious use, malfunctions or wider risks to society.

• Jailbreak

  A prompt crafted to get a model to ignore its safety training and produce something it was trained to refuse.

• Prompt injection

  An attack in which input makes a model behave in ways its builders did not intend, either typed directly or hidden in content the model reads, such as a web page, an email or a document.

• Model card

  A document published with a model that describes what it is for, how it was evaluated, and its known limits.