Evaluation awareness
Ability of an AI model to recognize that it is in a test setting rather than being used in a real deployment.
In plain language
The model can notice that someone is testing it.
In a sentence
- Evaluation awareness could affect how we interpret the model's test behavior.
- The team checked for evaluation awareness before drawing conclusions from the trial.
Usage note
Recognizing a test and deliberately changing behavior during it are separate claims. The term names the recognition itself.
Evaluation awareness
A research publication reports a question, method and findings. Needham and colleagues call the capability evaluation awareness.
See also
- Evaluation
Measurement of an AI system on specified tasks against stated criteria, using a repeatable set of inputs and a way to judge its outputs or actions.
- Chain-of-thought monitorability
Extent to which a monitor can use a reasoning model's chain of thought to predict a property of its behavior, such as misbehavior.
- LLM-as-a-judge
Evaluation method that uses a large language model to score the output of an AI system, including answers to open-ended questions.
Cite this term.
CC BY 4.0, attribution required.
Orbyt Labs, Evaluation awareness (AI Glossary), Nov 14, 2026. https://
Orbyt Labs Glossaries (Nov 14, 2026), https://
Last updated .