AI Glossary

Evaluation awareness

noun phraseAI safetypreferred in this glossary

Ability of an AI model to recognize that it is in a test setting rather than being used in a real deployment.

Source: Essifi, evaluation awareness 2026 (opens in a new tab)

In plain language

The model can notice that someone is testing it.

In a sentence

  • Evaluation awareness could affect how we interpret the model's test behavior.
  • The team checked for evaluation awareness before drawing conclusions from the trial.

Examples written for this glossary

Usage note

Recognizing a test and deliberately changing behavior during it are separate claims. The term names the recognition itself.

  • Evaluation awareness

    Research

    A research publication reports a question, method and findings. Needham and colleagues call the capability evaluation awareness.

    Source: Needham et al. 2025 (opens in a new tab)

See also

  • Evaluation

    Measurement of an AI system on specified tasks against stated criteria, using a repeatable set of inputs and a way to judge its outputs or actions.

  • Chain-of-thought monitorability

    Extent to which a monitor can use a reasoning model's chain of thought to predict a property of its behavior, such as misbehavior.

  • LLM-as-a-judge

    Evaluation method that uses a large language model to score the output of an AI system, including answers to open-ended questions.

Cite this term.

CC BY 4.0, attribution required.

This term

Orbyt Labs, Evaluation awareness (AI Glossary), Nov 14, 2026. https://www.orbytlabs.ai/orbyt-collective/glossary/evaluation-awareness

The dataset (JSON)

Orbyt Labs Glossaries (Nov 14, 2026), https://www.orbytlabs.ai/glossary#dataset

Last updated .