pass@k
Evaluation metric for the probability that an AI system produces at least one correct solution in k attempts at a task, rather than requiring every attempt to succeed.
In plain language
Count whether any attempt succeeds within the allowed tries.
In a sentence
- The report gave pass@k alongside the number of attempts allowed per task.
- A high pass@k did not mean the agent succeeded on every attempt.
Usage note
Success at least once and success on every trial answer different evaluation questions. State which event the reported score measures.
pass^k
A research publication reports a question, method and findings. The benchmark defines pass^k as the chance that all k independent, identically distributed task trials succeed, averaged across tasks.
See also
- Benchmark
Repeatable collection of tasks and scoring measures used to compare the performance of AI systems under stated conditions and on a stated range of work.
Cite this term.
CC BY 4.0, attribution required.
Orbyt Labs, pass@k (AI Glossary), Nov 14, 2026. https://
Orbyt Labs Glossaries (Nov 14, 2026), https://
Last updated .