Scaling laws
Measured relationships in which language model loss follows a power law as model size, training dataset size and the compute used for training change.
In plain language
Researchers measure how training resources relate to the model's prediction error.
In a sentence
- The researchers fitted scaling laws to the measured training runs.
- We checked which model sizes and datasets the scaling laws described.
Usage note
The relationships describe measured behavior over particular training runs. Extending a fitted relationship beyond those conditions is a forecast.
See also
- Frontier model
General-purpose AI model that outperforms, on a range of conventional performance benchmarks, the models that have been widely deployed for at least 12 months.
- Mixture of experts
Model design in which a trainable gate selects a combination of experts for each input; in the sparse form only a fraction of the parameters is active for each token.
Cite this term.
CC BY 4.0, attribution required.
Orbyt Labs, Scaling laws (AI Glossary), Nov 14, 2026. https://
Orbyt Labs Glossaries (Nov 14, 2026), https://
Last updated .