AI Glossary

Scaling laws

plural nounMachine learningpreferred in this glossary

Measured relationships in which language model loss follows a power law as model size, training dataset size and the compute used for training change.

Source: Kaplan et al. 2020 (opens in a new tab) (adapted)

Adapted: restates the measured relationship between loss and the three training quantities. Source: Kaplan et al. 2020 (opens in a new tab)

In plain language

Researchers measure how training resources relate to the model's prediction error.

In a sentence

  • The researchers fitted scaling laws to the measured training runs.
  • We checked which model sizes and datasets the scaling laws described.

Examples written for this glossary

Usage note

The relationships describe measured behavior over particular training runs. Extending a fitted relationship beyond those conditions is a forecast.

See also

  • Frontier model

    General-purpose AI model that outperforms, on a range of conventional performance benchmarks, the models that have been widely deployed for at least 12 months.

  • Mixture of experts

    Model design in which a trainable gate selects a combination of experts for each input; in the sparse form only a fraction of the parameters is active for each token.

Sources

Cite this term.

CC BY 4.0, attribution required.

This term

Orbyt Labs, Scaling laws (AI Glossary), Nov 14, 2026. https://www.orbytlabs.ai/orbyt-collective/glossary/scaling-laws

The dataset (JSON)

Orbyt Labs Glossaries (Nov 14, 2026), https://www.orbytlabs.ai/glossary#dataset

Last updated .