AI Glossary

Transformer

A neural network design built around attention, which lets a model weigh each part of its input against other parts; in a model that generates text, each position attends only to what came before it. It was introduced in the 2017 Google paper Attention Is All You Need, and many of today's large language models are built on it.

See also

  • Large language model

    A model with a very large number of parameters, trained on very large amounts of text to predict the next token, which lets it write, summarize and answer questions.

  • Parameters

    The numbers inside a model, its weights and biases, that are set during training and together hold what it learned.

Source: Vaswani et al., Attention Is All You Need (2017) (opens in a new tab)

Cite this term.

CC BY 4.0, attribution required.

This term

Orbyt Labs, Transformer (AI Glossary), Oct 18, 2026. https://www.orbytlabs.ai/orbyt-collective/glossary/transformer

The dataset (JSON)

Orbyt Labs Glossaries (Oct 19, 2026), https://www.orbytlabs.ai/glossary#dataset

Last updated .