Large language model

See also

• Transformer

  A neural network design built around attention, which lets a model weigh each part of its input against other parts; in a model that generates text, each position attends only to what came before it.

• Foundation model

  A large model trained on broad data that can be adapted to many different tasks, like the models behind chat assistants.

• Token

  The small unit of text a language model reads and writes, often a whole word or a piece of one.

• Pretraining

  The first stage of training, in which a model learns general patterns from a large, broad dataset before it is adapted to particular tasks.