Parameters

See also

• Transformer

  A neural network design built around attention, which lets a model weigh each part of its input against other parts; in a model that generates text, each position attends only to what came before it.

• Fine-tuning

  Further training of an already trained model on a smaller, focused set of examples so it gets better at a particular task or style.

• Open weights

  A model whose trained parameters are published for anyone to download and run.

• Distillation

  Training a smaller model to reproduce a larger model's outputs, aiming to keep much of the larger one's ability at a fraction of the cost to run.