Training is the iterative process by which an AI model learns to perform a task by repeatedly adjusting its parameters to minimise a loss function measured against labelled or self-supervised data. At each step, the model makes predictions, the loss quantifies how wrong those predictions are, and gradient descent propagates corrections back through the network via backpropagation. For modern LLMs, this process runs across billions of examples on clusters of thousands of GPUs or TPUs for weeks or months, consuming megawatt-hours of electricity and millions of dollars of compute.

Think of it like training for a marathon: you run repeatedly, receive feedback (your time and how your body feels), and make micro-adjustments to pacing, diet, and form over hundreds of sessions until performance peaks. Each training run is a lap; the Loss Function is the stopwatch.

for epoch in range(num_epochs):
    for batch in dataloader:
        optimizer.zero_grad()
        outputs = model(batch["input_ids"])
        loss = criterion(outputs, batch["labels"])
        loss.backward()   # compute gradients
        optimizer.step()  # update parameters

This inner loop—forward pass, loss, backward pass, update—is repeated billions of times during Pretraining.

Training is the foundational phase that precedes Fine-tuning and RLHF; it produces the base model weights that downstream techniques then specialise. It is also where Overfitting is most likely to occur if the training data is too small or homogeneous relative to model capacity.

You'll encounter training costs and timelines when evaluating Foundation Model providers, deciding whether to fine-tune an existing model or train from scratch, or reading research papers that report compute in petaFLOP-days.