Overfitting occurs when a model learns the training data so thoroughly that it captures noise and idiosyncratic patterns rather than the underlying structure it is meant to generalize from. The result is a model that scores well on its training examples but performs poorly on data it has never seen. It is one of the central problems in machine learning because the entire purpose of training a model is to make accurate predictions on new inputs, not to memorize a fixed dataset.

Think of it like a student who memorizes every question and answer from last year's exam rather than understanding the subject. The student scores perfectly on a practice run with those same questions but fails a slightly different test drawn from the same curriculum, because they learned the specific wording, not the concepts.

Overfitting is easy to spot by tracking training loss and validation loss separately during training. When training loss keeps falling while validation loss plateaus or rises, the model is overfitting:

import matplotlib.pyplot as plt
plt.plot(history["train_loss"], label="Train Loss")
plt.plot(history["val_loss"], label="Validation Loss")
plt.legend()
plt.title("Overfitting: val_loss diverging from train_loss")
plt.show()

Common remedies include dropout (randomly zeroing activations during training), L2 regularization (penalizing large weights), early stopping (halting training when validation loss stops improving), and data augmentation (artificially expanding the training set with transformations). For large language models, techniques like weight decay and RLHF serve analogous regularizing roles.

Overfitting is the flip side of Bias in the classic bias-variance tradeoff, and it is amplified when a model's Parameter count far exceeds the number of training examples. The Loss Function is the signal through which overfitting manifests, and Hyperparameter tuning — particularly learning rate, batch size, and regularization strength — is the main lever for controlling it.

You will encounter overfitting diagnostics in any supervised learning project, and understanding it is essential before deploying a model where distribution shift between training and production data is expected.