Backpropagation is the algorithm that computes the gradient of a loss function with respect to every parameter in a neural network by systematically applying the chain rule of calculus from the output layer back toward the input layer. During a forward pass, the network produces a prediction; the loss function measures the discrepancy between that prediction and the correct answer. Backpropagation then determines how much each weight contributed to that discrepancy, producing a gradient that Gradient Descent uses to adjust the weights in the direction that reduces the loss.

Think of it like tracing responsibility for a failed project back through an organisation. The final failure is visible at the top, but backpropagation works its way down through every layer of management and staff to assign each person a precise share of the blame — and a correspondingly sized corrective action.

A simplified view of one backward step for a two-layer network:

Forward:  x → [W1, b1] → hidden → [W2, b2] → output → loss
Backward: d_loss/d_W2 = d_loss/d_output * d_output/d_W2
          d_loss/d_W1 = d_loss/d_hidden * d_hidden/d_W1  (chain rule)

Modern deep learning frameworks (PyTorch, JAX, TensorFlow) implement automatic differentiation so engineers rarely write backprop code manually — a call to loss.backward() triggers the whole computation graph traversal. The Batch Size of each training step determines how many examples contribute to the gradient estimate before weights are updated.

Backpropagation is inseparable from Gradient Descent (it computes the gradients that descent uses), the Loss Function (it propagates that function backward), and the Activation Function (which must be differentiable, or approximated as such, for gradients to flow through). Overfitting, Hyperparameter tuning, and the entire discipline of Pretraining large models all depend on backprop running correctly and efficiently at scale.

You will encounter backpropagation in any discussion of neural network training, and understanding it is essential for diagnosing vanishing or exploding gradients during model development.