A Graphics Processing Unit is a processor built around thousands of small, parallel arithmetic cores rather than a handful of powerful sequential ones. Originally designed to rasterize pixels simultaneously across a screen, the GPU's architecture proved ideal for the matrix-multiplication-heavy workloads at the heart of deep learning — both training (computing gradients across millions of parameters) and inference (running forward passes on large models). NVIDIA's CUDA ecosystem, introduced in 2007, became the dominant software layer that allowed researchers to repurpose GPU hardware for general-purpose compute, and NVIDIA chips — A100, H100, H200, Blackwell B200 — now define the industry benchmark.

Think of it like comparing a small team of specialist surgeons (CPU cores: few, highly skilled, one complex task at a time) against a stadium of assembly-line workers (GPU cores: thousands, simpler tasks, all at once). Neural network math is inherently parallel, so the stadium wins by a factor of 100× or more on throughput.

In PyTorch, moving tensors to a GPU is a single call, and the entire backpropagation graph executes on-device:

import torch
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = MyModel().to(device)
inputs = inputs.to(device)
outputs = model(inputs)  # forward + backward both run on GPU

GPU compute is directly tied to Gradient Descent (each optimizer step requires GPU throughput), Quantization (reducing numerical precision to fit more computation in GPU memory), Inference (real-time serving bottleneck), and Parameter count (larger models demand more GPU VRAM and more GPUs in parallel).

You will encounter GPU discussions whenever reading cloud pricing for AI workloads, evaluating hardware for local model inference, or planning distributed training setups where inter-GPU bandwidth (NVLink, InfiniBand) becomes as important as raw FLOPS.