PagedAttention is a memory management algorithm for the key-value (KV) cache used during transformer inference, developed by the vLLM team at UC Berkeley. During autoregressive text generation, each generated token requires storing key and value vectors for every previous token in the sequence, and this cache grows linearly with sequence length. Traditional implementations pre-allocate a contiguous block of GPU memory per request, which wastes capacity through internal and external fragmentation. PagedAttention borrows the paging concept from operating system virtual memory management: the KV cache is divided into fixed-size blocks (pages) that are allocated on demand and can be stored non-contiguously in GPU memory, with a block table mapping logical sequence positions to physical memory locations.

Think of it like managing a library's stacks using a card catalogue instead of reserving one entire shelf per patron. Instead of blocking off a continuous run of shelves for each borrower (most of which sit empty), the librarian records which scattered shelves hold each patron's books and retrieves them by consulting the index — the physical storage is shared and filled efficiently.

The practical effect is dramatic: vLLM with PagedAttention achieves near-zero KV cache waste (typically less than 4% fragmentation versus up to 60–80% with contiguous allocation), which allows a single GPU to serve many more concurrent inference requests. This directly improves throughput without changing model quality:

# Launch a vLLM server that uses PagedAttention internally
python -m vllm.entrypoints.openai.api_server   --model meta-llama/Llama-3-8B   --max-model-len 8192   --gpu-memory-utilization 0.90

The block table also enables prefix caching, where shared prompt prefixes across requests reuse already-computed KV pages, cutting latency for templated or system-prompt-heavy workloads.

PagedAttention is most relevant alongside Attention and Transformer architecture concepts, since it specifically optimizes the KV cache generated by self-attention layers. It also intersects with Context Window handling, as longer contexts amplify exactly the memory fragmentation problem PagedAttention solves.

You will encounter PagedAttention when deploying high-throughput LLM serving systems, particularly with vLLM, SGLang, or similar inference frameworks designed for production workloads.