Grouped Query Attention (GQA) is a memory-efficient variant of the multi-head attention mechanism used in Transformer models. Standard multi-head attention assigns one separate key-value (KV) head for every query head, which means the KV cache grows proportionally with the number of attention heads and the sequence length — a major memory bottleneck during inference. GQA divides the query heads into groups and assigns one shared KV head to each group, so several query heads attend over the same key-value pairs. This reduces the KV cache by a factor equal to the number of query heads per group, cutting memory use substantially without the quality loss that comes from collapsing all query heads to a single KV pair (Multi-Query Attention). Llama 2 70B, Llama 3, and Mistral 7B all use GQA.

Think of it like a classroom where students (query heads) share reference copies of a textbook (key-value heads) rather than each student owning their own copy. Small groups share a single copy, so you need far fewer copies in total — but each student still has enough individual attention to grasp the material independently.

In a typical model config, GQA is expressed through the ratio of query heads to KV heads:

# Llama 3 8B uses 32 query heads and 8 KV heads (groups of 4)
config = {
    "num_attention_heads": 32,
    "num_key_value_heads": 8,   # GQA: 32 / 8 = 4 queries per KV group
    "hidden_size": 4096,
}

GQA is closely related to Self-Attention (the broader mechanism it modifies), Transformer (the architecture it appears in), Inference (KV cache memory is the primary bottleneck GQA addresses), and Context Window (longer contexts generate larger KV caches, amplifying GQA's savings). Quantization is a complementary technique that further reduces KV cache memory by lowering numerical precision.

You will encounter GQA when reading architecture cards for modern open-weight models, choosing between model variants for memory-constrained deployment, or configuring inference engines like vLLM and llama.cpp that expose KV cache tuning options.