A decoder-only architecture is a Transformer variant that retains only the masked self-attention stack, discarding the separate encoder. Each token can attend to all tokens that precede it but not to any that follow, which makes generation straightforward: the model is fed a prompt and then extends it one token at a time, each new token conditioned on all prior tokens. GPT, Claude, Llama, Mistral, and virtually every leading large language model released since 2020 use this design.

Think of it like writing in a journal where you can only read the pages you have already written. Each sentence you add is informed by everything before it, but you cannot peek ahead. This constraint is what makes the process generative rather than analytical.

In practice, running a decoder-only model for generation looks like this:

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-hf")
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")

inputs = tokenizer("The capital of France is", return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(output[0]))

The model autoregressively predicts each subsequent token until a stop condition is met.

Decoder-only models are contrasted with encoder-only architectures like BERT, which can see the full sequence at once and are better suited to classification and retrieval tasks. The Self-Attention mechanism is central to how the decoder reads its own previous outputs, and the Transformer backbone underpins both families. Temperature and other sampling settings control how the decoder samples from the predicted probability distribution at each step.

You will encounter decoder-only models any time you use a chat assistant, code completion tool, or text generation API, as they have become the dominant architecture for LLMs.