An autoregressive model generates output one element at a time, where each new element is conditioned on all previously generated elements. In text generation, this means the model produces tokens sequentially: it samples the first token from a probability distribution over the entire vocabulary, then feeds that token back as context before sampling the second token, and so on until a stop condition is reached. This left-to-right factorisation of the joint probability over a sequence is the foundation of most large language models in use today.

Think of it like writing a sentence letter by letter, where each choice of the next letter depends on everything you have already written. You cannot go back and change an earlier letter without regenerating everything that follows it.

The mechanics become clear in pseudocode:

tokens = [BOS_TOKEN]  # begin-of-sequence marker
while tokens[-1] != EOS_TOKEN:
    logits = model(tokens)           # forward pass over full context
    next_token = sample(logits[-1])  # sample from last position
    tokens.append(next_token)

Each forward pass re-reads the entire sequence so far, which is why the Context Window is a hard constraint: once the sequence exceeds the window, earlier tokens must be dropped or the model cannot accept them. Temperature controls how sharply peaked the sampling distribution is at each step, and techniques like Beam Search explore multiple candidate sequences in parallel rather than greedily committing to one token at a time.

Autoregressive generation is closely linked to Tokenization (the vocabulary over which probabilities are defined), the Transformer architecture (the dominant implementation), and Inference cost (sequential generation is inherently difficult to parallelise across the time dimension, unlike batched matrix operations during Pretraining).

You will encounter autoregressive behaviour any time you observe a language model streaming output token by token, and it explains why generation latency scales with output length rather than being fixed per request.