Mamba is a sequence modeling architecture built on selective state space models (SSMs) rather than the self-attention mechanism used in Transformers. Unlike attention-based models, whose memory and compute costs grow quadratically with sequence length, Mamba scales linearly, making it practical for processing very long sequences without the computational explosion that limits standard Transformers. Its "selective" mechanism means the model learns which parts of the input to remember or forget based on content, rather than attending to every position equally.

Think of it like a conveyor belt with a smart filter: as tokens pass through, the model dynamically decides which information to retain in its rolling summary and which to discard, so the summary stays informative rather than bloated.

A practical illustration: when summarizing a 100,000-token document, a Transformer must compute attention across every token pair — a prohibitively expensive operation. Mamba instead maintains a compact hidden state that updates as each token arrives:

# Mamba processes sequences recurrently at inference,
# but trains in parallel like a Transformer
model = MambaLMHeadModel.from_pretrained("state-spaces/mamba-2.8b")
output = model.generate(input_ids, max_new_tokens=200)

This makes inference fast even on sequences that would exceed a Transformer's Context Window budget entirely.

Mamba is closely related to the broader family of Neural Network sequence models and is often compared to the Transformer as an alternative backbone. It shares design goals with concepts like Attention (which it replaces) and connects to ideas in Inference efficiency and Quantization for deployment.

You will encounter Mamba most often in research contexts exploring alternatives to Transformers for long-document tasks, genomics, audio modeling, and any domain where sequence lengths regularly exceed tens of thousands of tokens.