What it measures
User-facing latency. TTFT (time to first token) drives perceived snappiness; throughput drives total response time on long outputs.
How it works
Standardised prompts of varying length are sent to the model's API. TTFT measures wall-clock time from request to first streamed token; throughput is tokens emitted per second after streaming begins. Reported as median + p95.
Caveats
- Latency depends on provider's deployment, region, current load, and the model's batch policy — single readings are noisy.
- Same model can have very different latency on different gateways (Together, Fireworks, Groq, the provider's own API).
- Reasoning models that 'think' before answering have low throughput by design — comparing them to standard models is unfair without noting it.
Primary source
Artificial Analysis — Latency & throughput
Related benchmarks
- Cost benchmarks (price per million tokens) — Dollars per million input + output tokens, plus cached-input and batch discounts.
How we read this benchmark
Our Benchmark Methodology explains which benchmarks we cite on model pages, the caveats we apply, and why we never present any single benchmark as a verdict.