What it measures
Competition-level mathematical reasoning that requires multi-step problem-solving and proof-like thinking.
How it works
AIME 2024/2025 questions, 15 each. Model produces an integer answer; exact match scores 1. Pass@N (e.g., pass@8) is reported when sampling multiple answers.
Caveats
- Tiny dataset (15-30 questions per year) — score variance is enormous.
- Sampling temperature and pass@k methodology dominate the result. A model scoring 60% pass@1 may show 95% pass@32.
- AIME problems concentrate on a narrow band of mathematical style — algebra, combinatorics, number theory. Performance doesn't generalise to applied math.
Primary source
Related benchmarks
- GPQA Diamond — Graduate-level science questions written to resist Google-search shortcuts.
- MATH-500 — 500 competition math problems across algebra, geometry, number theory, and pre-calculus.
How we read this benchmark
Our Benchmark Methodology explains which benchmarks we cite on model pages, the caveats we apply, and why we never present any single benchmark as a verdict.