What it measures
Breadth of mathematical reasoning across multiple competition styles, less variance than AIME.
How it works
A 500-question subset of the MATH dataset. Models output LaTeX answers; equivalence is checked. Score is accuracy.
Caveats
- Some of the original MATH benchmark is in pre-training data — frontier scores may be inflated.
- Answer-equivalence checking is imperfect; valid alternate forms can be marked wrong.
Primary source
HendrycksMATH paper (arXiv:2103.03874)
Related benchmarks
- AIME (American Invitational Mathematics Examination) — 15-question math olympiad-track exam with integer answers in [0, 999].
- GPQA Diamond — Graduate-level science questions written to resist Google-search shortcuts.
How we read this benchmark
Our Benchmark Methodology explains which benchmarks we cite on model pages, the caveats we apply, and why we never present any single benchmark as a verdict.