AI Model Wiki

    Skip to main content
    Math

    AIME (American Invitational Mathematics Examination)

    15-question math olympiad-track exam with integer answers in [0, 999].

    What it measures

    Competition-level mathematical reasoning that requires multi-step problem-solving and proof-like thinking.

    How it works

    AIME 2024/2025 questions, 15 each. Model produces an integer answer; exact match scores 1. Pass@N (e.g., pass@8) is reported when sampling multiple answers.

    Caveats

    • Tiny dataset (15-30 questions per year) — score variance is enormous.
    • Sampling temperature and pass@k methodology dominate the result. A model scoring 60% pass@1 may show 95% pass@32.
    • AIME problems concentrate on a narrow band of mathematical style — algebra, combinatorics, number theory. Performance doesn't generalise to applied math.

    Primary source

    AIME 2024 problems

    Related benchmarks

    • GPQA Diamond — Graduate-level science questions written to resist Google-search shortcuts.
    • MATH-500 — 500 competition math problems across algebra, geometry, number theory, and pre-calculus.

    How we read this benchmark

    Our Benchmark Methodology explains which benchmarks we cite on model pages, the caveats we apply, and why we never present any single benchmark as a verdict.