AI Model Wiki

    Skip to main content
    Multimodal

    MMMU (Massive Multi-discipline Multimodal Understanding)

    11.5k multimodal questions across 30 college subjects, requiring image + text reasoning.

    What it measures

    Vision-language reasoning at undergraduate/graduate difficulty — reading charts, diagrams, photographs, and combining with textual context.

    How it works

    11,550 questions across 30 subjects in 6 disciplines (Art, Business, Health & Medicine, Humanities, Science, Tech & Engineering). Each question includes 1+ images and is multiple-choice or open-ended. Score is accuracy.

    Caveats

    • Heavy emphasis on academic-style diagrams. Doesn't measure practical vision tasks like UI screenshots or photos in the wild.
    • Some answers can be inferred from text alone — pure-text models score higher than zero, suggesting image content isn't always required.

    Primary source

    MMMU benchmark site

    Related benchmarks

    • MMLU-Pro — Harder MMLU with 10 answer choices, dropped trivial questions, and stronger reasoning requirements.
    • GPQA Diamond — Graduate-level science questions written to resist Google-search shortcuts.

    How we read this benchmark

    Our Benchmark Methodology explains which benchmarks we cite on model pages, the caveats we apply, and why we never present any single benchmark as a verdict.