What it measures
Vision-language reasoning at undergraduate/graduate difficulty — reading charts, diagrams, photographs, and combining with textual context.
How it works
11,550 questions across 30 subjects in 6 disciplines (Art, Business, Health & Medicine, Humanities, Science, Tech & Engineering). Each question includes 1+ images and is multiple-choice or open-ended. Score is accuracy.
Caveats
- Heavy emphasis on academic-style diagrams. Doesn't measure practical vision tasks like UI screenshots or photos in the wild.
- Some answers can be inferred from text alone — pure-text models score higher than zero, suggesting image content isn't always required.
Primary source
Related benchmarks
- MMLU-Pro — Harder MMLU with 10 answer choices, dropped trivial questions, and stronger reasoning requirements.
- GPQA Diamond — Graduate-level science questions written to resist Google-search shortcuts.
How we read this benchmark
Our Benchmark Methodology explains which benchmarks we cite on model pages, the caveats we apply, and why we never present any single benchmark as a verdict.