AI Model Wiki

    Skip to main content
    Agents

    AgentBench

    Multi-environment benchmark for LLM-driven agents — OS, DB, web, games, scientific.

    What it measures

    Generalisation across heterogeneous agent tasks rather than a single task family.

    How it works

    8 environments (OS, DB, knowledge graph, digital card game, lateral thinking puzzles, household, web shopping, web browsing). Each environment has its own success metric; final score is normalised average.

    Caveats

    • Average across very different tasks is hard to interpret. Strong on web tasks, weak on OS, etc.
    • Some environments are gameable with simple heuristics; AgentBench score does not perfectly correlate with real-world agent quality.

    Primary source

    AgentBench paper (arXiv:2308.03688)

    Related benchmarks

    • Terminal-Bench — Agentic tasks completed in a real shell — installing, debugging, running multi-step workflows.
    • SWE-Bench Verified — 500-instance cleaned subset of SWE-Bench with verified test reliability.

    How we read this benchmark

    Our Benchmark Methodology explains which benchmarks we cite on model pages, the caveats we apply, and why we never present any single benchmark as a verdict.