AI Model Wiki

    Skip to main content
    Agents

    Terminal-Bench

    Agentic tasks completed in a real shell — installing, debugging, running multi-step workflows.

    What it measures

    End-to-end shell competence: file manipulation, package installation, multi-step workflows, error recovery.

    How it works

    Each task gives the agent a goal and a containerised Linux environment. The agent issues shell commands; success is graded by a test harness. Score is task-completion rate.

    Caveats

    • Heavy dependence on agent harness — same model can score 30% or 60% depending on tool wrapper.
    • Tasks skew toward Linux + Python. Limited coverage of other ecosystems.
    • Currently a small dataset; absolute scores will move as the benchmark grows.

    Primary source

    Terminal-Bench (terminal-bench.org)

    Related benchmarks

    • AgentBench — Multi-environment benchmark for LLM-driven agents — OS, DB, web, games, scientific.
    • SWE-Bench Verified — 500-instance cleaned subset of SWE-Bench with verified test reliability.

    How we read this benchmark

    Our Benchmark Methodology explains which benchmarks we cite on model pages, the caveats we apply, and why we never present any single benchmark as a verdict.