AI Model Wiki

    Skip to main content
    Safety

    Safety benchmarks (HarmBench, AdvBench, etc.)

    Tests resistance to harmful instructions, jailbreaks, and prompt injection.

    What it measures

    Refusal robustness on dangerous queries and resistance to adversarial inputs.

    How it works

    Benchmarks like HarmBench, AdvBench, and JailbreakBench provide curated adversarial prompts and adversarial-suffix attacks. The metric is attack success rate (lower is safer) or refusal rate (higher is safer).

    Caveats

    • Refusal is bimodal: over-refusal (refusing legitimate questions) is its own failure mode and is rarely measured in 'safety' scores.
    • Test sets are static and become exploitable. The same attack that gets a 95% pass rate today may be hard-coded out next month and the score drops to 0% without the underlying model changing.
    • Safety scores cannot capture every harm modality (CSAM, weapons, election misinformation, privacy, etc.) — a single number elides important distinctions.

    Primary source

    HarmBench (centerforaisafety)

    Related benchmarks

    • LMSYS Chatbot Arena — Blind pairwise human voting on real chat completions, aggregated into an Elo-style ranking.

    How we read this benchmark

    Our Benchmark Methodology explains which benchmarks we cite on model pages, the caveats we apply, and why we never present any single benchmark as a verdict.