What it measures
Refusal robustness on dangerous queries and resistance to adversarial inputs.
How it works
Benchmarks like HarmBench, AdvBench, and JailbreakBench provide curated adversarial prompts and adversarial-suffix attacks. The metric is attack success rate (lower is safer) or refusal rate (higher is safer).
Caveats
- Refusal is bimodal: over-refusal (refusing legitimate questions) is its own failure mode and is rarely measured in 'safety' scores.
- Test sets are static and become exploitable. The same attack that gets a 95% pass rate today may be hard-coded out next month and the score drops to 0% without the underlying model changing.
- Safety scores cannot capture every harm modality (CSAM, weapons, election misinformation, privacy, etc.) — a single number elides important distinctions.
Primary source
Related benchmarks
- LMSYS Chatbot Arena — Blind pairwise human voting on real chat completions, aggregated into an Elo-style ranking.
How we read this benchmark
Our Benchmark Methodology explains which benchmarks we cite on model pages, the caveats we apply, and why we never present any single benchmark as a verdict.