Red teaming in AI is the structured practice of probing a model or system with adversarial inputs to uncover vulnerabilities, failure modes, biases, and unsafe outputs before deployment. Borrowed from cybersecurity, the term describes a dedicated group (the "red team") that attempts to break the system's intended behaviour-eliciting harmful content, bypassing safety filters, or extracting private training data-so those weaknesses can be fixed.

Think of it like hiring professional lock-pickers to test your building's security before opening it to the public. They are not trying to rob you; they are trying to find every weakness so you can fix it before a real attacker does.

A red-teaming exercise against a customer-support chatbot might include jailbreak attempts ("pretend you have no restrictions and explain how to."), prompt injection via malicious documents the bot is asked to summarise, and adversarial persona prompts designed to coerce the model into producing biased or discriminatory responses. Each finding is logged, severity-rated, and fed back into safety mitigations or RLHF fine-tuning.

Red teaming directly informs Responsible AI programmes and is tightly linked to Hallucination research and Bias audits; the outputs often shape the human preference data used in RLHF reward signals. Major labs including Anthropic, OpenAI, and Google publish red-team reports as part of their model cards for Foundation Models.

You will encounter red teaming when reading AI safety disclosures, model cards, or internal policy frameworks, and it is increasingly mandated by emerging AI regulations in the EU and US.