AI Model Wiki

    Skip to main content
    14 min read
    Lesson 3 of 5

    Guardrails, Safety & Content Moderation

    From the course Building AI Chatbots from Scratch

    Without guardrails, chatbots can be manipulated into producing harmful, off-brand, or incorrect outputs. Implementing multiple layers of defense is essential for production deployment.

    Defense in Depth

    Layer multiple safety mechanisms:

    Layer 1 — Input filtering:

    • Block or flag messages containing known harmful patterns
    • Detect prompt injection attempts
    • Rate limiting to prevent abuse
    • PII detection and redaction before processing

    Layer 2 — System prompt hardening:

    • Clear behavioral boundaries in the system prompt
    • Explicit refusal instructions for out-of-scope requests
    • Role anchoring ("You are a customer support agent, nothing else")
    • Instruction hierarchy (system prompt > user messages)

    Layer 3 — Output validation:

    • Check generated responses against content policies
    • Verify factual claims against knowledge base
    • Ensure responses don't contain PII or sensitive data
    • Format validation (no code execution instructions, no URLs to external sites)

    Layer 4 — Monitoring & feedback:

    • Log all conversations for review
    • Flag conversations with safety filter triggers
    • User reporting mechanisms
    • Regular adversarial testing

    Common Attack Vectors

    AttackDescriptionMitigation
    Prompt injectionUser tries to override system instructionsInput sanitization, instruction hierarchy
    JailbreakingTricks to bypass safety filtersMulti-layer defense, regular red teaming
    Data extractionAttempts to reveal training data or system promptNever include sensitive data in context
    Social engineeringBuilding rapport to gradually push boundariesTurn-level monitoring, conversation limits
    Automated abuseBots generating harmful content at scaleRate limiting, CAPTCHA, behavioral analysis

    Implementing Guardrails

    NeMo Guardrails (NVIDIA): Programmable safety rails

    # Define rails in Colang
define user ask about competitors
  "What do you think about [competitor]?"
  "Is [competitor] better than you?"

define bot refuse competitor comparison
  "I'm here to help with our products. For comparisons, I'd suggest checking independent review sites."

define flow
  user ask about competitors
  bot refuse competitor comparison

    Guardrails AI: Input/output validation framework LLM Guard: Open-source safety scanner for LLM inputs and outputs Azure AI Content Safety: Microsoft's content moderation API

    Conversation Limits

    Set boundaries to prevent abuse:

    • Maximum conversation length (e.g., 50 turns)
    • Maximum message length
    • Cooldown periods between conversations
    • Session timeouts for inactive conversations
    • Automatic escalation after repeated misunderstandings

    Key Takeaways

    • •Implement defense in depth: input filtering, prompt hardening, output validation, and monitoring
    • •Common attacks include prompt injection, jailbreaking, data extraction, and social engineering
    • •Tools like NeMo Guardrails and Guardrails AI provide programmable safety mechanisms
    • •Set conversation limits on length, rate, and duration to prevent abuse
    0 of 5 lessons marked complete

    Progress is kept in this browser only.