AI Model Wiki

    Skip to main content
    14 min read
    Lesson 4 of 5

    Deploying & Scaling Your Chatbot

    From the course Building AI Chatbots from Scratch

    Getting a chatbot from prototype to production involves infrastructure decisions, integration challenges, and operational considerations.

    Deployment Options

    Widget embed: JavaScript snippet on your website

    • Lowest friction for users
    • Customizable UI matching your brand
    • Tools: Botpress, Voiceflow, custom React components

    Platform integration: Deploy where your users already are

    • Slack, Microsoft Teams, Discord bots
    • WhatsApp Business API, Facebook Messenger
    • SMS via Twilio
    • In-app messaging (Intercom, Zendesk)

    API-first: Expose your chatbot as an API

    • Other systems call your chatbot programmatically
    • Enables multi-channel deployment from a single backend
    • Best for custom integrations

    Architecture for Scale

    Load Balancer → API Gateway → Chat Service → LLM Provider
                                    ↓
                             Message Queue (async tasks)
                                    ↓
                        Knowledge Base / Vector Store
                                    ↓
                           Database (conversations)

    Key scaling considerations:

    • Stateless chat service: Store conversation state in database, not in memory
    • Queue long operations: Knowledge retrieval and LLM calls can be queued for resilience
    • Cache aggressively: Cache frequent knowledge base queries and common responses
    • Fallback providers: If OpenAI is down, route to Anthropic or a self-hosted model

    Performance Optimization

    • Streaming responses: Show tokens as they're generated (critical for UX)
    • Parallel retrieval: Fetch knowledge base results while classifying intent
    • Precomputed responses: Cache answers for the top 100 most common questions
    • Model selection routing: Use a smaller/faster model for simple queries, larger for complex ones

    Multi-Channel Consistency

    If your bot operates across channels:

    • Shared conversation state (user can start on web, continue on mobile)
    • Channel-appropriate formatting (rich cards on web, plain text on SMS)
    • Consistent personality and knowledge across all channels
    • Unified analytics across channels

    Key Takeaways

    • •Choose deployment based on user behavior: website widget, platform integration, or API-first
    • •Stateless architecture with database-backed conversation state enables horizontal scaling
    • •Stream responses, cache common queries, and route to appropriate model sizes for performance
    • •Multi-channel bots need shared state, consistent personality, and channel-appropriate formatting
    0 of 5 lessons marked complete

    Progress is kept in this browser only.