12 min read
Lesson 5 of 6
Continuous Evaluation in Production
From the course Evaluating & Benchmarking AI Models
Evaluation doesn't stop at deployment. Production environments surface behaviors that offline testing can't predict.
Why Continuous Evaluation?
- Distribution shift: Real user queries differ from your test set
- Model updates: Provider model updates can change behavior without notice
- Emergent patterns: New use patterns appear as your user base grows
- Adversarial users: Real attackers are more creative than your test suite
Production Evaluation Techniques
Sampling and review: Randomly sample 1-5% of production interactions for human review. Flag conversations with:
- Low user satisfaction signals (thumbs down, repeated questions, session abandonment)
- High-risk topics (medical, legal, financial advice)
- Unusual patterns (very long conversations, rapid tool calls)
Automated quality checks: Run LLM-as-judge on production outputs to catch:
- Hallucinations (claim verification against knowledge base)
- Safety violations (content policy adherence)
- Format compliance (JSON schema validation, tone consistency)
User feedback loops:
- Thumbs up/down ratings on responses
- Free-text feedback forms
- Implicit signals (copy/paste behavior, follow-up questions, task completion)
Alerting
Set up alerts for:
- Quality score drops below threshold (rolling average)
- Error rate spikes (failed tool calls, API errors)
- Safety filter trigger rate increases
- Latency degradation
Dashboard Metrics
Track these daily:
| Metric | Target | Alert Threshold |
|---|---|---|
| User satisfaction (thumbs up %) | >85% | <75% |
| Hallucination rate | <5% | >10% |
| Safety violation rate | <0.1% | >0.5% |
| Average response latency | <3s | >5s |
| Task completion rate | >70% | <50% |
Feedback → Improvement Loop
- Collect production failures and low-quality outputs
- Add them to your evaluation dataset
- Use them as fine-tuning data or prompt improvements
- Deploy improvements and verify with the expanded eval set
- Monitor production metrics for improvement
Key Takeaways
- •Production evaluation catches distribution shift and emergent behaviors that offline testing misses
- •Sample 1-5% of production interactions for human review, prioritizing high-risk and low-satisfaction cases
- •Set up automated quality checks and alerting for hallucination rate, safety violations, and latency
- •Feed production failures back into evaluation datasets and training data for continuous improvement
0 of 6 lessons marked complete
Progress is kept in this browser only.