Human-in-the-Loop & Production
From the course AutoGen: Multi-Agent Conversations
Human-AI Collaboration
Human Input Modes
# Always ask human before acting
user_proxy = UserProxyAgent(human_input_mode="ALWAYS")
# Ask only when termination condition met
user_proxy = UserProxyAgent(human_input_mode="TERMINATE")
# Fully automated
user_proxy = UserProxyAgent(human_input_mode="NEVER")
Termination Conditions
Control when conversations end:
user_proxy = UserProxyAgent(
max_consecutive_auto_reply=5,
is_termination_msg=lambda msg: "TASK COMPLETE" in msg.get("content", ""),
)
Production Architecture
State Management:
- Save conversation state for resuming interrupted workflows
- Log all agent messages for auditing and debugging
- Track token usage and costs per conversation
Error Handling:
- Catch and retry failed API calls
- Handle code execution failures gracefully
- Set global timeouts for entire workflows
- Implement fallback agents for critical tasks
Scaling Considerations
- Use async agents for concurrent conversations
- Implement request queuing for rate-limited APIs
- Cache agent responses for repeated queries
- Monitor conversation quality with automated evaluations
AutoGen vs. Alternatives
| Feature | AutoGen | CrewAI | LangGraph |
|---|---|---|---|
| Focus | Conversation | Role-based | State machines |
| Code execution | Built-in | Via tools | Via tools |
| Human-in-loop | Native | Limited | Native |
| Complexity | Medium | Low | High |
Common Pitfall: Not Logging Agent Messages for Debugging
Multi-agent conversations are hard to debug without full logging. AutoGen conversations can be inspected via groupchat.messages after they complete, but this is lost if the process crashes. Log every message to a database or file during the run:
def log_message(sender, recipient, message):
db.insert("agent_logs", {
"sender": sender.name,
"recipient": recipient.name if recipient else "group",
"content": message.get("content", ""),
"timestamp": datetime.now().isoformat(),
})
# Register the logging hook
for agent in [assistant, user_proxy]:
agent.register_hook("process_message_before_send", log_message)
Best Practice: Budget Tokens per Conversation
In production, set a token budget per conversation to prevent runaway API costs. AutoGen supports per-model token tracking through the llm_config cache mechanism. Monitor the total_tokens_used from the LLM config after each run and alert when it exceeds a threshold:
result = user_proxy.initiate_chat(assistant, message="Build a data pipeline")
# Check token usage after conversation ends
usage = user_proxy.chat_messages[assistant][-1].get("usage", {})
print(f"Total tokens used: {usage.get('total_tokens', 'unknown')}")
Named Product Reference: AutoGen v0.4 AgentChat
Microsoft's AutoGen v0.4 introduced a new agentchat package with a redesigned async-first API. If you are starting a new project, use pip install autogen-agentchat (v0.4) rather than the older pyautogen package. The new API supports async agents, better streaming, and a more modular architecture — the legacy package is in maintenance mode.
Key Takeaways
- •Three human input modes: ALWAYS, TERMINATE, and NEVER
- •Set termination conditions with message patterns and reply limits
- •Log all agent interactions for auditing and debugging
- •AutoGen excels at conversational multi-agent patterns with human oversight
Progress is kept in this browser only.