Queue Patterns & Retry Logic
From the course AI Webhooks & Event-Driven Pipelines
Reliable AI Task Processing
Queue-Based Processing
// Producer: Add tasks to queue
async function enqueueAITask(task: AITask) {
await redis.lpush("ai-tasks", JSON.stringify({
id: crypto.randomUUID(),
...task,
createdAt: Date.now(),
attempts: 0,
}));
}
// Consumer: Process tasks from queue
async function processQueue() {
while (true) {
const item = await redis.brpop("ai-tasks", 30); // Block 30s
if (!item) continue;
const task = JSON.parse(item[1]);
try {
const result = await processAITask(task);
await storeResult(task.id, result);
await notifyClient(task.callbackUrl, result);
} catch (error) {
await handleFailure(task, error);
}
}
}
Retry with Exponential Backoff
async function handleFailure(task: any, error: Error) {
task.attempts++;
if (task.attempts >= 5) {
await moveToDeadLetterQueue(task, error);
return;
}
const delay = Math.pow(2, task.attempts) * 1000; // 2s, 4s, 8s, 16s
await redis.lpush("ai-tasks-delayed", JSON.stringify({ ...task, processAfter: Date.now() + delay }));
}
Dead Letter Queue
Failed tasks that exceed max retries go to a dead letter queue for:
- Manual investigation
- Pattern analysis (are certain tasks always failing?)
- Bulk reprocessing after fixing the underlying issue
Priority Queues
// High-priority tasks processed first
await redis.lpush("ai-tasks:high", task); // Premium users
await redis.lpush("ai-tasks:normal", task); // Free users
await redis.lpush("ai-tasks:low", task); // Background jobs
Common Pitfall: Setting Max Retries Too High on Rate-Limited Tasks
If the AI provider is rate-limiting requests (HTTP 429), retrying with exponential backoff is correct. But if you set maxRetries=10, a batch of 1,000 tasks can each retry up to 10 times — suddenly creating 10,000 API requests and amplifying the rate-limit problem. Distinguish between retryable errors (network failures, 5xx) and non-retryable ones (rate limits need coordinated backoff):
async function handleFailure(task: any, error: any) {
const isRateLimit = error.status === 429;
const maxRetries = isRateLimit ? 3 : 5;
if (task.attempts >= maxRetries) {
await moveToDeadLetterQueue(task, error);
return;
}
const baseDelay = isRateLimit ? 30000 : 1000; // 30s for rate limits, 1s for others
const delay = baseDelay * Math.pow(2, task.attempts);
await scheduleRetry(task, delay);
}
Best Practice: Add Jitter to Retry Delays
When many tasks fail simultaneously (e.g., during an API outage), fixed exponential backoff causes all retries to fire at the same moment — creating a "thundering herd" that overwhelms the recovering service. Add random jitter:
const jitter = Math.random() * 1000; // 0-1000ms random delay
const delay = Math.pow(2, task.attempts) * 1000 + jitter;
Named Product Reference: Inngest for Event-Driven AI Workflows
Inngest is a cloud service for event-driven workflows that handles retry logic, rate limiting, concurrency, and scheduling out of the box. You define workflow steps as TypeScript functions and Inngest manages the execution infrastructure. For AI workflows that need reliable multi-step execution with retries and human-in-the-loop steps, Inngest is a strong alternative to building a custom queue system.
Key Takeaways
- •Queues decouple producers from consumers for reliable processing
- •Retry with exponential backoff handles transient failures
- •Dead letter queues capture permanently failed tasks for investigation
- •Priority queues ensure important tasks are processed first
Progress is kept in this browser only.