Cost Management & Scaling
From the course Serverless AI Deployment
Controlling Serverless AI Costs
Cost Components
- Compute: Function execution time × memory
- API calls: AI provider charges per token
- Data transfer: Egress charges for responses
- Storage: Caching, queues, databases
Cost Optimization
Right-size memory:
- More memory = faster execution but higher cost per ms
- Profile your function to find the optimal memory/speed tradeoff
- AI proxy functions typically need 256-512MB
Minimize execution time:
- Use streaming to free the function as soon as the stream starts
- Cache frequent responses (Redis, CDN)
- Set timeouts to prevent runaway executions
Rate limiting:
// Simple in-memory rate limiter
const requests = new Map<string, number[]>();
function isRateLimited(userId: string, limit = 10, windowMs = 60000): boolean {
const now = Date.now();
const userRequests = requests.get(userId)?.filter(t => now - t < windowMs) || [];
if (userRequests.length >= limit) return true;
userRequests.push(now);
requests.set(userId, userRequests);
return false;
}
Common Pitfall: Not Separating Compute Costs from AI API Costs
Serverless billing dashboards show you Lambda/Edge function costs, but the AI provider charges (OpenAI tokens, Anthropic tokens) appear in a completely separate account and can be 10–100x larger than the compute costs. Teams routinely hit surprise $500–$2000 AI API bills because they only monitored Lambda spend. Tag all AI requests with a shared tracing ID and track token consumption in your own database alongside compute metrics.
Best Practice: Right-Size Memory Using Lambda Power Tuning
AWS Lambda's execution time and cost are both affected by memory allocation. Adding more memory speeds up the function (more CPU is allocated proportionally), and there is often a "sweet spot" where more memory reduces total cost because the function finishes much faster. Use the open-source AWS Lambda Power Tuning tool (a Step Functions state machine) to automatically benchmark your function at multiple memory settings and plot the cost-performance curve.
Named Product Reference: Upstash Redis for Serverless Rate Limiting
Upstash (upstash.com) provides serverless Redis with per-request pricing — no idle cost — making it ideal for serverless AI rate limiting. Their "@upstash/ratelimit" library offers sliding window, fixed window, and token bucket algorithms with a single SDK call. Unlike in-memory rate limiting (which resets per Lambda container), Upstash Redis is shared across all containers:
import { Ratelimit } from "@upstash/ratelimit";
import { Redis } from "@upstash/redis";
const ratelimit = new Ratelimit({
redis: Redis.fromEnv(),
limiter: Ratelimit.slidingWindow(10, "60 s"),
});
const { success } = await ratelimit.limit(userId);
if (!success) return new Response("Too Many Requests", { status: 429 });
Monitoring
- Track invocation count, duration, and errors
- Set billing alerts before hitting unexpected costs
- Monitor AI API usage separately from compute costs
- Use dashboards to visualize cost trends over time
Key Takeaways
- •Serverless costs come from compute, API calls, transfer, and storage
- •Right-size function memory for optimal cost-performance balance
- •Implement rate limiting to prevent abuse and cost overruns
- •Monitor and alert on both compute and AI API costs
Progress is kept in this browser only.