AI Model Wiki

    Skip to main content
    45 min
    Lesson 3 of 6

    Prompt Caching & Cost Optimization

    From the course Anthropic Claude API Deep Dive

    Prompt Caching

    Prompt caching reduces costs by up to 90% and latency by up to 85% for repeated prompt prefixes.

    How Caching Works

    When you mark content with cache control, Anthropic stores the processed tokens:

    const response = await client.messages.create({
      model: "claude-sonnet-4-20250514",
      max_tokens: 1024,
      system: [{
        type: "text",
        text: "You are an expert on the following documentation: [10,000 words of docs...]",
        cache_control: { type: "ephemeral" }
      }],
      messages: [{ role: "user", content: "How do I configure authentication?" }],
    });
    

    Cache Behavior

    • Minimum cacheable length: 1,024 tokens for Haiku, 2,048 for Sonnet/Opus
    • TTL: 5 minutes from last use (automatically refreshed on hit)
    • Pricing: Cache writes cost 25% more, but cache reads cost 90% less
    • Match requirement: Exact prefix match — any change invalidates the cache

    Cost Calculation Example

    For a 10,000-token system prompt called 100 times:

    • Without caching: 100 × 10,000 = 1M input tokens
    • With caching: 1 write (12,500 effective) + 99 reads (99,000 at 10% cost) = ~112K effective tokens
    • Savings: ~89%

    Optimization Strategies

    • Place static content (instructions, documentation) at the start of messages
    • Use caching for few-shot examples that don't change
    • Batch similar requests to maximize cache hits
    • Monitor cache hit rates in response headers

    Common Caching Pitfall

    The cache only applies to an exact prefix match. If you append the current date or user ID to your system prompt, you will never get a cache hit because the prefix changes on every request. Keep all variable content in the user messages array, and keep the system prompt truly static. Anthropic exposes cache usage in the response's usage field — log "cache_creation_input_tokens" and "cache_read_input_tokens" to verify hit rates before optimizing further.

    Real-World Savings Example

    A legal tech company processes 10,000 contracts per day, each requiring a 15,000-token system prompt describing extraction rules. Without caching: 10,000 x 15,000 = 150 million input tokens per day. With caching (one write + 9,999 reads at 10% cost): effectively ~16.5 million token-equivalents per day — an 89% reduction that saves thousands of dollars monthly on Sonnet pricing.

    Key Takeaways

    • •Prompt caching saves up to 90% on costs for repeated prefixes
    • •Mark cacheable content with cache_control: { type: 'ephemeral' }
    • •Cache has a 5-minute TTL that refreshes on each hit
    • •Place static content first in messages to maximize cache efficiency
    0 of 6 lessons marked complete

    Progress is kept in this browser only.