API rate limiting is the practice of capping how many requests a client can send to a model provider within a defined time window — for example, 60 requests per minute or 1 million tokens per day. When a caller exceeds the cap, the API returns an error (typically HTTP 429 Too Many Requests) rather than processing the call. Providers use rate limits to distribute capacity fairly across users, protect infrastructure stability, and prevent a single misbehaving application from degrading service for everyone else.

Think of it like a supermarket checkout lane with a maximum item count. If you arrive with 200 items when the limit is 20, you are redirected to a different queue or asked to come back later — your request is not simply ignored, but it is not processed immediately either.

In practice, well-written API clients handle 429 responses with exponential back-off: they wait a short period, retry, and double the wait on each successive failure. Here is a minimal example:

import time, requests

def call_with_backoff(payload, max_retries=5):
    wait = 1
    for attempt in range(max_retries):
        r = requests.post(API_URL, json=payload, headers=HEADERS)
        if r.status_code == 429:
            time.sleep(wait)
            wait *= 2
        else:
            return r.json()
    raise RuntimeError("Rate limit retries exhausted")

Rate limits interact directly with Inference cost management, Tokenization (limits are often token-based rather than request-based), and Context Window size (a single long-context request consumes more of a token-per-minute quota than a short one). Higher pricing tiers on most platforms — whether for an LLM or a Multimodal model — primarily differ from free tiers in their rate limit ceiling rather than in model quality.

You will encounter rate limiting when building production applications against any commercial AI API, and the provider dashboard or HTTP response headers (often X-RateLimit-Remaining) are the fastest way to monitor how close you are to the cap.