OpenAI Error: rate_limit_exceeded — Too Many Requests

import openai

try:
    response = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello"}],
    )
except openai.RateLimitError as e:
    # e.status_code == 429
    # e.code == 'rate_limit_exceeded'
    # e.message includes 'Rate limit reached for ... in organization ...'
    retry_after = int(e.response.headers.get('Retry-After', '60'))

OpenAI’s rate_limit_exceeded is HTTP 429 with the structured error code rate_limit_exceeded. It’s the single most common production error developers hit when integrating OpenAI APIs, especially after a feature launch or traffic spike. The fix is almost never to retry harder — it’s to be smarter about when and how you call the API.

Why this happens

• Burst of concurrent requests. Most rate-limit hits come from sudden bursts — a deployment kicking off worker pods, a batch job retrying simultaneously, or a user-facing feature getting linked on Hacker News. The 60s rolling window means even a 1-second spike can trigger 429s for the rest of the minute.
• Token-per-minute (TPM) ceiling, not request count. OpenAI enforces both RPM (requests/min) and TPM (tokens/min). On `gpt-4o`, tier 1 is 500 RPM but 30,000 TPM. A few long prompts can blow TPM while RPM is still healthy. The error message tells you which limit you hit.
• Streaming requests counting toward concurrent limits. Streaming responses hold a connection open for the full generation. If you spawn many streaming requests in parallel, you can hit per-org concurrent limits before RPM/TPM, with the same `rate_limit_exceeded` code.
• Wrong tier for the model. Limits scale with usage tier (free, tier 1, tier 2, …, tier 5). New accounts on free tier have very low caps for `gpt-4o`. You can only move up tiers by paying and waiting; you can't request an exception.

How to fix it

Fixes are ordered by likelihood. Start with the first one that matches your context.

import time, random
from openai import OpenAI, RateLimitError

client = OpenAI()

def call_with_retry(messages, model="gpt-4o", max_attempts=5):
    for attempt in range(max_attempts):
        try:
            return client.chat.completions.create(
                model=model, messages=messages
            )
        except RateLimitError as e:
            if attempt == max_attempts - 1:
                raise
            retry_after = int(e.response.headers.get('Retry-After', 0))
            backoff = max(retry_after, 2 ** attempt)
            jitter = random.uniform(0, backoff * 0.25)
            time.sleep(backoff + jitter)

from limits import storage, strategies, parse
store = storage.MemoryStorage()
limiter = strategies.MovingWindowRateLimiter(store)
rule = parse("450/minute")  # 90% of tier-1 RPM cap

def allow(key="openai"):
    return limiter.hit(rule, key)

import tiktoken

def count_tokens(messages, model="gpt-4o"):
    enc = tiktoken.encoding_for_model(model)
    return sum(len(enc.encode(m["content"])) for m in messages) + 4 * len(messages)

Detection and monitoring in production

Track 429s as a metric, not just a log. Alarm if rate-limit errors exceed 1% of total requests over a 5-minute window. Add a tag for which model and which limit (RPM vs TPM) — they need different fixes. Send a Slack alert on tier-saturation events so you know to upgrade before users feel it.

Read more: /guide/handling-rate-limits/