OpenAI Error: context_length_exceeded — Prompt Too Long

import openai

long_text = open("legal-contract.txt").read()  # ~140k tokens
try:
    response = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": long_text}],
        max_tokens=2000,
    )
except openai.BadRequestError as e:
    # e.status_code == 400
    # e.code == 'context_length_exceeded'
    # e.message: "This model's maximum context length is 128000 tokens.
    #             However, your messages resulted in 142537 tokens.
    #             Please reduce the length of the messages."
    ...

context_length_exceeded is OpenAI’s hard ceiling on how much text you can fit into a single API call. The model has a fixed context window — 128k tokens for gpt-4o, 1M for gpt-4.1, 200k for o1 — and your prompt plus the requested max_tokens for completion must fit inside that envelope. Cross it by even one token and the request is rejected with HTTP 400 before any generation starts.

The error message is unusually helpful: it tells you the exact model limit and the exact token count it computed for your messages. Use those numbers — log them, alert on them, drive auto-truncation from them. The cure for context_length_exceeded is almost always pre-flight token counting plus one of three strategies: switch to a bigger-window model, summarise older context, or chunk the work into multiple calls.

Why this happens

• Prompt alone exceeds the model's window. The most direct cause — you tried to send more input tokens than the model can hold. `gpt-4o` is 128k tokens (~96k English words). Even a long PDF, a transcript of a 30-min call, or a few thousand lines of code can push past that.
• Prompt + max_tokens combined exceeds the window. OpenAI counts both directions. If your prompt is 120k tokens on a 128k model and you ask for `max_tokens: 10000`, the request fails before generation starts. The total of input + max_tokens must fit, leaving you only 8k for output in this example.
• Long conversation history accumulating. Multi-turn chat apps that append every message to `messages[]` blow past the limit silently. By turn 30 of a deep conversation, you're routinely sending 50k-100k tokens of history just to ask a one-line follow-up.
• Wrong tokeniser assumed (character vs token mismatch). Developers often estimate at 4 characters per token. That's a rough average — code, JSON, URLs, and non-English languages tokenise denser, sometimes 1-2 chars per token. A `len(text) / 4` estimate is wildly optimistic for code-heavy prompts.
• Embedded images counted as tokens. Vision inputs add tokens based on resolution and detail level. A high-detail 2048x2048 image adds ~1,105 tokens. Send 10 images plus a long prompt and you'll easily clear 128k without realising.

How to fix it

Fixes are ordered by likelihood. Start with the first one that matches your context.

import tiktoken

def count_messages_tokens(messages, model="gpt-4o"):
    enc = tiktoken.encoding_for_model(model)
    tokens = 0
    for m in messages:
        tokens += 4  # per-message overhead
        tokens += len(enc.encode(m["content"]))
        tokens += len(enc.encode(m["role"]))
    return tokens + 2  # priming

MODEL_LIMITS = {"gpt-4o": 128_000, "gpt-4.1": 1_047_576, "gpt-3.5-turbo": 16_385}

def fits(messages, model, max_tokens=1000):
    return count_messages_tokens(messages, model) + max_tokens <= MODEL_LIMITS[model]

def route_by_length(messages):
    tokens = count_messages_tokens(messages)
    if tokens < 100_000:
        return "gpt-4o"
    if tokens < 200_000:
        return "o1"
    return "gpt-4.1"  # 1M context fallback

SUMMARY_EVERY_N_TURNS = 10
KEEP_RECENT_PAIRS = 6

def compress(history, model="gpt-4o-mini"):
    if len(history) <= KEEP_RECENT_PAIRS * 2:
        return history
    old = history[:-KEEP_RECENT_PAIRS * 2]
    recent = history[-KEEP_RECENT_PAIRS * 2:]
    summary = openai.chat.completions.create(
        model=model,
        messages=[{"role": "system", "content": "Summarise this conversation in <300 words, preserving facts and decisions."},
                  *old],
        max_tokens=400,
    ).choices[0].message.content
    return [{"role": "system", "content": f"Summary so far:\n{summary}"}, *recent]

Detection and monitoring in production

Log token counts on every successful response (`response.usage.prompt_tokens` and `completion_tokens`). Plot the p99 of `prompt_tokens` per endpoint over time — a creeping rise means conversation history isn't being trimmed and you'll hit `context_length_exceeded` soon. Add a pre-send guard that logs (and optionally rejects) any request over 90% of the model's limit.