Claude API Prompt Caching Best Practices

Prompt caching lets you reuse large, unchanging chunks of a prompt — system instructions, documents, tool definitions — across multiple API calls without paying full price or full latency each time. The best practice summary is: cache the stable, reusable parts of your prompt (not the parts that change per request), place cache breakpoints at the right boundaries, and structure requests so the cached prefix is byte-identical across calls.

If you're asking this question, you're probably trying to cut Claude API costs on an app that sends the same large context repeatedly — a RAG pipeline with a long document, a chatbot with a big system prompt, or an agent with a large toolset. Below is a practical rundown of what to cache, how to structure it, and what commonly breaks caching in production.

How Prompt Caching Actually Works

Caching works by hashing a prefix of your prompt (system message, tool definitions, early conversation turns) and storing the processed version server-side for a short window. On the next request, if the prefix matches exactly, Claude skips re-processing that portion and you get a cache read instead of a full compute pass. Cache reads are cheaper than fresh input tokens, and they reduce latency because less of the prompt needs to be re-evaluated.

The catch: caching is prefix-based and exact-match. Any change — a single character, a reordered field, a different timestamp — invalidates the cache for everything after that point. This is the single biggest source of "my caching isn't working" complaints.

What to Cache

Cache content that is:

• Large — caching a 50-token system prompt isn't worth the overhead; caching a 10,000-token document or tool spec is.
• Reused across many requests — static system instructions, reference documents, few-shot examples, tool/function definitions.
• Stable within a session or longer — it doesn't need to be permanent, but it should repeat identically across several calls.

Good candidates:

• Long system prompts with detailed instructions, style guides, or persona definitions
• Reference documents in RAG or document-QA setups (the document itself, not the user's question)
• Few-shot examples used to steer output format
• Large tool/function definition blocks in agentic workflows

Bad candidates:

• The user's current message or query
• Timestamps, request IDs, or any dynamically generated content inside the "cached" portion
• Short prompts where the processing overhead of caching outweighs the savings

Structuring Your Prompt for Cache Hits

The order of content in your prompt determines cache boundaries. Put static content first, dynamic content last:

[system prompt - static]
[reference document - static]
[few-shot examples - static]
---- cache breakpoint ----
[conversation history - grows over time]
[current user message - always changes]

Everything before the breakpoint must be byte-identical across requests for the cache to hit. A common mistake is interleaving static and dynamic content — for example, injecting a live timestamp into the system prompt. That one field invalidates caching for the entire prefix, even if the rest of the system prompt is 5,000 tokens of unchanged instructions.

If you need dynamic metadata (user ID, session ID, current date), push it to the end of the prompt, after the cached portion, not inside it.

Multi-Turn Conversations

In chat applications, each new turn appends to the conversation. If you cache the system prompt plus the first few turns, every subsequent request reuses that cached prefix and only pays full price for the newest turns. As the conversation grows, you can periodically move older turns into a new cached checkpoint to avoid re-caching an ever-growing history on every call.

Document-Heavy Workflows

For RAG or document QA, put the full document block immediately after the system prompt and before the user's question. Cache the system prompt + document together. If users ask multiple questions about the same document in sequence, this turns what would be N full-document passes into one full pass plus N cheap cache reads.

Cache Lifetime and Expiration

Cached prefixes expire after a short period of inactivity — they're meant for active, repeated use within a session or burst of requests, not long-term storage. Best practice: don't rely on caching for content you touch once a day. It's built for workloads with tight repetition — multiple calls within minutes, not once-daily batch jobs.

If your traffic pattern is bursty (lots of requests, then silence, then more requests), expect to pay the full "cache write" cost again after each gap. Caching pays off most for high-frequency, high-reuse traffic: chatbots mid-conversation, agents making repeated tool calls, batch jobs processing many items against the same reference document in quick succession.

Common Mistakes

• Dynamic content inside the cached block. Timestamps, random IDs, or user-specific data placed before the cache breakpoint silently kill your hit rate.
• Caching content that's too small. The overhead isn't worth it below a few hundred tokens.
• Reordering JSON keys or whitespace differences. Even non-semantic formatting changes break exact-match hashing. Serialize deterministically.
• Not measuring cache hit rate. Without visibility into cache reads vs. writes, you won't know if your caching strategy is actually working — you're guessing at savings.
• Assuming caching is "free" latency reduction with no setup. You still need a correct prefix structure; it doesn't happen automatically.

Measuring Whether It's Working

Every response includes usage metadata showing cache creation and cache read token counts. Track this per request type. If you're building on top of Claude and routing through a gateway like SubToAPI, that per-request usage metadata — including cache reads vs. fresh tokens — shows up in the request logs and usage dashboard automatically, so you can verify caching is actually reducing cost rather than assuming it is. See /docs/messages for request/response structure and /docs/streaming if you're combining caching with streamed responses.

A Simple Mental Model

Before optimizing further, ask three questions about your prompt:

1. What part is identical across many calls? → cache it.
2. What part changes every call? → put it after the cache breakpoint.
3. How often do calls repeat within a short window? → if rarely, caching won't help much.

Get these three right and prompt caching becomes a straightforward cost and latency win rather than a source of confusing, inconsistent savings.

questions

Does reordering the same content break the cache? Yes. Caching matches on exact byte sequence, so reordering fields, changing whitespace, or regenerating JSON with different key order invalidates the cached prefix even if the content is semantically identical.

Is prompt caching worth it for small system prompts? Generally no. Caching has setup overhead, so it pays off for large, reused blocks (documents, long instructions, tool definitions) — not short prompts under a few hundred tokens.

How do I know if my caching is actually reducing costs? Check the cache creation and cache read token counts in each response's usage metadata. If cache reads are consistently low relative to total requests, your prefix structure likely isn't stable across calls.