Caching Claude API Responses to Save Cost

Caching Claude API Responses to Save Cost

If you're sending the same system prompt, documents, or repeated queries to Claude on every request, you're paying for tokens you've already paid for. There are two distinct ways to fix this: prompt caching, which lets Claude skip re-processing large static context you reuse across calls, and response caching, which is a layer you build yourself to avoid calling the API at all when you already know the answer.

Both reduce cost, but they solve different problems. Prompt caching helps when your input is large and repeated (long system prompts, reference documents, tool definitions). Response caching helps when your output is deterministic enough that returning a stored answer is acceptable instead of generating a new one. Most production systems that care about cost use a combination of both.

Where the cost actually comes from

Claude pricing is per token, split between input and output. For many real workloads — RAG pipelines, document Q&A, agents with long tool schemas — the input side dominates because you're resending the same context on every turn. If your system prompt plus retrieved documents is 8,000 tokens and the user's actual question is 20 tokens, you're paying almost entirely for context that didn't change. That's the cost caching is meant to eliminate.

Native prompt caching

Anthropic's prompt caching lets you mark a block of your prompt (system instructions, long documents, tool definitions) with cache_control so Claude can reuse its internal processing of that block across requests instead of recomputing it every time.

curl https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-3-5-sonnet-20241022",
    "max_tokens": 1024,
    "system": [
      {
        "type": "text",
        "text": "You are a support agent. Here is the full product manual: ...",
        "cache_control": {"type": "ephemeral"}
      }
    ],
    "messages": [
      {"role": "user", "content": "How do I reset the device?"}
    ]
  }'

The first call writes to the cache and costs a bit more than a normal input token for that block. Every subsequent call that reuses the identical prefix within the cache lifetime reads from the cache instead, at a fraction of the normal input token price. The savings compound fast if you're making many calls per minute against the same large context — think customer support bots, document analysis tools, or agents with large tool schemas that don't change between calls.

The catch: the cached portion must match exactly, token for token, from the start of the prompt. If you insert dynamic content (today's date, a user's name) before the cached block, you break the cache hit. Structure your prompts so static content comes first and variable content comes last.

Application-level response caching

Prompt caching still calls the model. If a request is a true duplicate — same question, same context, deterministic enough that the answer won't meaningfully change — you don't need to call Claude at all. That's where your own response cache comes in.

A simple approach: hash the normalized request (model, messages, temperature, relevant parameters) and store the response keyed by that hash.

import crypto from "crypto";

function cacheKey(request) {
  const normalized = JSON.stringify({
    model: request.model,
    messages: request.messages,
    temperature: request.temperature ?? 0,
  });
  return crypto.createHash("sha256").update(normalized).digest("hex");
}

async function getCachedCompletion(request, cacheStore) {
  const key = cacheKey(request);
  const cached = await cacheStore.get(key);
  if (cached) return cached;

  const response = await callClaude(request);
  await cacheStore.set(key, response, { ttlSeconds: 3600 });
  return response;
}

This works well for:

• FAQ-style endpoints where the same question gets asked repeatedly
• Batch or report generation where inputs are identical across reruns
• Development and testing, where you don't want to burn real tokens re-running the same prompt while debugging

It works poorly for anything involving conversation history, user-specific data, or non-deterministic creative output where a stale cached answer is actively wrong. Set a sensible TTL and invalidate on any change to the underlying inputs — a stale cache is worse than no cache.

Semantic caching (use with caution)

A more aggressive variant matches on meaning rather than exact text, using embeddings to find near-duplicate past queries and returning a cached response if similarity exceeds a threshold. This catches "What's your refund policy?" and "How do refunds work?" as the same question.

It saves more tokens but introduces real risk: a false match returns a wrong answer with full confidence. Only use semantic caching where an occasional mismatch is low-stakes — general FAQ content, not anything involving pricing, legal terms, or user-specific account data.

Deciding what's worth caching

Before building a caching layer, look at your actual traffic:

1. Identify repeated prefixes — system prompts, tool schemas, long reference documents that don't change between calls. These are candidates for native prompt caching.
2. Identify repeated full requests — identical questions or jobs run more than once. These are candidates for response caching.
3. Identify what must never be cached — anything with user-specific context, live data, or where correctness depends on freshness.

If you're running multiple applications or team members against Claude through a single integration, it's worth having visibility into which keys and endpoints are generating the most token usage before you decide where to invest caching effort. SubToAPI exposes per-key usage metadata so you can see which application is driving cost, which makes it easier to prioritize caching work where it actually pays off rather than guessing. You can review the request/response format in the messages docs and get a key running in minutes via the quickstart.

Putting it together

A practical setup for a cost-conscious Claude integration looks like: static system prompts and large reference documents marked for native prompt caching, a response cache in front of any endpoint that sees repeated identical requests, and usage monitoring so you can confirm the caching is actually reducing spend rather than just assuming it is. None of this requires architectural complexity — it's a hashing function, a TTL, and the cache_control flag.

FAQ

Does caching reduce the quality of Claude's responses? No, native prompt caching doesn't change model behavior at all — it only affects how Claude processes a repeated prefix internally. Application-level response caching returns a previously generated answer verbatim, so quality only degrades if the cached answer is stale or the request wasn't actually a true duplicate.

How long does Anthropic's prompt cache last? Cached prompt prefixes have a short, limited lifetime measured in minutes, refreshed each time they're reused. It's designed for bursty, repeated traffic against the same context rather than long-term storage — for that, use your own response cache with an explicit TTL.

Should I cache streamed responses? Yes, but store the fully assembled output, not the stream itself. Build the complete response as it streams, cache that final text, and replay it as a single response (or re-stream it) on a cache hit. See the streaming docs for how streamed responses are structured if you're assembling them manually.