How to Cache Claude API Responses Efficiently

Caching Claude API responses efficiently means combining two distinct layers: Anthropic's native prompt caching, which reduces the cost of repeated input tokens, and application-level response caching, which skips the API call entirely when you already have the answer. Most teams only implement one of these and leave money and latency on the table.

The short version: cache the static parts of your prompt (system instructions, long context, tool definitions) with cache_control, and cache full responses on your own infrastructure keyed by a hash of the normalized request. Below is how to do both correctly, including the mistakes that quietly break caching in production.

Why caching matters for cost and latency

Every uncached call to Claude pays full price for every input token, and every response takes a full round trip through the model. If your app sends the same system prompt, the same document context, or the same user question repeatedly, you're paying for compute you've already paid for once. For high-traffic endpoints — chatbots, support tools, search assistants — this adds up fast and directly affects your API bill and p95 latency.

Layer 1: Prompt caching

Claude supports prompt caching through the cache_control field on content blocks. When you mark a block as cacheable, Anthropic stores the processed version of that prefix for a short window (typically five minutes, refreshed on each hit), so subsequent requests that reuse the same prefix skip reprocessing it.

This is most useful for:

• Long system prompts that don't change between requests
• Large context documents (PDFs, codebases, knowledge base chunks) reused across many user questions
• Tool definitions that stay constant across a session

{
  "model": "claude-opus-4",
  "system": [
    {
      "type": "text",
      "text": "You are a support assistant. Here is the full product manual: ...",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [
    { "role": "user", "content": "How do I reset my password?" }
  ]
}

The key rule: the cached prefix must be byte-identical across requests. Reordering fields, changing whitespace, or injecting a timestamp into the system prompt invalidates the cache silently — you'll still get a response, just at full price. Keep variable content (user messages, timestamps, session IDs) out of the cached block and append it afterward.

Layer 2: Application-level response caching

Prompt caching reduces cost per call. Response caching eliminates the call entirely. If a user asks the exact same question twice, or your app regenerates the same summary for the same document, there's no reason to hit Claude again.

The pattern is simple: hash a normalized representation of the request, check a store (Redis, a KV store, even an in-memory LRU for low-traffic cases) before calling the API, and write the response back after a successful call.

import crypto from "crypto";
import Redis from "ioredis";

const redis = new Redis(process.env.REDIS_URL);

function cacheKey(model, systemPrompt, messages) {
  const normalized = JSON.stringify({ model, systemPrompt, messages });
  return "claude:" + crypto.createHash("sha256").update(normalized).digest("hex");
}

async function getCachedOrFetch(model, systemPrompt, messages, callClaude) {
  const key = cacheKey(model, systemPrompt, messages);
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  const response = await callClaude(model, systemPrompt, messages);
  await redis.set(key, JSON.stringify(response), "EX", 3600); // 1 hour TTL
  return response;
}

A few details that matter more than they look:

• Normalize before hashing. Trim whitespace, sort object keys if your serialization isn't deterministic, and strip request-specific metadata like timestamps or request IDs before hashing.
• Include the model name in the key. A response from claude-opus-4 and claude-haiku-4 for the same prompt are not interchangeable.
• Don't cache non-deterministic outputs blindly. If temperature > 0, two identical requests can legitimately produce different valid answers. Caching is safest for low-temperature, deterministic-style tasks like classification, extraction, or FAQ answering — exactly the use cases where caching is also most profitable.

Deduplicating concurrent requests

A cache miss under load can cause the classic "thundering herd" problem: ten identical requests arrive at once, all miss the cache, and all hit the API simultaneously. Guard against this with an in-flight request map so only one actual call happens per unique key:

const inFlight = new Map();

async function dedupedFetch(key, fn) {
  if (inFlight.has(key)) return inFlight.get(key);
  const promise = fn().finally(() => inFlight.delete(key));
  inFlight.set(key, promise);
  return promise;
}

Caching with streaming responses

Streaming complicates caching because you don't have a full response object until the stream closes. The usual approach is to buffer the stream server-side, store the complete assembled response once it finishes, and still stream it to the end user token by token on the first request. On a cache hit, you can replay the stored response as a synthetic stream (chunked by whitespace or sentence) so the client-side experience stays consistent. If you're building this through SubToAPI, the streaming endpoints documented at /docs/streaming give you clean SSE output you can buffer and cache the same way.

Setting sane TTLs and invalidation rules

• Static knowledge (docs, policies): cache for hours or days, invalidate when the source document changes.
• User-specific but repeatable queries (account lookups, FAQ answers): cache for minutes to an hour.
• Anything involving live data (stock prices, current time, user balances): don't cache the response, but still use prompt caching for the static instruction portion.

Tag cache entries with a version string tied to your prompt template. When you edit the system prompt, bump the version — old cache entries become unreachable automatically instead of serving stale instructions under a new prompt.

Where SubToAPI fits

If you're exposing Claude to your own app or team through a REST layer, the usage metadata returned on every request (see /docs/messages) makes it straightforward to measure how much a caching layer is actually saving — compare token counts before and after you turn it on. Getting an API key and testing this against your own traffic takes a few minutes at /signup, and the full request/response reference is in /docs if you want to see exactly what fields are available to build your cache keys from.

Questions

Does prompt caching and application-level caching conflict with each other? No — they operate at different layers. Prompt caching reduces the cost of a call that still happens; response caching skips the call. Use both together: cache static prefixes with cache_control, and cache full responses for deterministic, repeatable queries.

How long should I cache Claude API responses? It depends on how fast the underlying answer goes stale. Static content (documentation, policies) can be cached for hours or days; user-specific but repeatable queries are usually safe for minutes to an hour. Anything involving live or personalized data in real time shouldn't be cached at all.

Can I cache responses that use a high temperature setting? Not reliably. High-temperature requests are intentionally non-deterministic, so caching them means serving one random output as if it were canonical. Reserve response caching for low-temperature, deterministic tasks like classification, extraction, or FAQ-style answers.