How to Cache Claude API Responses for Speed

Caching Claude API responses is the single highest-leverage optimization you can make if your app sends repeated or similar prompts. The core idea is simple: store the output of a request keyed by its input, and serve the stored result instead of calling the model again when the same (or similar enough) input comes in. Done right, this cuts p99 latency from seconds to milliseconds and reduces your token spend on duplicate work.

There are two distinct caching layers worth understanding, and most production systems need both: application-level response caching (your own cache, full control, works with any provider) and prompt caching (a model-side feature that reuses processed context across requests to skip redundant prompt processing, not the final output). This article covers both, plus the practical pitfalls that trip people up.

Why caching matters for Claude API performance

Every call to the Claude API involves network round-trip time, queueing, and inference time proportional to input and output length. For a 2,000-token prompt with a 500-token response, you're commonly looking at 2-6 seconds depending on load and model. If your users hit the same question, the same support ticket template, or the same classification task repeatedly, you're paying that latency and cost tax every single time — for zero new information.

Caching fixes this by trading storage for compute. A cache hit is typically sub-10ms. That's the difference between a snappy UI and a spinner your users complain about.

Application-level response caching

This is the most reliable way to speed up repeated Claude API calls. You own the cache, you control invalidation, and it works regardless of which model or provider sits behind it.

Step 1: Build a stable cache key

Don't just hash the raw prompt string — normalize it first so semantically identical requests produce the same key.

import crypto from "crypto";

function cacheKey(systemPrompt, messages, model, maxTokens) {
  const normalized = JSON.stringify({
    system: systemPrompt.trim(),
    messages: messages.map(m => ({ role: m.role, content: m.content.trim() })),
    model,
    maxTokens,
  });
  return crypto.createHash("sha256").update(normalized).digest("hex");
}

Include every parameter that affects the output: model name, temperature, max_tokens, tool definitions. If you vary temperature per request, either cache per-temperature or only cache when temperature is 0 (deterministic-ish outputs are much safer to cache).

Step 2: Check cache before calling the API

async function getCachedOrFetch(key, fetchFn, ttlSeconds = 3600) {
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  const result = await fetchFn();
  await redis.set(key, JSON.stringify(result), "EX", ttlSeconds);
  return result;
}

Redis is the standard choice: fast, supports TTL natively, and scales horizontally. For lower-traffic apps, an in-memory LRU cache (e.g. lru-cache in Node) or even a Postgres table with a UNIQUE constraint on the hash works fine — don't over-engineer this if you're serving a few hundred requests a day.

Step 3: Set a sensible TTL

TTL depends on how stable the "correct" answer is:

• Static reference content (docs Q&A, policy lookups): hours to days.
• User-specific but repeatable (same support query template): 15-60 minutes.
• Time-sensitive or personalized output: don't cache, or cache for seconds only.

Step 4: Invalidate deliberately

If the underlying source data changes (you update a knowledge base, change a system prompt, bump the model version), bust the cache. The cleanest way is to fold a version number into the cache key itself:

const key = cacheKey(systemPrompt, messages, model, maxTokens) + ":v3";

Bumping :v3 to :v4 instantly invalidates everything without a manual flush.

Prompt caching: a different kind of speedup

Claude models support prompt caching, where a large, repeated block of context (a system prompt, a long document, a tool schema) is marked as cacheable so the model doesn't reprocess it from scratch on every call. This is not the same as application-level response caching — it doesn't return a stored answer, it speeds up how fast the model processes a prompt it has seen the shape of before.

This matters most when you have a large static prefix (e.g., a 10,000-token document) followed by a small varying question. Instead of reprocessing the whole document every time, the cached prefix is reused, cutting both latency and cost for that portion of the request.

If you're calling Claude through SubToAPI, you get standard message and streaming endpoints (see /docs/messages and /docs/streaming) that you can layer your own application-level cache on top of — the two techniques are complementary, not competing.

Combining both layers

A practical setup for a RAG or support-bot style app:

1. Keep a long, stable system prompt + document context — let prompt caching handle reprocessing cost on the model side.
2. Wrap the full request/response in your own Redis cache keyed by the user's actual question + a version tag.
3. Set TTL based on how often the underlying knowledge changes.
4. Log cache hit rate so you know if caching is actually paying off — if your hit rate is under 10%, your traffic may just be too varied for response caching to help much, and you should focus on prompt caching instead.

curl https://api.subtoapi.app/v1/messages \
  -H "Authorization: Bearer $SUBTOAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-5",
    "max_tokens": 500,
    "messages": [{"role": "user", "content": "Summarize our refund policy"}]
  }'

Wrap this call in your cache layer exactly as shown above — the API call itself doesn't need to change.

Common mistakes to avoid

• Caching non-deterministic outputs blindly. High temperature means two identical prompts can return different (both valid) answers. Caching one and serving it forever can feel "wrong" to users even if it's technically a cache hit.
• Forgetting to key on model version. If you upgrade models, old cached responses from a previous model can linger and confuse quality comparisons.
• No cache metrics. Without hit-rate tracking, you're optimizing blind. Log it from day one.
• Caching PII-laden responses without expiry. Short TTLs and proper access control matter if cached content includes user data.

questions

Does caching reduce Claude API costs, not just latency? Yes. A cache hit means you don't send tokens to the model at all, so you save on both input and output token costs for that request, on top of the latency win.

Should I cache at the application level or rely on prompt caching alone? Use both. Prompt caching speeds up reprocessing of repeated context (documents, system prompts); application-level caching skips the API call entirely for genuinely repeated requests. They solve different problems.

How do I avoid serving stale cached answers? Set TTLs matched to how often your underlying data changes, and bake a version tag into your cache key so you can invalidate everything instantly by bumping one value.