Claude API Long-Context Summarization Strategy Guide
The short answer
If you're trying to summarize a document, transcript, or codebase that's larger than a single context window can comfortably handle, the reliable strategy is: chunk the input, summarize each chunk independently, then summarize the summaries (map-reduce), with a hierarchical pass if the source is very large. Don't just stuff everything into one prompt and hope the model "remembers" the beginning by the time it reaches the end — quality degrades with distance, even with large context windows.
The second thing that matters as much as the algorithm is how you manage token budget and requests. Long-context summarization is expensive in tokens and slow if done sequentially. Getting this right means understanding chunk sizing, overlap, streaming responses so you're not blocked on multi-minute generations, and tracking usage so costs don't surprise you. Below is a strategy that works whether you're calling the Claude API directly or through a proxy like SubToAPI.
Why "just paste the whole document" fails
Claude's context windows are large, but three problems show up in practice:
- Recall degrades with position. Content buried in the middle of a huge prompt gets less attention than content near the start or end.
- Cost scales linearly with input tokens, even if the useful signal is small relative to the noise in a long document.
- Latency scales too. A 150K-token input takes meaningfully longer to process than a 5K-token one, and you can't parallelize a single call.
So the goal isn't "fit everything in," it's "process everything, but in pieces that stay accurate and let you parallelize."
Strategy 1: Map-reduce summarization
This is the default approach for most long documents (reports, books, legal contracts, meeting transcripts).
Map step — split the source into chunks (by tokens, not characters) with a small overlap so context isn't lost at boundaries:
function chunkText(text, maxTokens = 3000, overlapTokens = 200) {
const words = text.split(/\s+/);
const wordsPerChunk = maxTokens * 0.75; // rough token-to-word ratio
const overlapWords = overlapTokens * 0.75;
const chunks = [];
let i = 0;
while (i < words.length) {
chunks.push(words.slice(i, i + wordsPerChunk).join(" "));
i += wordsPerChunk - overlapWords;
}
return chunks;
}Send each chunk to the API in parallel with a tight, consistent instruction:
curl https://api.subtoapi.app/v1/messages \
-H "Authorization: Bearer $SUBTOAPI_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "claude-3-5-sonnet",
"max_tokens": 500,
"messages": [
{"role": "user", "content": "Summarize this section in 5 bullet points. Preserve names, dates, and numbers exactly. Do not add commentary.\n\n<chunk text>"}
]
}'Reduce step — concatenate all chunk summaries and run a final pass that produces the actual output (executive summary, structured brief, whatever the use case needs). Because the chunk summaries are already compressed, the reduce step's input is a fraction of the original size and fits comfortably in one call.
The API details for message structure are the same as any standard call — see /docs/messages if you're integrating this for the first time.
Strategy 2: Hierarchical summarization for very large inputs
For inputs so large that even the reduce step's input (all chunk summaries combined) exceeds a comfortable context size, add a middle layer:
- Summarize each chunk (map).
- Group chunk summaries into batches and summarize each batch (intermediate reduce).
- Summarize the batch summaries into the final output (final reduce).
This is essentially a summarization tree. It costs more API calls but keeps every individual prompt small and accurate, and it parallelizes well — steps 1 and 2 can both run many requests concurrently.
Strategy 3: Rolling summary for streams and transcripts
For live transcripts or logs that arrive incrementally, don't re-summarize from scratch each time. Maintain a running summary and update it with each new chunk:
Current summary: <previous summary>
New content: <latest chunk>
Update the summary to incorporate the new content. Keep it under 200 words.This keeps token usage flat regardless of how long the session runs, instead of growing with every turn.
Practical implementation notes
Instruct for extraction, not compression alone. Asking Claude to "summarize" without constraints often produces vague prose. Ask for specific extracted fields (decisions, action items, numbers, dates) and a fixed format — this makes the reduce step far more reliable because it's combining structured data, not paraphrasing paraphrases.
Use streaming for the final long-form output. If your final summary is itself long (a detailed report rather than a one-paragraph digest), stream the response so your UI isn't blocked for 30+ seconds waiting for the full generation. See /docs/streaming for how streamed responses work.
Track token usage per stage. Map-reduce summarization multiplies your request count, so usage metadata matters more here than in single-turn chat. If you're running this through SubToAPI, every response includes usage data you can log per pipeline stage, which makes it easy to see whether the map step or the reduce step is your cost driver.
Parallelize the map step, but respect rate limits. Running 50 chunks through 50 concurrent calls is fine for throughput but can hit provider or plan limits. Batch concurrency (e.g., 5–10 at a time) is usually a safer default.
Cache stable inputs. If you're re-summarizing the same source repeatedly (e.g., a document that gets re-processed after minor edits), don't discard prior chunk summaries — only re-run the map step for chunks that actually changed.
Where SubToAPI fits
If you're building this pipeline as a product feature rather than a one-off script, you'll want application-scoped API keys (so a summarization service doesn't share credentials with other parts of your stack), clear per-key usage tracking to see the real cost of map-reduce fan-out, and team access for whoever maintains the pipeline. SubToAPI wraps Claude access into an HTTPS API with exactly that — get started at /signup, check /pricing for plan details, or follow /docs/quickstart to make your first summarization call in a few minutes.
Questions
Do I need map-reduce if Claude's context window can technically fit my whole document? Fitting isn't the same as summarizing well. Even within the context limit, accuracy drops for content far from the prompt edges, so chunking still improves quality on large inputs — it's not only a workaround for size limits.
How big should each chunk be? 2,000–4,000 tokens per chunk is a reasonable default for the map step. Smaller chunks give more accurate individual summaries but increase total request count and cost; tune based on your document structure (e.g., chunk by section or speaker turn when possible instead of arbitrary token cutoffs).
Should the reduce step use a different model or settings than the map step? It's common to use a lower max_tokens and stricter format instructions on the map step (since outputs should be compressed), and allow more tokens and freer structure on the reduce step, where the final human-readable output is produced.