How to Chunk Documents for Claude API

Chunking documents for Claude means splitting long text into smaller, semantically coherent pieces before sending them to the API — either because the source exceeds the context window, or because you're feeding chunks into a retrieval system and only want the most relevant ones in each request. The right chunk size and split strategy directly affect answer quality, cost, and latency, so it's worth getting right rather than picking an arbitrary number and moving on.

This guide covers the practical decisions: how big chunks should be, where to split, how to preserve context across boundaries, and how to structure the request once your chunks are ready.

Why chunking matters

Claude's models have large context windows, but "fits in the context window" and "produces a good answer" are different problems. Three issues show up if you get chunking wrong:

• Broken semantics: splitting mid-sentence or mid-table destroys meaning, and Claude has to guess at what's missing.
• Lost context: a chunk pulled from the middle of a document often loses the surrounding context that made it meaningful (which section it's in, what it's referring to).
• Wasted tokens: chunks that are too large increase cost and latency without improving relevance, especially in retrieval setups where you're paying for tokens on every request.

Good chunking is a tradeoff between chunk size, semantic coherence, and how much overlap you keep between chunks.

Choosing a chunk size

There's no single right number, but here are reasonable starting points depending on your use case:

• Direct summarization/analysis (no retrieval): chunk by natural document boundaries — chapters, sections, or pages — rather than a fixed token count. If the whole document fits in context, you often don't need to chunk at all.
• RAG / retrieval pipelines: 300–800 tokens per chunk is a common sweet spot. Small enough to be specific, large enough to retain context.
• Structured documents (contracts, manuals, code): chunk along structural boundaries (clauses, headings, functions) rather than a fixed size — structure usually matters more than token count here.

Start with 500 tokens and overlap of 10–15%, then adjust based on how often retrieved chunks miss relevant context or return too much noise.

Splitting strategies

1. Fixed-size splitting

Simplest approach: split every N tokens, with some overlap between chunks so context isn't lost at the boundary.

function chunkText(text, chunkSize = 500, overlap = 50) {
  const words = text.split(/\s+/);
  const chunks = [];
  let start = 0;

  while (start < words.length) {
    const end = Math.min(start + chunkSize, words.length);
    chunks.push(words.slice(start, end).join(" "));
    start += chunkSize - overlap;
  }

  return chunks;
}

This is fast and predictable but ignores document structure — it will happily cut a sentence in half. Fine for rough drafts, not ideal for production RAG.

2. Semantic / structural splitting

Split on natural boundaries: paragraphs, headings, sentence groups. Most document formats give you these for free.

function chunkByParagraphs(text, maxTokens = 500) {
  const paragraphs = text.split(/\n\s*\n/);
  const chunks = [];
  let current = "";

  for (const para of paragraphs) {
    if ((current + para).split(/\s+/).length > maxTokens) {
      if (current) chunks.push(current.trim());
      current = para;
    } else {
      current += "\n\n" + para;
    }
  }
  if (current) chunks.push(current.trim());
  return chunks;
}

This keeps sentences and paragraphs intact, which noticeably improves answer quality compared to fixed-size splitting, at the cost of slightly variable chunk sizes.

3. Recursive splitting

Try to split on the largest structural boundary first (sections), then fall back to smaller ones (paragraphs, then sentences) only if a chunk is still too big. This is what most production RAG libraries do under the hood, and it's worth implementing yourself if you're not using one — it's not much more code than the paragraph splitter above.

4. Markdown/HTML-aware splitting

If your source documents are markdown or HTML, split on headings first so each chunk stays under one topic:

function chunkByHeadings(markdown) {
  return markdown.split(/(?=^#{1,3}\s)/m).filter(Boolean);
}

Combine this with a size limit per section so long sections still get subdivided.

Preserving context across chunks

Two techniques make a real difference here:

• Overlap: repeat the last 1–3 sentences of a chunk at the start of the next one, so a fact split across the boundary still appears whole somewhere.
• Header injection: prepend each chunk with its document title and section heading (e.g. # Refund Policy > Eligibility). This costs a handful of tokens per chunk and meaningfully improves retrieval relevance and Claude's ability to answer questions about where information came from.

function withContext(chunk, docTitle, sectionTitle) {
  return `Document: ${docTitle}\nSection: ${sectionTitle}\n\n${chunk}`;
}

Sending chunks to Claude

Once your chunks are ready, the pattern depends on the use case. For summarizing a whole document, send chunks sequentially and ask Claude to maintain a running summary. For Q&A over a large corpus, retrieve the top-k relevant chunks and pass them as context in a single request.

If you're calling the Anthropic API directly, or routing through SubToAPI so you get a standard HTTPS endpoint, application API keys, and usage metadata alongside your existing Claude access, the request shape is the same — you're just assembling the chunks into the messages payload:

curl https://api.subtoapi.app/v1/messages \
  -H "Authorization: Bearer $SUBTOAPI_KEY" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-sonnet-4-5",
    "max_tokens": 1024,
    "messages": [
      {"role": "user", "content": "Context:\n\n" + chunk1 + "\n\n" + chunk2 + "\n\nQuestion: ..."}
    ]
  }'

Details on request structure are in the Messages docs; if you're chunking a document too large for one call, streaming responses helps avoid timeouts on long generations. Getting the API wired up is a separate step from chunking — see the quickstart if you haven't set that up yet.

Common mistakes to avoid

• Chunking by character count instead of tokens — character counts don't map cleanly to tokens, especially with non-English text or code.
• No overlap at all — facts and references that span a chunk boundary become unrecoverable.
• Ignoring document structure — a PDF with tables split mid-table produces garbage chunks; extract tables separately when possible.
• One chunk size for everything — a legal contract and a chat transcript have very different natural boundaries; don't force both through the same splitter.

questions

What chunk size should I use for Claude API requests? For retrieval-based use cases, 300–800 tokens per chunk with 10–15% overlap is a solid default. For direct document analysis, chunk by natural sections rather than a fixed token count.

Should I chunk by tokens or by characters? Tokens. Character counts don't correspond reliably to token counts across languages and formats, so a character-based splitter can produce wildly inconsistent chunk sizes.

Do I need to chunk documents that fit in Claude's context window? Not necessarily. If the whole document fits and you're doing a single analysis or summary, sending it whole is simpler and preserves all context. Chunking mainly matters for retrieval pipelines or documents that exceed the context window.