Claude API Batch Processing: Handling Bulk Requests

What "batch processing" means for the Claude API

If you're searching for how to batch process Claude API requests, you're almost certainly trying to send dozens, hundreds, or thousands of prompts — summarizing documents, classifying support tickets, generating embeddings-adjacent text, scoring content — without hand-rolling a fragile loop that dies on the first rate limit error. The short answer: Claude's API doesn't have a single "upload a file, get a batch job ID" endpoint like some providers offer for offline processing. Instead, you build your own batch layer on top of the standard /v1/messages endpoint using controlled concurrency, retries with backoff, and request queuing.

This article covers the practical patterns that actually work in production: how to structure the batch, how to respect rate limits without guessing, how to handle partial failures, and how to keep costs predictable when you're firing off thousands of calls.

The core pattern: concurrency-limited queues, not one giant loop

The naive approach — a for loop awaiting each request sequentially — works but is slow. Firing all requests at once with Promise.all is fast but will immediately hit rate limits and burn retries. The right middle ground is a concurrency-limited worker pool: a fixed number of "workers" pull tasks from a queue, each worker processes one request at a time, and you tune the worker count to match your account's rate limit.

async function runBatch(items, worker, concurrency = 5) {
  const queue = [...items];
  const results = [];

  async function runWorker() {
    while (queue.length) {
      const item = queue.shift();
      try {
        results.push({ item, result: await worker(item) });
      } catch (err) {
        results.push({ item, error: err.message });
      }
    }
  }

  await Promise.all(Array.from({ length: concurrency }, runWorker));
  return results;
}

This pattern scales from 10 to 10,000 items by just changing concurrency and the batch chunk size you feed it. Start low (3–5) and increase until you see rate-limit errors, then back off.

Handling rate limits and retries properly

Claude API rate limits are expressed per minute, both in requests and tokens. When you're batch processing, you will hit them — that's expected, not a bug in your code. What matters is how you respond:

• Respect 429 responses with exponential backoff. Don't retry instantly; wait, then double the wait on each subsequent failure, with a cap (e.g., max 60 seconds).
• Add jitter to avoid every worker retrying at the exact same millisecond.
• Cap total retries per item (3–5 is typical) so a permanently broken prompt doesn't loop forever.
• Log failures separately so you can re-run just the failed subset instead of the whole batch.

async function withRetry(fn, maxRetries = 4) {
  let attempt = 0;
  while (true) {
    try {
      return await fn();
    } catch (err) {
      attempt++;
      if (attempt > maxRetries) throw err;
      const wait = Math.min(2 ** attempt * 500, 30000) + Math.random() * 300;
      await new Promise(r => setTimeout(r, wait));
    }
  }
}

Wrap every individual request in this helper, not the whole batch — otherwise one bad item retries the entire run.

Chunking large jobs

For very large jobs (tens of thousands of items), don't hold everything in memory and don't fire it all in one process run. Split into chunks of a few hundred, process each chunk, write results to disk or a database, and move to the next chunk. This gives you natural checkpoints: if the process crashes at chunk 40 of 100, you resume from there instead of starting over.

A simple approach: store item IDs and status (pending, done, failed) in a lightweight table or even a JSON file, and have your batch runner skip anything already marked done.

Keeping batch jobs cost-predictable

Batch jobs are where token costs add up fast, because you're multiplying a per-request cost by thousands of calls. Before running a full batch:

1. Test on a small sample (10–20 items) and measure actual token usage per call.
2. Set max_tokens conservatively — don't default to the model's maximum if your outputs are short.
3. Trim input prompts — strip boilerplate, truncate long documents to only the relevant section when possible.
4. Track usage per request so you can project total cost before committing to the full run.

If you're using SubToAPI instead of calling Anthropic directly, every response includes usage metadata in the dashboard, so you can watch token consumption per batch in real time rather than finding out at the end of the month. That's particularly useful when a batch job is run by multiple team members under different application keys — each sub_live_... key's usage is tracked separately, which makes it easy to see which job or team is driving cost. See /docs/messages for the request/response shape.

A minimal real batch script

Putting it together — summarizing a list of text blocks with controlled concurrency and retry:

import fetch from "node-fetch";

async function callClaude(prompt) {
  const res = await fetch("https://api.subtoapi.app/v1/messages", {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${process.env.SUBTOAPI_KEY}`,
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      model: "claude-sonnet-4",
      max_tokens: 300,
      messages: [{ role: "user", content: prompt }]
    })
  });
  if (!res.ok) throw new Error(`HTTP ${res.status}`);
  return res.json();
}

async function processItem(text) {
  return withRetry(() => callClaude(`Summarize in one sentence: ${text}`));
}

const results = await runBatch(documents, processItem, 5);

This structure — queue, worker pool, retry wrapper — is the same whether you're calling Anthropic's API directly or through a service like SubToAPI that fronts it with application keys and streaming support (/docs/streaming). The batching logic lives in your application; the API layer just needs to be reliable under concurrent load.

Why an API key layer matters for batch jobs

When multiple scripts, cron jobs, or team members are all running batches against the same underlying Claude access, a single shared secret becomes a liability — you can't tell which job caused a spike, and rotating the key breaks everything at once. SubToAPI issues separate sub_live_... application keys per project or team member, each with its own usage visibility, so a runaway batch job is easy to isolate and kill without touching other integrations. Plans start with a free trial at /signup, and pricing scales from Solo to Team seats at /pricing.

FAQ

Does the Claude API have a dedicated batch endpoint?

Not in the sense of uploading a file and polling for a job result. You batch by sending many individual requests to the standard messages endpoint under your own concurrency and retry logic, as shown above.

How many requests can I run in parallel?

It depends on your account's rate limits (requests per minute and tokens per minute). Start with a concurrency of 3–5 workers, watch for 429 responses, and tune up or down from there rather than guessing a fixed number.

What's the safest way to handle a batch job that partially fails?

Track per-item status (pending/done/failed) in a file or database, retry only the failed subset with backoff, and avoid re-running completed items — this turns a crash mid-batch into a quick resume instead of a full restart.