How to Batch Process Requests with Claude API

Batch processing with the Claude API means sending a large number of prompts — hundreds, thousands, or more — without hitting rate limits, blowing your budget, or losing track of which request failed. There isn't a single "batch button" that solves this; it's a combination of concurrency control, retry logic, and usage tracking applied to the same messages endpoint you already use for single requests.

The short answer: run your requests through a bounded concurrency pool (not a for loop with sequential await, and not 10,000 requests fired at once), add exponential backoff on 429s, and log token usage per request so you can catch runaway costs before they show up on an invoice. Below is how to actually build that.

Why naive loops fail

The two most common anti-patterns are:

• Sequential loops. Awaiting each request one at a time is safe but slow — if each call takes 2–4 seconds, 1,000 requests take over an hour.
• Unbounded Promise.all. Firing every request simultaneously looks fast in a demo and then immediately triggers rate limit errors in production, because every API has a cap on concurrent connections and tokens-per-minute.

The fix is a concurrency pool: a fixed number of "workers" pull jobs from a queue, process them, and move to the next one. This keeps throughput high while staying under your rate limit.

Building a concurrency pool

Here's a minimal, dependency-free implementation in JavaScript:

async function processBatch(items, worker, concurrency = 5) {
  const results = new Array(items.length);
  let index = 0;

  async function runNext() {
    const current = index++;
    if (current >= items.length) return;
    try {
      results[current] = await worker(items[current], current);
    } catch (err) {
      results[current] = { error: err.message };
    }
    await runNext();
  }

  const workers = Array.from({ length: concurrency }, runNext);
  await Promise.all(workers);
  return results;
}

Then wire it up to the Claude API:

const prompts = ["Summarize doc A", "Summarize doc B", "..."]; // up to thousands

const results = await processBatch(
  prompts,
  async (prompt) => {
    const response = await fetch("https://api.anthropic.com/v1/messages", {
      method: "POST",
      headers: {
        "x-api-key": process.env.ANTHROPIC_API_KEY,
        "anthropic-version": "2023-06-01",
        "content-type": "application/json",
      },
      body: JSON.stringify({
        model: "claude-3-5-sonnet-latest",
        max_tokens: 1024,
        messages: [{ role: "user", content: prompt }],
      }),
    });
    if (!response.ok) throw new Error(`HTTP ${response.status}`);
    return response.json();
  },
  5 // concurrency — tune based on your rate limit tier
);

Start with a concurrency of 5 and increase gradually while watching for 429 responses. Most teams find a sweet spot between 5 and 20 concurrent requests, depending on model and plan.

Handling rate limits inside a batch

Even with bounded concurrency, you'll occasionally hit a 429 under load. Wrap your worker function with retry-and-backoff logic rather than letting one failed request kill the whole batch:

async function withRetry(fn, retries = 4) {
  for (let attempt = 0; attempt <= retries; attempt++) {
    try {
      return await fn();
    } catch (err) {
      if (attempt === retries) throw err;
      const delay = 2 ** attempt * 500 + Math.random() * 250;
      await new Promise((r) => setTimeout(r, delay));
    }
  }
}

Call withRetry(() => yourClaudeRequest()) inside the pool worker. This handles transient 429s and 5xxs without requiring you to restart the whole job.

Chunking large jobs

If you're processing a dataset — support tickets, documents, product descriptions — break it into fixed-size chunks and process one chunk at a time instead of loading the entire dataset into memory. This also gives you natural checkpoints: write results to disk or a database after each chunk so a crash halfway through doesn't mean starting from zero.

function chunk(array, size) {
  const out = [];
  for (let i = 0; i < array.length; i += size) {
    out.push(array.slice(i, i + size));
  }
  return out;
}

for (const batch of chunk(allPrompts, 200)) {
  const batchResults = await processBatch(batch, worker, 10);
  await saveResults(batchResults);
}

Tracking cost and token usage across a batch

Batch jobs are where API costs sneak up on you — a bug that silently reruns the same 10,000 requests twice is expensive. Capture usage.input_tokens and usage.output_tokens from every response and sum them as the batch runs, not after the fact. If you're using SubToAPI to run your Claude traffic, every request returns usage metadata and shows up in the dashboard automatically, so a batch job that spikes unexpectedly is visible in real time rather than discovered on the monthly bill. See the messages docs for the exact response shape.

If your batch involves tool calls (e.g., having Claude extract structured data from each item), the same concurrency pool pattern applies — just make sure your worker function parses tool_use blocks correctly. The tools documentation covers the request/response format in detail.

Should you stream in a batch job?

Generally no. Streaming is built for showing partial output to a human in real time; batch jobs are usually unattended and only care about the final result. Skip streaming for batch work unless you specifically need to process partial tokens as they arrive — it adds complexity without benefit when you're just collecting final outputs. (If you do need it for a specific pipeline, the streaming guide walks through server-sent events.)

Getting started quickly

If you're prototyping a batch job and don't want to manage API key rotation, per-key rate limit tiers, or usage dashboards yourself, SubToAPI wraps Claude access behind a standard API key (sub_live_...) with usage metadata on every response and team-level visibility into spend — useful when a batch script is running across a shared budget. Check the quickstart to get a key, or see pricing for Solo, Team, and Scale plans with a free trial at signup.

FAQ

What's the ideal concurrency level for batch requests? There's no universal number — it depends on your rate limit tier and average response time. Start at 5 concurrent requests, monitor for 429 errors, and increase gradually until you find the ceiling for your account.

Should I use a queue system like SQS or BullMQ for large batches? For jobs under a few thousand items, an in-process concurrency pool with chunked checkpoints is usually enough. For recurring or very large batches (hundreds of thousands of items, multi-day jobs), a proper queue gives you persistence, retries, and observability that a script running in a single process can't.

How do I avoid duplicate charges if a batch job crashes midway? Write results to a database or file after each chunk completes, and track which items have already been processed by ID. On restart, skip anything already marked complete instead of re-running the entire list from scratch.