How to Throttle Claude API Requests in Node.js

If you're hitting 429 "rate limit exceeded" errors or watching your Claude API bill spike from runaway concurrent requests, you need to throttle your Node.js client. Throttling means controlling how many requests go out per second and how many run concurrently, so you stay under Anthropic's rate limits and keep costs predictable.

The short answer: use a request queue with a concurrency cap and a minimum delay between calls, add exponential backoff for 429/529 responses, and track token usage so you throttle on actual load, not just request count. Below are the patterns that work in production, from a five-line sleep loop to a proper queue library.

Why Claude API requests need throttling

Anthropic enforces rate limits per organization and per model tier: requests per minute (RPM), tokens per minute (input and output, tracked separately), and sometimes concurrent request caps. If your Node.js app fires off a batch of 50 requests in a Promise.all, you'll blow through RPM limits almost immediately and get a wave of 429 responses.

Throttling solves three distinct problems:

• Rate limit compliance — staying under RPM/TPM thresholds so requests don't get rejected.
• Cost control — capping concurrent streams prevents accidental spend spikes from bugs or retries.
• Backpressure — when you're processing a queue (batch summarization, embeddings pipelines, chat fan-out), you need predictable throughput, not bursts.

Simple throttling: delay between requests

The most basic approach is a fixed delay between sequential calls. This works for scripts and low-volume batch jobs:

async function sleep(ms) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

async function processQueue(items, delayMs = 1000) {
  const results = [];
  for (const item of items) {
    const res = await callClaude(item);
    results.push(res);
    await sleep(delayMs);
  }
  return results;
}

This is easy to reason about but wastes time when the API is fast and under-throttles when it's slow. It also doesn't handle concurrency — only sequential execution.

Concurrency limiting with p-queue

For anything beyond a toy script, use a queue library that limits concurrent in-flight requests and optional requests-per-interval. p-queue is a solid choice:

import PQueue from "p-queue";

const queue = new PQueue({
  concurrency: 5,       // max 5 requests in flight
  intervalCap: 50,      // max 50 requests
  interval: 60000,      // per 60 seconds
});

async function callClaude(prompt) {
  const res = await fetch("https://api.anthropic.com/v1/messages", {
    method: "POST",
    headers: {
      "x-api-key": process.env.ANTHROPIC_API_KEY,
      "anthropic-version": "2023-06-01",
      "content-type": "application/json",
    },
    body: JSON.stringify({
      model: "claude-sonnet-4-5",
      max_tokens: 1024,
      messages: [{ role: "user", content: prompt }],
    }),
  });
  return res.json();
}

const prompts = ["Summarize X", "Summarize Y", "Summarize Z"];
const tasks = prompts.map((p) => queue.add(() => callClaude(p)));
const results = await Promise.all(tasks);

This gives you both a concurrency cap and a rolling rate window in one config block. It's the pattern most teams converge on once they move past a single sequential script.

Token-aware throttling

RPM throttling alone isn't enough once you're sending long prompts or streaming responses — Anthropic's TPM (tokens per minute) limit can bite before RPM does. A practical approach is to estimate tokens per request and track a rolling budget:

let tokensUsedThisMinute = 0;
let windowStart = Date.now();
const TPM_LIMIT = 40000;

function resetWindowIfNeeded() {
  if (Date.now() - windowStart > 60000) {
    tokensUsedThisMinute = 0;
    windowStart = Date.now();
  }
}

async function throttledCall(estimatedTokens, fn) {
  resetWindowIfNeeded();
  while (tokensUsedThisMinute + estimatedTokens > TPM_LIMIT) {
    await sleep(500);
    resetWindowIfNeeded();
  }
  tokensUsedThisMinute += estimatedTokens;
  return fn();
}

Pair this with a real token counter (not just prompt.length / 4) for accuracy, especially if your prompts include system instructions or tool definitions.

Handling 429s with exponential backoff

Throttling reduces but doesn't eliminate rate limit errors — bursts and shared-org usage still cause occasional 429s. Always wrap calls in a retry loop:

async function callWithBackoff(fn, maxRetries = 5) {
  let attempt = 0;
  while (attempt < maxRetries) {
    try {
      return await fn();
    } catch (err) {
      if (err.status === 429 || err.status === 529) {
        const wait = Math.min(2 ** attempt * 500, 10000);
        await sleep(wait);
        attempt++;
      } else {
        throw err;
      }
    }
  }
  throw new Error("Max retries exceeded");
}

Check for a retry-after header when present and use it instead of a fixed backoff schedule — it's more accurate.

Throttling when multiple services share one API key

If several internal services, team members, or app environments all call the same Anthropic key, per-service throttling in each codebase doesn't prevent one service from starving another. In that case it's worth putting a dedicated API layer in front of Claude that enforces limits centrally and gives every consumer its own key.

This is exactly the gap SubToAPI fills: it turns your existing Claude access into an HTTPS API with per-application keys (sub_live_...), so each service, script, or team member authenticates separately and you can see usage per key instead of guessing which part of your stack is burning through rate limits. You still implement client-side throttling as described above, but you get visibility into who's consuming what. Streaming and tool use work the same way they do against the Anthropic API directly — see the streaming docs and tool use docs for request shapes.

Setup takes a few minutes: grab a key after signing up, then follow the quickstart to swap your base URL. Plans start at Solo €9/month, with Team and Scale tiers for multi-seat usage — see pricing for details.

Putting it together

A production-grade throttling setup for a Node.js app typically combines:

1. A concurrency-limited queue (p-queue or bottleneck) for request fan-out.
2. A token-budget tracker for TPM compliance.
3. Exponential backoff with retry-after header support for 429/529 responses.
4. Per-key or per-service usage visibility if multiple consumers share one account.

Start with the queue and backoff — that alone eliminates most rate-limit pain. Add token-aware throttling once you're running longer prompts or high-volume batch jobs.

Questions

Does throttling slow down my app unnecessarily? Not if configured correctly — a concurrency cap of 5–10 with a reasonable interval rarely bottlenecks real traffic; it mainly prevents burst spikes that would trigger 429s anyway, which cost more time than the throttle itself.

Should I throttle on requests or tokens? Both. RPM limits protect against request bursts, but TPM limits are often the real constraint for long prompts — track token usage per request to avoid hitting TPM caps even when request counts look fine.

What's the easiest library for this in Node.js? p-queue for simple concurrency and interval caps, or bottleneck if you need more granular priority queues and reservoir-based rate limiting across multiple job types.