Claude API Circuit Breaker Pattern Design Guide
What a circuit breaker does for Claude API calls
A circuit breaker protects your application from a failing or slow Claude API dependency by stopping requests before they pile up and take down the rest of your system. Instead of letting every request wait for a timeout against an unresponsive endpoint, the circuit "opens" after a threshold of failures, returns fast errors (or falls back to a cached/alternate response), and periodically tests whether the API has recovered.
This is different from retry logic. Retries help with transient, isolated failures. A circuit breaker is for sustained outages or degraded performance — when Anthropic's API, your network path, or a rate-limit wall is causing failures at scale. Without one, a single upstream incident can exhaust your thread pool, queue, or connection pool as thousands of requests pile up waiting on dead connections.
The three states
A circuit breaker design has three states, and the whole pattern is really a state machine:
- Closed — normal operation. Requests pass through to the Claude API. Failures are counted in a rolling window.
- Open — the failure threshold was exceeded. Requests are rejected immediately (or routed to a fallback) without touching the API. A reset timer starts.
- Half-open — after the timer expires, a limited number of trial requests are allowed through. If they succeed, the circuit closes. If they fail, it reopens and the timer resets (often with backoff).
Minimal implementation
class CircuitBreaker {
constructor({ failureThreshold = 5, resetTimeoutMs = 30000, halfOpenTrials = 2 } = {}) {
this.failureThreshold = failureThreshold;
this.resetTimeoutMs = resetTimeoutMs;
this.halfOpenTrials = halfOpenTrials;
this.state = "closed";
this.failureCount = 0;
this.nextAttempt = 0;
this.trialCount = 0;
}
canRequest() {
if (this.state === "open") {
if (Date.now() >= this.nextAttempt) {
this.state = "half-open";
this.trialCount = 0;
return true;
}
return false;
}
return true;
}
onSuccess() {
this.failureCount = 0;
if (this.state === "half-open") {
this.trialCount++;
if (this.trialCount >= this.halfOpenTrials) {
this.state = "closed";
}
}
}
onFailure() {
this.failureCount++;
if (this.state === "half-open") {
this.state = "open";
this.nextAttempt = Date.now() + this.resetTimeoutMs;
return;
}
if (this.failureCount >= this.failureThreshold) {
this.state = "open";
this.nextAttempt = Date.now() + this.resetTimeoutMs;
}
}
}Wrap your actual API call:
const breaker = new CircuitBreaker({ failureThreshold: 5, resetTimeoutMs: 30000 });
async function callClaude(payload) {
if (!breaker.canRequest()) {
throw new Error("circuit_open");
}
try {
const res = await fetch("https://api.subtoapi.app/v1/messages", {
method: "POST",
headers: {
"Authorization": `Bearer ${process.env.SUBTOAPI_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify(payload),
});
if (!res.ok) throw new Error(`http_${res.status}`);
breaker.onSuccess();
return await res.json();
} catch (err) {
breaker.onFailure();
throw err;
}
}What should count as a failure
Not every non-200 response should trip the breaker the same way. Separate your error classes:
- 5xx / network timeouts / connection resets — these count toward the breaker. They indicate the API or the path to it is unhealthy.
- 429 rate limit — treat this separately. It's not an outage; it's your own usage exceeding a quota. Trip a local rate-limit breaker or queue, not the general health breaker, otherwise you'll falsely mark a healthy API as down.
- 4xx client errors (bad request, invalid auth) — don't count these at all. They won't resolve by waiting, and counting them will open the circuit for a bug that retries won't fix anyway.
If you're calling the Claude API directly, 529 ("overloaded") responses are a strong signal to trip the breaker, since they explicitly mean the upstream is shedding load — hammering it harder makes things worse for everyone.
Combine with retries and backoff, not instead of them
Circuit breakers and retry-with-backoff solve different problems and should be layered: retries handle a single flaky request, while the breaker handles sustained degradation across many requests. A sane order of operations per request is: try the call, retry a couple of times with exponential backoff on transient errors, then report the final failure to the breaker. If the breaker is already open, skip retries entirely and fail fast — retrying into an open circuit defeats its purpose.
What to do when the circuit is open
Fast failure is only useful if your caller does something sensible with it:
- Return a cached or degraded response if the use case allows it (e.g., last known summary, canned message).
- Queue the request for later processing if it's not latency-sensitive (background jobs, batch summarization).
- Fail the request to the end user with a clear retry-after signal rather than a generic 500.
- Switch to a secondary path — a different model tier, a different provider, or a cached static response — if your architecture supports it.
What you should avoid is silently queuing unlimited requests behind an open circuit, since that just delays the pileup instead of preventing it.
Per-dependency granularity
If you call Claude for multiple purposes (chat, summarization, extraction) through different endpoints or with different timeout profiles, use a separate breaker instance per logical dependency. A slow streaming endpoint and a fast classification endpoint have different latency baselines; one shared breaker will either trip too eagerly for the fast path or too late for the slow one.
Where SubToAPI fits
If you're running Claude through SubToAPI, you still need application-level circuit breakers — this pattern protects your service regardless of upstream, and no API gateway removes the need for it. What SubToAPI does give you is consistent sub_live_... application keys, usage metadata per request, and streaming support (see /docs/streaming) so you can distinguish real upstream failures from your own quota or auth issues when deciding what should trip the breaker. Check /docs/messages for response shapes and /docs/quickstart to get a key running in minutes, or see /pricing for plan details.
Questions
Does a circuit breaker replace retry logic? No. Retries fix isolated, transient failures on a single request. Circuit breakers stop you from hammering an API that's already degraded across many requests. Use both, in that order: retry a few times, then let sustained failures trip the breaker.
Should 429 rate-limit responses trip the circuit breaker? Generally no, not the same breaker used for outages. A 429 means you're exceeding your own quota, not that the API is down. Handle it with local throttling or a separate rate-limit-specific breaker so you don't falsely mark a healthy API as unhealthy.
How long should the open state last before testing recovery? Start with 20–30 seconds for interactive APIs and increase with backoff if the half-open trial fails again. Too short causes flapping; too long delays recovery once the dependency is healthy again.