API Gateway Timeout: Causes and How to Fix It

An API gateway timeout happens when the gateway gives up waiting for a response from an upstream service and returns an error to the client before the real answer ever arrives. It usually shows up as an HTTP 504 Gateway Timeout (sometimes 502 or 503 depending on the gateway), and it means the gateway itself is fine — the problem is that something downstream took too long, or the gateway's own timeout setting is too aggressive for the workload.

The fix depends on which side is actually slow: your backend, a third-party API you're calling through the gateway, or the gateway's configuration. The rest of this article walks through how to tell them apart and what to change.

What a 504 actually tells you

A 504 is a gateway-level error, not an application error. It means:

• The client sent a request to the gateway.
• The gateway forwarded it to an upstream (your service, a container, a load balancer target, or an external API).
• The upstream didn't respond within the gateway's configured timeout window.
• The gateway closed the connection and returned 504 to the client.

Critically, this doesn't tell you why the upstream was slow. It could still be processing the request when the gateway gave up — meaning the work you paid for might complete anyway, just with no one listening for the result.

Common causes

1. The upstream is genuinely slow. Database queries without indexes, N+1 calls, cold starts on serverless functions, or a downstream API that's degraded. The gateway timeout is just the messenger.

2. The gateway timeout is shorter than the real processing time. Many gateways default to something in the 15–30 second range. If your endpoint legitimately needs 45 seconds — a large report generation, a batch job, a call to an LLM producing a long completion — you'll get a 504 even though nothing is actually broken.

3. Streaming responses aren't configured correctly. If a gateway buffers the full response before forwarding it, and the upstream takes a while to finish streaming (common with LLM APIs that generate long outputs token by token), the gateway may time out waiting for the stream to close, even though data was flowing the whole time.

4. Connection pool exhaustion. If the gateway or a proxy in front of it runs out of available connections to the upstream, new requests queue and eventually time out — this looks identical to a slow upstream but the actual bottleneck is concurrency, not latency.

5. DNS or network issues between gateway and upstream. Less common, but a flaky internal network hop can add enough latency to trip the timeout on requests that are normally fast.

How to diagnose it

Start by separating "slow" from "stuck":

• Check upstream logs for the request in question. If it completed (even late), you have a timeout mismatch, not a broken service.
• Check gateway access logs for the actual duration before the 504 was returned — this tells you the exact timeout value in effect.
• Reproduce with a direct call to the upstream, bypassing the gateway. If it's still slow, the problem is your backend, not the gateway.
• If it's fast direct but slow through the gateway, look at gateway-side connection limits, retry logic, or middleware (auth checks, logging, transformation) adding overhead per request.

Fixes that actually work

Increase the timeout — but only after you understand why it's slow. Bumping a 30-second timeout to 120 seconds masks a performance problem instead of solving it. It's a valid fix for genuinely long-running work (report generation, video processing, large model completions), but not a substitute for fixing an N+1 query.

Move long work off the request/response cycle. If a job can take more than a few seconds, return a job ID immediately and let the client poll or use a webhook. This avoids fighting timeout limits entirely and is more resilient to network blips.

Use streaming instead of waiting for the full response. For workloads like LLM text generation, streaming tokens as they're produced keeps the connection active and avoids a single long silent wait that gateways are quick to kill. This is one of the reasons SubToAPI supports streaming on every plan — long completions are sent incrementally instead of buffered, so gateways in front of your app see continuous activity instead of one long pause.

Set client-side timeouts that match server-side ones. If your client gives up before the server does (or vice versa), you get confusing partial failures. Align timeout budgets across the whole chain: client → gateway → backend → any third-party API.

Add backpressure and connection limits deliberately. If timeouts correlate with traffic spikes, the real fix is often rate limiting or queueing at the gateway, not a longer timeout. A gateway that queues cleanly under load produces fewer timeouts than one that lets every request pile up on a saturated backend.

Timeouts when calling AI APIs specifically

If you're building on top of a model provider through a gateway — your own or a third-party one — timeouts are especially common because completion time varies with output length and isn't fully predictable up front. A short factual answer might return in a second; a long structured output can take much longer. Two practical patterns help:

• Always stream when the client can consume a stream (see the messages docs for the request/response shape SubToAPI uses).
• Set generous but bounded timeouts specifically on the model-calling code path, separate from the rest of your app's timeout config, so one slow completion doesn't force you to loosen timeouts everywhere else.

If you're routing Claude access through a gateway layer, check pricing and the quickstart to see how request handling, streaming, and API keys are structured before you build your own timeout logic around it.

Questions

Is a 504 always the gateway's fault? No. Most 504s originate from a slow or unresponsive upstream. The gateway is just the component that enforces a time limit and reports the failure.

What's a reasonable gateway timeout value? There's no universal number — it depends on your slowest legitimate request. A common approach is setting the timeout slightly above your p99 latency for that endpoint, then handling genuinely long jobs asynchronously instead of raising the limit further.

Can retries fix a timeout? Sometimes, if the cause was transient (a brief spike, a dropped connection). Retrying a request that's timing out due to a consistently slow query or an overloaded backend just adds more load and makes things worse — fix the root cause first.