Claude API Cold Start Latency on AWS Lambda

Why Claude API calls feel slow from AWS Lambda

If you're calling Claude from a Lambda function and seeing multi-second delays on the first request after a deploy or idle period, you're hitting two separate latency sources that stack on top of each other: Lambda's own cold start (runtime init, dependency loading, possible VPC ENI attachment) and network/API latency to Anthropic's servers (TLS handshake, DNS resolution, time-to-first-token). Neither is fixable by "just retrying" — you need to address each independently.

The fix isn't one setting. It's a combination of: keeping your Lambda warm, minimizing what loads during init, avoiding unnecessary VPC attachment, and reducing the size and complexity of the HTTP client you use to talk to the API. Below is what actually moves the numbers.

The two latency sources, separated

Lambda cold start

This is AWS spinning up a new execution environment: downloading your deployment package, initializing the runtime, running top-level imports, and (if you're in a VPC) attaching an elastic network interface. For a Node.js function with a heavy SDK bundle, this can easily add 300ms–1.5s before your handler code even runs.

Claude API round-trip latency

Separately, every HTTPS call to the Claude API involves DNS resolution, a TLS handshake (unless the connection is reused), and the model's own processing time before the first token streams back. On a cold Lambda container, you pay the TLS handshake cost every time because there's no warm HTTP connection pool carried over.

Combined, a "cold" Lambda invocation calling Claude can look like: 500ms Lambda init + 150ms TLS handshake + 400ms+ time-to-first-byte from the model = well over a second before anything useful happens. A warm invocation with a warm connection can be a fraction of that.

Reducing Lambda-side cold start

Trim your deployment package. Every unused dependency adds to init time. If you're bundling a full SDK with retry logic, logging, and telemetry you don't use, that's extra parse time on every cold start. A lightweight fetch call to a REST endpoint often beats a heavyweight SDK for pure latency, even if it means writing a bit more boilerplate yourself.

Avoid VPC attachment unless you need it. If your Lambda doesn't need to reach a private resource (RDS, internal VPC service), don't attach it to a VPC just because it's the default in your IaC template. VPC-attached Lambdas historically had much worse cold starts due to ENI provisioning; AWS has improved this, but it's still not free, and it adds unpredictability.

Use provisioned concurrency for latency-sensitive paths. If a specific endpoint (e.g., a chat completion route) needs consistent sub-second response, provisioned concurrency keeps a pool of initialized execution environments ready. It costs more per hour than on-demand, but for user-facing latency it's usually worth it.

Lazy-load what you can. Move imports and client initialization inside the handler only if they depend on per-request data. Otherwise, initialize clients at module scope — this runs once per container and gets reused across warm invocations, which is the correct place for connection pooling to happen.

Reducing API-side latency

Reuse HTTP connections across invocations. Initialize your HTTP client (or fetch agent with keep-alive) outside the handler function, at module scope. On a warm Lambda container, this connection can be reused for the next invocation, skipping the TLS handshake entirely.

// Initialized once per container, reused across warm invocations
const keepAliveAgent = new (require('https').Agent)({ keepAlive: true });

exports.handler = async (event) => {
  const res = await fetch("https://api.subtoapi.app/v1/messages", {
    method: "POST",
    agent: keepAliveAgent,
    headers: {
      "Authorization": `Bearer ${process.env.SUBTOAPI_KEY}`,
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      model: "claude-sonnet",
      max_tokens: 500,
      messages: [{ role: "user", content: event.prompt }]
    })
  });

  return await res.json();
};

Stream responses instead of waiting for the full completion. If your use case allows it (chat UI, incremental rendering), streaming gets you a usable first token much sooner than waiting for the entire response to buffer. See /docs/streaming for the event format.

Pick the right model for the latency budget. Larger models produce higher-quality output but take longer to generate the first token and to finish. For latency-sensitive paths — autocomplete, quick classification, chat typing indicators — a smaller/faster model is often the correct tradeoff, even if you use a larger model elsewhere in the same product.

Measure time-to-first-byte separately from total completion time. Log both. A slow total completion with a fast TTFB is a different problem (long generation) than a slow TTFB (connection/queue issue), and they need different fixes.

Where SubToAPI fits

If part of your latency problem is SDK overhead and connection management complexity inside Lambda, routing calls through a single HTTPS endpoint with application API keys (sub_live_...) simplifies the client side: no SDK to bundle, no multi-step auth flow, just a REST call with a bearer token. That keeps your deployment package small, which directly helps Lambda cold start time. Check /docs/quickstart for the request format, /docs/messages for the message schema, and /pricing if you're evaluating team usage across multiple Lambda-backed services. A free trial is available at /signup.

Practical checklist

• Initialize HTTP clients and agents at module scope, not inside the handler
• Use provisioned concurrency for latency-critical Lambda routes
• Avoid VPC attachment unless strictly necessary
• Keep deployment packages small — fewer dependencies, faster cold init
• Stream responses when the UI can render incrementally
• Log TTFB and total completion time separately to diagnose the right bottleneck

Questions

Does provisioned concurrency eliminate cold starts entirely? It keeps a pool of pre-initialized environments ready, so requests routed to them skip cold start. If demand exceeds the provisioned pool size, overflow requests still cold start normally.

Will keep-alive connections actually persist between Lambda invocations? Yes, but only on warm containers — if the same execution environment is reused for the next invocation, the connection pool initialized at module scope survives. On a fresh cold start, there's no prior connection to reuse.

Is streaming always faster than a regular request for perceived latency? For user-facing output, yes — you get the first token much sooner. For backend processing where you need the full response before doing anything else, streaming adds complexity without a latency benefit, since you're waiting for completion regardless.