How to Implement an API Gateway in Microservices

An API gateway sits between clients and your microservices, acting as the single entry point for every request. To implement one, you pick a gateway (build with something like Express/Kong/Envoy, or use a managed product), define routes that map incoming paths to internal services, then layer in authentication, rate limiting, request/response transformation, and observability on top of that routing layer.

The implementation itself is mostly configuration and a handful of well-known cross-cutting concerns. The hard part is deciding what belongs in the gateway versus what belongs in each service, and rolling it out without breaking existing clients. Below is a practical path from zero to a production-ready gateway.

Step 1: Decide what the gateway owns

Before writing any routing rules, draw a clear line between gateway responsibilities and service responsibilities. A typical split:

• Gateway owns: TLS termination, authentication/token validation, rate limiting, request logging, CORS, basic request/response transformation, routing/load balancing.
• Services own: Business logic, authorization details specific to a resource, data validation beyond schema shape, database access.

Getting this split wrong is the most common implementation mistake. Teams either push too much business logic into the gateway (making it a bottleneck for every deploy) or leave auth checks scattered across services (making it impossible to audit access consistently).

Step 2: Choose a gateway approach

You have three realistic options:

1. Off-the-shelf gateway (Kong, NGINX, Envoy, Traefik) — configuration-driven, battle-tested, good for teams that don't want to maintain custom code.
2. Cloud-managed gateway (AWS API Gateway, Azure API Management) — less infrastructure to run, but you're tied to the provider's routing model and pricing.
3. Custom gateway (a thin Node.js/Go service using a framework) — full control, more code to maintain, makes sense when your routing logic is unusual or you need something a config file can't express.

For most microservices setups, an off-the-shelf gateway with declarative config is the fastest path to something reliable. Build a custom one only when you have a concrete reason (e.g., non-standard auth flows, per-tenant routing logic).

Step 3: Define routes and service discovery

Each route maps a public path to an internal service. A minimal config-style example:

routes:
  - path: /users/*
    service: user-service
    upstream: http://user-service.internal:3000
  - path: /orders/*
    service: order-service
    upstream: http://order-service.internal:3001
  - path: /payments/*
    service: payment-service
    upstream: http://payment-service.internal:3002

In dynamic environments (Kubernetes, ECS), don't hardcode upstream hosts — wire the gateway into service discovery (Consul, Kubernetes DNS, or your cloud provider's registry) so routes resolve to healthy instances automatically. If you're building the gateway yourself, a simple health-check loop against each upstream, with the unhealthy ones removed from rotation, gets you most of the reliability benefit without a full service mesh.

Step 4: Add authentication at the edge

Validate tokens once, at the gateway, instead of duplicating auth logic in every service. A common pattern with JWTs:

function authenticate(req, res, next) {
  const token = req.headers.authorization?.replace("Bearer ", "");
  if (!token) return res.status(401).json({ error: "missing token" });

  try {
    const payload = verifyJwt(token, process.env.JWT_PUBLIC_KEY);
    req.user = payload;
    next();
  } catch {
    res.status(401).json({ error: "invalid token" });
  }
}

The gateway verifies the signature and expiry, then forwards a trusted internal header (like X-User-Id) downstream. Services trust that header because the network between gateway and services is private — they don't need to re-verify the JWT themselves.

If you're exposing an AI model through your own API rather than internal microservices, this same edge-authentication pattern is exactly what a product like SubToAPI applies to Claude access: your subscription becomes application API keys (sub_live_...) issued and checked at the gateway layer, so the services calling Claude never need to handle raw credentials. See the quickstart for how key validation and request forwarding work in that setup.

Step 5: Rate limit and throttle per client

Rate limiting protects downstream services from traffic spikes and gives you a lever for tiered pricing. Implement it as middleware keyed by API key or user ID, backed by Redis or an in-memory token bucket for a single instance:

async function rateLimit(req, res, next) {
  const key = `rl:${req.user.id}`;
  const count = await redis.incr(key);
  if (count === 1) await redis.expire(key, 60);
  if (count > 100) return res.status(429).json({ error: "rate limit exceeded" });
  next();
}

Return standard Retry-After and X-RateLimit-* headers so clients can back off correctly instead of hammering the gateway.

Step 6: Handle cross-cutting transformations

Gateways are a good place to normalize responses, strip internal fields, aggregate calls to multiple services for a single client request (the backend-for-frontend pattern), and add correlation IDs for tracing. Keep this logic thin — if you find yourself writing significant business logic in the gateway, that's a signal it belongs in a service instead.

Step 7: Add observability and roll out gradually

Log every request with status code, latency, and upstream service at the gateway level — this gives you a single place to see cross-service performance instead of stitching logs from ten services together. Roll the gateway out incrementally: put it in front of one or two low-risk services first, verify latency and error rates match direct calls, then migrate the rest of your traffic. Keep a fallback path (direct service URLs) available during the transition in case the gateway introduces an unexpected bottleneck.

Common pitfalls

• Single point of failure: run at least two gateway instances behind a load balancer; a gateway outage takes down everything behind it.
• Latency creep: every hop adds milliseconds — measure gateway overhead specifically, not just end-to-end latency.
• Config drift: keep gateway route configuration in version control, reviewed like code, not edited by hand in a dashboard.
• Over-centralizing logic: resist the urge to put authorization decisions specific to a resource in the gateway — that couples every service's business rules to gateway deploys.

What is the difference between an API gateway and a reverse proxy?

A reverse proxy forwards requests to backends; an API gateway does that plus auth, rate limiting, transformation, and routing logic aware of your API's structure. Every API gateway is a reverse proxy, but not every reverse proxy handles those extra concerns.

Do I need an API gateway for a small number of microservices?

Not necessarily. If you have two or three services and one client, a gateway adds operational overhead without much benefit. It becomes worthwhile once you have multiple client types, need centralized auth/rate limiting, or want to hide internal service boundaries from consumers.

Should the gateway call multiple services for one client request?

It can, using the backend-for-frontend pattern, but keep the aggregation logic simple (parallel calls, basic merging). Complex orchestration with retries and compensation logic belongs in a dedicated orchestration service, not the gateway itself.