How API Gateway Works in Microservices Architecture

An API gateway in a microservices architecture works by sitting between clients and your backend services as a single entry point. Instead of a mobile app or frontend calling ten different services directly, it sends one request to the gateway, which then figures out which service (or services) should handle it, forwards the request, collects the response, and sends a single answer back.

That's the short version. The longer version is that the gateway isn't just a router — it's where you centralize the cross-cutting concerns that would otherwise be duplicated across every service: authentication, rate limiting, request logging, response shaping, and sometimes protocol translation. This article walks through the mechanics of how that actually happens and what decisions it forces on your architecture.

The basic request flow

When a client sends a request to a microservices system fronted by a gateway, the sequence typically looks like this:

1. Client sends a request to a single public host, e.g. api.example.com/orders/123.
2. Gateway terminates the connection, handling TLS, and parses the request path/headers.
3. Authentication check — the gateway validates an API key, JWT, or session token before anything reaches internal services.
4. Routing decision — based on the path, method, or headers, the gateway decides which upstream service owns this request.
5. Request forwarding — the gateway rewrites the request as needed (path, headers, sometimes body) and sends it to the internal service, usually over a private network.
6. Response handling — the gateway receives the service's response, may transform it, applies rate-limit headers or logging, and returns it to the client.

None of this requires the client to know that orders, payments, and inventory are three separate services running on three separate hosts. That knowledge lives entirely in the gateway's routing table.

Client → Gateway → [auth check] → [route match] → Service (orders-svc:8081)
                                                  → Service (inventory-svc:8082)
       ← Gateway ← [merge/transform] ←

Why microservices specifically need this layer

A single monolith doesn't need a gateway in this sense — it has one process, one host, one set of concerns. Microservices split that into many independently deployed units, and that split creates problems a gateway is built to solve:

• Service discovery. Services scale up and down, get redeployed, and change IPs. The gateway (often paired with a service registry) keeps track of where each service currently lives so clients don't have to.
• Auth duplication. Without a gateway, every service reimplements token validation, API key checks, and rate limiting. With one, this logic exists in exactly one place.
• Request aggregation. A single client request sometimes needs data from three services (user profile, order history, recommendations). The gateway can call all three and merge the response, so the client makes one round trip instead of three.
• Protocol and version translation. Internally you might run gRPC between services but expose REST or JSON over HTTP externally. The gateway is the translation point.
• Consistent observability. Every request passes through one component, so logging, tracing IDs, and metrics can be captured centrally instead of stitched together after the fact from a dozen services.

Routing patterns you'll actually see

Gateways implement routing in a few common ways, and it's worth knowing the difference because it affects how you structure your services:

• Path-based routing — /orders/ goes to the orders service, /users/ goes to the users service. Simple, predictable, the default for most setups.
• Header or version-based routing — routing on an Accept header or a custom X-API-Version header, useful for running multiple API versions side by side without changing URLs.
• Weighted routing — splitting traffic between two versions of a service (canary releases), gradually shifting percentage to the new version.
• Aggregation routing — one incoming endpoint fans out to multiple internal services and the gateway assembles a composite response.

Most production gateways combine path-based routing as the default with weighted or header-based rules layered on top for specific endpoints.

The tradeoffs it introduces

An API gateway solves real problems, but it's not free:

• It becomes a single point of failure if not run with redundancy — if the gateway goes down, every service behind it is effectively unreachable.
• It adds a network hop, which adds latency, usually single-digit milliseconds but worth measuring under load.
• Routing logic and auth rules live in a shared component that every team depends on, which means changes need coordination, unlike a service a single team fully owns.

These tradeoffs are why some smaller systems delay adopting a gateway until the number of services and clients actually justifies the operational overhead.

A concrete example: gateways for third-party API access

The same pattern shows up outside of internal microservices, in how you expose access to a provider's API. SubToAPI applies exactly this gateway model to Claude access: instead of every internal service or team member handling raw provider credentials, you get one HTTPS endpoint with your own sub_live_... keys, and the gateway handles routing, streaming, and usage metadata behind it.

curl https://api.subtoapi.app/v1/messages \
  -H "Authorization: Bearer $SUBTOAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-5",
    "max_tokens": 512,
    "messages": [{"role": "user", "content": "Summarize this ticket."}]
  }'

Each service or team member gets its own key, requests are logged centrally, and nobody needs direct access to the underlying provider account — the same benefits a gateway gives you in a microservices system, applied to a single external API. Check the quickstart or messages docs if you want to see the request/response shape in detail, and pricing covers the Solo, Team, and Scale plans.

questions

Does every microservices system need an API gateway? No. Systems with a handful of services and a single client type can often route directly or use a simple reverse proxy. Gateways earn their complexity once you have multiple client types, need centralized auth, or want to hide internal service topology from the outside world.

Does the gateway talk to services synchronously or can it queue requests? Most gateway traffic is synchronous request/response over HTTP or gRPC. Some setups pair the gateway with an async layer (message queues) for long-running work, but the gateway itself typically just forwards and waits for a response within a timeout window.

Is an API gateway the same as a load balancer? No. A load balancer distributes traffic across identical instances of one service. A gateway routes to different services based on the request and adds cross-cutting concerns like auth and aggregation on top — many gateways sit in front of a load balancer, not instead of one.