LLM Gateway vs Agent Gateway: What's the Difference?

An LLM gateway sits between your application and one or more model providers, normalizing requests, handling API keys, and giving you usage metadata across models. An agent gateway sits a layer higher: it manages agent identity, permissions, tool execution, and multi-step orchestration across one or more agents that may themselves call LLMs underneath. The short answer is that they solve different problems — one is about talking to models, the other is about controlling what autonomous systems are allowed to do.

If you're evaluating which one you need, the practical test is this: are you routing chat/completion requests to a model provider, or are you managing something that takes actions, calls tools, and makes decisions across multiple steps without a human in the loop for each one? The first is an LLM gateway problem. The second is an agent gateway problem. Many teams end up needing both, stacked on top of each other.

What an LLM gateway actually does

An LLM gateway is the infrastructure layer for model access. Its core responsibilities:

• Provider abstraction — one API shape regardless of whether requests go to Claude, GPT, or another model
• Key management — application-level API keys instead of raw provider credentials in every service
• Usage tracking — tokens, cost, and latency per key, per team, per endpoint
• Streaming and tool-call passthrough — forwarding SSE streams and structured tool_use blocks without breaking them
• Rate limiting and retries — protecting both your budget and provider quotas

This is the layer SubToAPI operates at. If you already have a Claude subscription and want to expose it as a standard HTTPS API — with sub_live_... keys, streaming, tool use, and per-key usage in a dashboard — that's exactly what an LLM gateway does. See /docs/quickstart for the setup and /docs/messages for the request format.

What an LLM gateway does not do: decide which tools an agent is allowed to call, enforce multi-step task boundaries, or manage agent-to-agent communication. It's a request/response layer, even when streaming.

What an agent gateway actually does

An agent gateway is a control plane for autonomous or semi-autonomous systems. It typically handles:

• Agent identity — distinguishing which agent, workflow, or task is making a request
• Permission scoping — restricting which tools, data sources, or actions a given agent can invoke
• Tool execution mediation — sitting between an agent's tool-call intent and the actual execution, often with approval steps
• Orchestration state — tracking multi-turn, multi-agent workflows, handoffs, and task completion
• Audit trails for actions — not just "what was said" but "what was done" (a file written, an email sent, an API called)

Agent gateways exist because agents don't just generate text — they take actions with side effects. The risk profile is different. A misconfigured LLM gateway might leak a completion to the wrong log file. A misconfigured agent gateway might let an agent delete a production database or send an unauthorized payment.

Where the two layers overlap

In practice, most agent frameworks call an LLM under the hood for reasoning, then use their own logic to decide what to do with the model's tool-call output. That means:

Application
   → Agent Gateway (permissions, tool routing, audit)
        → LLM Gateway (model access, streaming, usage)
             → Model Provider (Claude, etc.)

The agent gateway doesn't replace the LLM gateway — it wraps it. If you build an agent that needs Claude's tool-use capability, the LLM gateway is still the thing translating your tool schema into a proper tool_use request and parsing the response. See /docs/tools for how tool definitions and responses are structured over the API.

This is a common point of confusion in vendor marketing: some "agent gateway" products are really LLM gateways with an orchestration UI bolted on, and some "LLM gateway" products have added lightweight permission rules and started calling themselves agent-aware. Read the feature list, not the label.

A concrete example

Say you're building a support bot that can look up order status and issue refunds.

LLM gateway does this part:

curl https://api.subtoapi.app/v1/messages \
  -H "Authorization: Bearer $SUBTOAPI_KEY" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-sonnet-4",
    "max_tokens": 512,
    "messages": [{"role": "user", "content": "Refund order 4471"}],
    "tools": [{"name": "issue_refund", "input_schema": {...}}]
  }'

It sends the request, streams the response, returns a tool_use block, and logs tokens against your key. That's the full extent of its job.

Agent gateway does this part: deciding whether this specific agent, in this specific context, is allowed to call issue_refund at all, whether the amount exceeds an auto-approval threshold, whether a human needs to confirm, and logging the action once it executes. None of that is model inference — it's policy and execution logic that sits around the model call.

If you skip the agent gateway layer and just wire tool_use output directly to a refund API, you've built an LLM-powered feature, not a governed agent. That's fine for low-risk actions and often overkill for simple use cases — but it's a deliberate tradeoff, not an accident.

Which one should you build or buy first

• If you're building a single-model chat feature, a coding assistant, or a summarization pipeline — you need an LLM gateway, full stop. Check /pricing if you're comparing costs against running raw provider API calls yourself.
• If you're building something that autonomously takes actions across multiple tools or systems — you need an agent gateway, and it will call into an LLM gateway (yours or a provider's) as one of its components.
• If you're not sure yet, start with the LLM gateway. It's the smaller, more foundational piece, and adding agent-level permissioning on top later is easier than retrofitting model access into an agent framework that assumed direct provider calls.

questions

Is an agent gateway a replacement for an LLM gateway? No. An agent gateway typically sits on top of an LLM gateway, using it for model access while adding permission scoping, tool execution control, and audit logging for actions the agent takes.

Do I need an agent gateway if my app only answers questions? Not usually. If your system doesn't execute actions with side effects (writing data, calling paid APIs, sending messages), an LLM gateway alone covers key management, streaming, and usage tracking.

Can I add agent-level controls on top of SubToAPI? Yes — SubToAPI handles the LLM gateway layer (keys, streaming, tool_use passthrough, usage metadata via /docs/streaming and /docs/tools), and you can build permissioning logic for your agents on top of it.