Self-Hosted LLM API Gateway: Open Source Options
What a self-hosted LLM API gateway actually does
A self-hosted LLM API gateway is a server you deploy yourself that sits between your application and one or more LLM providers (Claude, OpenAI, local models, etc.). It normalizes requests, handles auth, routing, rate limiting, caching, and logging, and exposes a single HTTP API your app talks to instead of calling each provider's SDK directly. People search for this because they want provider flexibility, cost control, and data residency without being locked into someone else's infrastructure.
The short answer: yes, there are solid open source options (LiteLLM, Portkey's open source gateway, Helicone, LangChain's gateway tooling, and plain reverse proxies like Kong or Envoy with custom plugins), but "open source" doesn't mean "free to run." You still need to provision servers, manage TLS, handle secrets, monitor uptime, and patch the thing every time a provider changes their API. If your actual goal is just "give my app a clean API key and usage dashboard for Claude," a managed service like SubToAPI gets you there in minutes without the ops burden — more on that trade-off below.
Why teams look for a self-hosted gateway
Common reasons developers search this out:
- Multi-provider routing — fallback from Claude to OpenAI to a local model if one is down or rate-limited.
- Centralized key management — one place to rotate provider API keys instead of scattering them across services.
- Cost and usage visibility — per-team, per-project token tracking without building it from scratch.
- Compliance/data residency — some orgs need requests to pass through infrastructure they control.
- Rate limiting and caching — protecting upstream quota and cutting duplicate calls.
Not every reason requires self-hosting. Key rotation, usage dashboards, and per-key rate limits are things a hosted gateway can give you with zero infrastructure. Multi-provider routing and strict data residency are the cases where self-hosting genuinely matters.
Open source options worth evaluating
LiteLLM
LiteLLM is the most popular option for this specific use case. It exposes an OpenAI-compatible /chat/completions endpoint and translates requests to over 100 providers, including Anthropic's Claude models. It supports virtual API keys, budgets per key, and a Postgres-backed usage database. Good fit if you want OpenAI-SDK compatibility across providers and are comfortable running a Python service plus a database.
Portkey Gateway (open source core)
Portkey's gateway core is open source and focuses on request routing, retries, and fallbacks between providers. The hosted version adds observability and caching, but the core proxy can run standalone. Good fit if routing logic (fallback chains, load balancing across API keys) is your main pain point.
Helicone
Helicone started as an observability proxy — you point your existing provider calls through it and get logging, cost tracking, and caching for free, self-hosted via Docker. It's less of a full gateway and more of a transparent proxy layer you drop in front of calls you're already making directly to Anthropic or OpenAI.
Generic reverse proxies (Kong, Envoy, Traefik)
If you already run API infrastructure, you can build gateway behavior with plugins: rate limiting, auth, logging. This gives maximum control but means writing and maintaining your own translation layer for each provider's request/response format — Claude's Messages API shape is not the same as OpenAI's Chat Completions shape, so you own that mapping.
What self-hosting actually costs you in time
Every option above requires:
- A server or container runtime (VM, Kubernetes, or at minimum a Docker host) with uptime SLAs you're responsible for.
- A database for usage/key tracking in most cases.
- TLS certificates and DNS you manage.
- Monitoring and alerting so you know when the gateway itself is down — not just when the upstream provider is down.
- Ongoing maintenance when Anthropic or OpenAI change request/response schemas, deprecate models, or update rate limit headers.
None of this is hard individually, but it adds up to a real ops surface for something that, for a lot of teams, is just "I want a stable API key in front of Claude with usage tracking." If multi-provider routing and infrastructure control aren't actually required by your use case, running your own gateway is often solving a problem you don't have.
A managed alternative: SubToAPI
SubToAPI is built for the common subset of this problem: turning your Claude access into a clean HTTPS API without deploying or maintaining anything. You get application API keys (sub_live_...), streaming responses, tool use support, usage metadata per key, and team seats, all from a dashboard — no server, no database, no cert management.
A basic request looks like a normal REST call:
curl https://api.subtoapi.app/v1/messages \
-H "Authorization: Bearer $SUBTOAPI_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "claude-sonnet-4",
"max_tokens": 1024,
"messages": [{"role": "user", "content": "Summarize this ticket."}]
}'If you need streaming or tool calling, the same key works across both — see streaming and tool use for the request formats, or start with the quickstart and messages API reference. Plans start with a free trial at signup, with Solo at €9, Team at €19/seat, and Scale at €49/seat on the pricing page.
The trade-off is honest: SubToAPI doesn't do multi-provider routing to OpenAI or local models — it's focused on Claude. If you need a gateway that fails over across multiple LLM vendors, an open source option like LiteLLM is the right tool. If you need a stable, documented API in front of Claude with per-key usage and team management and you don't want to run infrastructure, a hosted option saves real time.
How to choose
- Need multi-provider fallback or strict data residency? Self-host LiteLLM or build on Kong/Envoy.
- Need request/response observability on calls you already make directly? Helicone's proxy mode is lightweight.
- Need a clean, maintained Claude API with keys, streaming, tools, and usage tracking, with no infra to run? A managed option like SubToAPI gets you there faster.
- Running a large team with strict compliance requirements? Self-hosting gives you the control, but budget real engineering time for maintenance, not just initial setup.
questions
Is a self-hosted LLM gateway actually free? The software license is free, but you pay in server costs, database hosting, TLS/DNS setup, and the engineering time to maintain it as provider APIs change.
Do I need a gateway if I only use one provider? Not necessarily for routing, but a gateway (self-hosted or managed) still helps with key rotation, rate limiting, and usage tracking even with a single provider like Claude.
What's the difference between a gateway and a proxy like Helicone? A full gateway handles routing, auth, and request translation across multiple providers; a proxy like Helicone mainly adds logging and caching in front of calls you're already making directly.