LLM Cost Analysis: A Practical Framework for Teams

LLM cost analysis is the process of collecting your actual usage data, breaking it down by model, feature, and team, and turning that into decisions about what to keep, cut, or optimize. It's different from a cost estimate (a forecast before you ship) — analysis happens after you have real traffic, and its output should be a report or dashboard that someone actually acts on.

This article walks through how to structure that analysis: what data you need, how to segment it, which metrics matter, and how to present findings so they lead to real changes instead of sitting in a spreadsheet nobody opens again.

Why Ad Hoc Cost Tracking Fails

Most teams start with a monthly invoice from their LLM provider and a vague sense of "that seems like a lot." The problem is that a single total number tells you nothing actionable. You can't tell:

• Which feature or endpoint is driving spend
• Whether cost is growing because of more users or more waste per request
• Which prompts are burning tokens unnecessarily
• Whether a cheaper model would work just as well for a given task

A proper analysis requires request-level data — not just a total bill, but a table with one row per API call, including model, input tokens, output tokens, latency, and some tag for what feature or customer generated it.

Step 1: Collect Request-Level Data

At minimum, log the following for every LLM call:

timestamp, model, endpoint/feature, input_tokens,
output_tokens, cached_tokens (if applicable), latency_ms,
status (success/error/retry), user_or_team_id

If you're calling providers directly, this means adding logging middleware around every call site — easy to forget, hard to keep consistent across a codebase with multiple integrations. If you're routing calls through a gateway that already returns structured usage metadata per request, you skip that step. SubToAPI, for example, returns token counts and usage metadata with every response, and the dashboard aggregates it by API key, which gives you a natural per-feature or per-team breakdown if you issue separate keys for separate use cases. See /docs/messages for the response format.

Step 2: Segment Spend, Don't Just Sum It

Once you have raw data, break it down along at least three axes:

By feature or endpoint. A chatbot, a summarizer, and a code review tool have completely different cost profiles. Summing them into one number hides which one is actually expensive.

By model. If you're calling multiple models (a fast cheap one for simple tasks, a larger one for complex reasoning), track spend separately. It's common to discover that 80% of spend comes from a model that's only used for 20% of requests — sometimes that's justified, sometimes it's a routing bug.

By customer or team. If you're building a product with per-customer usage, know your cost per customer. This is the number that tells you whether your pricing actually covers your infrastructure cost, and it's the first thing you'll need if you ever negotiate a usage-based pricing tier.

A simple SQL-style breakdown looks like this:

SELECT feature, model,
  SUM(input_tokens) AS total_input,
  SUM(output_tokens) AS total_output,
  COUNT(*) AS requests
FROM llm_calls
WHERE timestamp >= '2025-01-01'
GROUP BY feature, model
ORDER BY total_output DESC;

Step 3: Separate Volume Growth from Inefficiency

When spend goes up month over month, the first question is always: more requests, or more tokens per request? These require completely different responses.

• More requests usually means more users or more usage per user — generally a good sign, and the response is to make sure margins hold as you scale.
• More tokens per request at a stable request count usually means prompt bloat: growing system prompts, uncontrolled conversation history, or retrieved context that keeps expanding. This is a bug, not growth, and it's fixable without touching the product.

Plot average tokens per request over time, separately for input and output. A steadily climbing input-token average with flat output is the most common silent cost leak — it usually means someone added more context to a prompt template and nobody ever trimmed it back.

Step 4: Compute Cost Per Unit of Value

Total spend is a lagging indicator. The number that actually matters is cost per unit of value — cost per completed support ticket, cost per generated report, cost per resolved query. This requires joining your LLM usage data with whatever business event marks "value delivered."

If cost per unit of value is flat or falling while total spend rises, you're in a healthy scaling pattern. If cost per unit of value is rising, something is wrong — retries, redundant calls, or a model upgrade that didn't earn its higher price.

Step 5: Turn It Into a Report Someone Reads

A cost analysis that stays in a notebook doesn't change anything. Package findings into something recurring:

• A weekly or monthly summary: total spend, spend by feature, spend by model, month-over-month delta
• A short list of anomalies: any feature or key whose cost grew disproportionately to its usage
• One or two concrete recommendations, not just charts

If you're already using a gateway with a billing dashboard, this step is mostly free — you get usage broken down by key and time period without building anything. Check /pricing to see which plan tier includes the reporting granularity you need; team and scale plans generally give you more per-seat visibility than a single shared key would.

Building This Without a Gateway

If you're calling model providers directly and don't want to build logging infrastructure from scratch, a lightweight option is to route all your traffic through a single proxy layer that logs and exposes usage automatically. This is one of the practical reasons teams adopt an API layer like SubToAPI even when they already have model access — /docs/quickstart shows the minimal setup, and the per-key usage view gives you segmentation by feature or team for free, without writing your own logging pipeline.

FAQ

How often should I run an LLM cost analysis? Weekly for active development, monthly for stable production systems. Set up an automated summary rather than doing it manually each time — the value comes from noticing trends, and trends require consistent, regular data points.

What's the difference between cost analysis and cost estimation? Estimation predicts spend before you build something, based on assumed usage. Analysis uses real, logged usage data after the fact. You need both — estimation to decide whether to build, analysis to check if the estimate held up and to catch drift over time.

Do I need a data warehouse to do this properly? No. A simple table of request-level logs with a handful of dimensions (model, feature, tokens, timestamp) is enough for most teams. Aggregation can be done with basic SQL or even a spreadsheet pivot table until volume grows large enough to justify more tooling.