LLM Cost Tracking: Attribute Spend by User, Feature, and Model

Contents

Tag every call, attribute every dollar, and answer "which feature caused that?" before your coffee cools

Your LLM bill arrived. It's big. Someone asks, "Which feature caused that?" and you stare at a single line item that says "API usage."

I lived that moment last year. Our monthly spend jumped 60%, and nobody could say why. It took us two days of log digging to find a retry loop in one little summarization feature, quietly burning tokens around the clock.

This guide shows you how to build LLM cost tracking that answers that question in seconds. You'll learn what to log, how to attribute spend by user, feature, and model, and how to act on the numbers — the unit-economics half of the LLMOps production guide.

Why Your Provider Dashboard Isn't Enough

Provider dashboards show total spend and maybe a breakdown by model or API key. That helps a little. It can't tell you which customer, feature, or prompt version drives the cost.

You need that detail for real decisions:

• Pricing: which customers cost more to serve than they pay?
• Product: which features earn their token budget?
• Engineering: which prompt change doubled the cost per request?
• Finance: what will next quarter's bill look like?

Without attribution, you guess. With it, you decide.

Know What Actually Drives LLM Cost

Before you track anything, understand what you pay for. Most providers charge per token, and different token types carry different prices:

• Input tokens: your system prompt, user message, retrieved context, and conversation history.
• Output tokens: the model's reply, which usually costs more per token than input.
• Cached input tokens: repeated prompt prefixes that some providers discount.
• Reasoning tokens: hidden "thinking" tokens on some models that bill as output.
• Embeddings and tools: separate calls for retrieval, search, or other services.
• Retries and failures: calls that fail or repeat still cost money.

Output length and conversation history cause the sneakiest growth. A chat that runs 20 turns resends all prior turns on every call — that's the same compounding the caching guide kills with prefix reuse.

Decide Your Attribution Dimensions

Attribution means tagging each call with the context that explains it. Pick your dimensions deliberately, because you can't add them retroactively to old logs.

The Core Four

Start with these:

• User or tenant ID: who triggered the spend? Use an internal ID, not an email address.
• Feature name: which product surface made the call — chat, summarize, or search_rerank?
• Model: the exact model name and version.
• Prompt version: the release of the prompt template in use.

Useful Extras

Add these once the basics work:

• Environment: production, staging, or evals. Test traffic shouldn't muddy your numbers.
• Request or session ID: a key that links multiple calls in one user action.
• Outcome: success, failure, retry, or user feedback.
• Experiment variant: the A/B arm, if you test prompts.

IMO, the feature tag pays off fastest. It turns an anonymous bill into a map of your product.

Instrument Every Call in One Place

Scattered logging code leads to missing data. Put instrumentation in a single choke point instead. You have three solid options:

1. A wrapper function around your LLM client that every call uses.
2. Middleware or a callback in your framework.
3. An LLM gateway or proxy that sits between your app and the provider.

A gateway gives you the most coverage with the least code. A wrapper works fine for small codebases. Whichever you choose, forbid direct calls that bypass it.

What to Log on Every Call

Capture this minimum record:

• Timestamp and request ID.
• User, feature, model, prompt version, and environment.
• Input, output, cached, and reasoning token counts straight from the provider's response.
• Computed cost in dollars.
• Latency and status (success, error, retry).

Avoid logging full prompts and responses in your cost table. They bloat storage and raise privacy risk. Keep them in a separate, access-controlled trace store if you need them.

A Minimal Code Sketch

This example computes cost from token counts and emits a structured event. The prices are placeholders, so load your real rates from config:

PRICES = {  # USD per 1M tokens, placeholder values
    "model-small": {"in": 0.25, "out": 1.25},
    "model-large": {"in": 3.00, "out": 15.00},
}

def track(resp, user_id, feature, model, prompt_version):
    p = PRICES[model]
    usage = resp.usage
    cost = (usage.input_tokens * p["in"] + usage.output_tokens * p["out"]) / 1_000_000
    emit({
        "user_id": user_id,
        "feature": feature,
        "model": model,
        "prompt_version": prompt_version,
        "input_tokens": usage.input_tokens,
        "output_tokens": usage.output_tokens,
        "cost_usd": cost,          # the number finance will ask about
    })
    return resp

Keep Your Price Table Honest

Providers change prices, and they release new models often. A hard-coded rate from last spring will quietly corrupt your reports. Treat the price table like data, not code.

• Store prices in config or a database with an effective date.
• Record the price version on each event, so old costs stay reproducible.
• Review rates monthly against your provider's pricing page.
• Reconcile totals against the actual invoice — more on that below.

Build Reports That Answer Real Questions

Raw events don't help anyone. Aggregate them into a handful of views your team will open every week.

Cost by Feature

This view shows where your budget goes. Rank features by spend, then compare each against the value it delivers. A cheap feature nobody uses and an expensive one everybody loves need very different responses.

Cost by User or Tenant

This view reveals your heavy users. Expect a long tail: a small slice of customers often drives most of the spend. Compare each tenant's cost against their revenue, and you'll spot unprofitable accounts fast.

Cost by Model

This view shows how your model mix affects the bill. If an expensive model handles simple tasks, you've found a savings opportunity — routing easy requests to a smaller model is one of the biggest levers available.

Cost per Outcome

Total spend matters less than cost per successful outcome. Track cost per resolved ticket, per accepted suggestion, or per completed task. A feature that costs more but succeeds twice as often may win on unit economics.

SELECT feature,
       COUNT(*)      AS calls,
       SUM(cost_usd) AS spend,
       SUM(cost_usd) / NULLIF(COUNT(DISTINCT request_id), 0) AS per_action
FROM llm_usage
WHERE environment = 'prod'
GROUP BY feature
ORDER BY spend DESC;

Figure 1: Four views beat one invoice — feature, user, model, and cost per outcome

Set Budgets and Alerts

Visibility without alarms means you learn about problems after the invoice. Add alerts that fire early:

• Daily spend threshold for the whole app.
• Per-feature budgets that flag sudden jumps.
• Per-user rate and cost caps to stop runaway usage or abuse.
• Anomaly alerts when cost per request rises sharply after a deploy.

Pair cost alerts with your release process. When a prompt change goes out and cost per request jumps 40%, you want a Slack message within the hour — the same hour-level signal the CI/CD cost gate enforces before merge.

Handle Shared and Hidden Costs

Some costs don't map neatly to one user or feature. Decide how you'll allocate them, then stick to the rule:

• Shared system prompts and cached prefixes: split by usage share.
• Embedding and indexing jobs: tag them to the feature that owns the knowledge base.
• Background jobs and evals: track them under their own feature and environment tags.
• Failed calls and retries: count them. They cost real money, and they point to reliability problems.

Reconcile With the Invoice

Your internal numbers should match the provider's bill within a small margin. Compare them every month. A gap means missing instrumentation, stale prices, or untracked services.

Treat a mismatch as a bug. Hunt it down while the trail is fresh.

Use the Data to Cut Costs

Tracking pays for itself when you act on it. Once you see where money goes, these levers become easy to pull:

• Route by difficulty: send simple tasks to cheaper models.
• Trim prompts and history: cut redundant context and summarize old turns.
• Cap output length: set sensible max_tokens limits.
• Cache repeated work: reuse answers and prompt prefixes where safe.
• Batch non-urgent jobs: use discounted batch options when latency doesn't matter.

Always measure quality after each change. A cheaper prompt that hurts answers doesn't save money — it moves the cost somewhere else, which is exactly what the token budgets and routing guide measures end to end.

Mistakes I Made So You Don't Have To

Here's my short list of regrets:

• Tagging too late. I added feature tags after a month of data, and that month stayed unattributed forever.
• Logging prompts in the cost table. Storage costs and privacy headaches followed.
• Ignoring retries. Silent retries inflated one feature's cost by a third.
• Mixing eval and production traffic. Our test runs made one feature look wildly expensive.

Common Pitfalls

• Dimensions designed after the logs exist — retroactive attribution is impossible; pick the core four first day.
• Bypassable instrumentation — one direct SDK call anywhere and the report silently undercounts.
• Prices hard-coded in the source — last spring's rate quietly corrupts every dashboard after a provider repricing.
• Prompt and response bodies in the cost table — storage bloat plus privacy risk; use an access-controlled trace store.
• Retries uncounted — failed calls cost real money and hide reliability bugs at the same time.
• Evals mixed with production — test traffic distorts every per-feature number.
• Invoice never reconciled — without the monthly diff, missing instrumentation compounds for quarters.
• Raw totals as the headline metric — cost per successful outcome is the number that changes decisions.

Unlock AI That Actually Works

Get lifetime access to GPT-6 Astra, Claude Fable 5.1, Gemini 3.5, Grok 4.5, and more — all in one platform. Build websites, apps, videos, content, and digital products from a single command. No monthly fees. No tool-hopping.

Click here to get GPTAstra Max now — one-time payment, lifetime access.

Frequently Asked Questions

Why isn't a provider dashboard enough for LLM cost tracking?

Provider dashboards show total spend and maybe a breakdown by model or API key, but they can't tell you which customer, feature, or prompt version drives the cost — and those are exactly the dimensions real decisions need. Pricing asks which customers cost more to serve than they pay. Product asks which features earn their token budget. Engineering asks which prompt change doubled cost per request. Finance asks what next quarter's bill will look like. Without attribution you guess; with it you decide. The fix is tagging every call with the context that explains it, at a single choke point, before your logs ever contain gaps.

What dimensions should every LLM call be tagged with?

Start with the core four: user or tenant ID (an internal ID, never an email address), feature name (chat, summarize, search_rerank), the exact model name and version, and the prompt version in use. Then add useful extras: environment (production, staging, evals — test traffic shouldn't muddy your numbers), a request or session ID linking multiple calls in one user action, outcome (success, failure, retry, user feedback), and experiment variant if you A/B test prompts. The feature tag pays off fastest — it turns an anonymous bill into a map of your product. Capture everything at one wrapper, middleware, or gateway, and forbid direct calls that bypass it.

How do you keep LLM cost data accurate over time?

Three disciplines. First, treat the price table like data, not code: store rates in config or a database with an effective date, stamp the price version on each event so old costs stay reproducible, and review rates monthly against the provider's pricing page. Second, separate concerns — token counts and computed cost go in the cost table, while full prompts and responses live in a separate access-controlled trace store, because logging them inline bloats storage and raises privacy risk. Third, reconcile internal totals against the actual invoice every month; treat any mismatch as a bug (missing instrumentation, stale prices, or untracked services) and hunt it down while the trail is fresh.

How do you cut LLM costs without hurting quality?

Act on the reports: route simple tasks to cheaper models, trim prompts and conversation history (summarize old turns instead of resending them), cap output length with sensible max_tokens limits, cache repeated work and prompt prefixes where safe, and batch non-urgent jobs when latency doesn't matter. Measure quality after each change — a cheaper prompt that hurts answers doesn't save money, it moves the cost somewhere else. And put alarms around the whole system: daily spend thresholds, per-feature budgets, per-user rate and cost caps, and anomaly alerts when cost per request spikes after a deploy, so you hear about it within the hour rather than next month.

Wrapping This Up

You can't manage what you can't see. Tag every call, log tokens and cost in one place, keep your price table current, and report cost per outcome instead of raw totals. Then add alerts so surprises find you early.

Start with one wrapper and one feature tag this week. When the next big invoice lands, you'll answer "what caused that?" before your coffee cools.

Related Articles

• Reduce LLM API Costs: Token Budgets, Caching, and Model Routing
• Semantic Caching for LLM Apps: Cut Latency and Cost
• CI/CD for LLM Applications: Automated Tests Before Every Deploy
• What Is LLMOps? Complete Guide to Running LLM Apps in Production