How to Audit LLM Spend: Tools and Techniques for Cost Control

Most organizations cannot explain their largest LLM cost spike until the invoice arrives. By then, the expensive feature is already in production, and engineering teams are reacting to overspend rather than preventing it.

The reason is often a lack of cost attribution. Without it, teams cannot trace spending to the feature, team, prompt change, model upgrade, or deployment responsible for the increase.

This guide gives you a 30-minute audit methodology, a tool comparison, and a framework for turning that one-time audit into ongoing monitoring.

Understanding What You’re Paying For

LLM pricing has moved from flat per-token rates to layered models: usage tiers, committed-spend discounts, and provider-specific token types billed differently. Anthropic discounts cache reads by 90% but applies a 25% premium on the initial cache write — a detail often left out of savings estimates.

Reasoning tokens (o-series, GPT-5.x, Gemini “thinking,” Claude extended thinking) bill at the same rate as standard output — there’s no separate higher price tier. What drives up the bill is volume: a model can generate several times more reasoning tokens than the visible answer requires.

Your total spend is a function of model mix, caching behavior, retry patterns, and token-type distribution — not simply tokens × price. The main cost drivers:

• API call volume and model tier — the combination of model, token count, and call frequency.
• Output tokens — typically 4–8x the cost of input tokens depending on tier, so long-output features (reports, commented code, detailed explanations) dominate spend.
• Context compounding — conversation history is re-sent as input on every turn, so long sessions and verbose tool outputs (file reads, logs, search results) get re-billed repeatedly; tool output alone can account for over half of a typical coding-agent bill.
• Retry storms and runaway agent loops — transient errors or malformed tool results can trigger repeated billing across every retry or loop iteration.
• Prompt and context bloat — larger system prompts and retrieved context raise the average cost per request, ahead of showing up as a total spend problem.

Step 1: Establish Cost Attribution

What to track

• Spend by project, team, model, and month-over-month, to see trends and concentration.
• Input vs. output tokens, tracked and alerted on independently — the ratio between them is a leading indicator of anomalies.
• Token types split into input, output, cached input, and reasoning buckets where the provider exposes them.
• Cost per task or workflow, so you have unit economics you can track over time.

Data sources

• Provider dashboards and APIs — OpenAI Usage API/CSV export, Anthropic Billing API, Google Cloud Billing, self-hosted inference logs.
• Request logs — application logs or tracing platforms capturing token counts, model, and metadata per call.
• Gateway or observability platforms — tools like Braintrust, Langfuse, and Helicone that normalize and tag usage across providers.

Attribution options

• Separate API keys per team or project, tracked in provider dashboards or via a proxy such as LiteLLM.
• Request tagging — attach feature, team, tenant, and prompt-version metadata using provider fields or SDK-level tracing.
• Spreadsheet or BI aggregation — export raw data, normalize cost, and roll up by tag for a first pass.

Cost forecast template

Forecast = Baseline spend × (1 + expected growth) × (1 − planned efficiency gains), where baseline is average daily or weekly spend by feature and model, growth is expected traffic or rollout impact, and efficiency covers planned optimizations like caching, model routing, or prompt compression. Update monthly against actuals.

Step 2: Choose Your Audit Tools

Tool categories

• Vendor-native: OpenAI Usage API, Anthropic Admin/Billing API, Google Cloud Billing.
• Platforms: Braintrust, Langfuse, Helicone, LiteLLM, OpenLLMetry, Pezzo, FinOps LLM.
• DIY: custom scripts or spreadsheets that normalize provider CSVs against internal logs.

Tool comparison

Decision tree

• 1–2 providers, under $5k/month: vendor-native dashboards plus spreadsheet attribution.
• 3+ providers or $5k–$50k/month: a proxy (LiteLLM) or observability platform (Langfuse, Braintrust) for cross-provider normalization and tagging.
• Over $50k/month or multi-team chargeback: an end-to-end trace and ownership contract with request-level tagging and automated alerts, potentially via a dedicated FinOps platform.
• Coding-heavy workflows: add a token-compression layer alongside whichever platform you choose above.

Step 3: Run Your First Audit

Step 4: Set Up Continuous Monitoring

Cadence

• Weekly: high-spend teams review top drivers, cost-per-task trends, and alert fires.
• Monthly: all teams run the full audit; update the forecast and budget allocations.
• Quarterly/annually: review pricing changes, provider contracts, and the optimization roadmap.

Key metrics

• Total cost over time (daily/weekly/monthly).
• Cost breakdown by agent, team, and model.
• Average cost per task or user interaction.
• Input-to-output token ratio per feature — a sudden shift signals a prompt or behavior change.
• Cost per successful eval — spend divided by requests passing quality thresholds.

Common Audit Findings

• Spend above the PoC estimate — usually context compounding, verbose outputs, or unbounded agent loops that weren’t present under controlled PoC conditions.
• Unexplained “other” spend — shared API keys or missing request tags; fix by enforcing tagging at the gateway.
• Month-over-month spikes — often correlate with prompt-version changes, feature rollouts, or retry storms during provider outages.
• API key sprawl — multiple untracked keys per team fragment visibility; consolidate under a proxy or observability platform.

From Audit to Action

Running this audit gives you cost visibility, spend attributed to teams and features, and alerts that catch anomalies early — the prerequisites for optimization you can’t do without measurement first.

Once cost is integrated into system design from day one rather than discovered after launch, AI stops being a runaway corporate expense and starts behaving like any other mature component in the tech stack — exceptionally powerful where it fits, ruinously expensive where misapplied, and valuable when governed with discipline.

Marcin Marczyk

Delivery Director, Addepto

Who Owns This

Optimizing LLM spend is often treated as a model-layer problem — pick a cheaper model, tune a prompt, enable caching. But the cost drivers in this audit sit at the boundary between the model call and the surrounding application: context compounding, retry storms, and tool-output bloat are architectural issues, not prompting issues.

Practitioners increasingly frame this as a three-function problem, and warn that assigning it to a single function is a primary reason cost-optimization efforts fail:

• Engineering owns the actual cost-per-task levers — model selection, routing, caching, and retry/circuit-breaker design — because these require commit access to the request pipeline, not just dashboard visibility.
• Platform/infrastructure owns the tagging standards, gateway configuration, and allocation rules that make attribution possible in the first place; without this, teams build incompatible tracking schemes independently.
• Finance/FinOps owns budgets, thresholds, and review cadence, but can’t own the fix — cost targets set without engineering-level implementation authority tend to produce reporting theater rather than actual savings.

Whether “LLM-specific expertise” deserves recognition as a fourth, distinct skill is contested, not settled. Some practitioners treat prompt and context design — tool contracts, output schemas, eval design — as a specialized skill alongside software engineering; others argue it’s an extension of requirements engineering, fully absorbed into existing roles. What’s not contested: someone on the team needs deep familiarity with token pricing tiers, cache mechanics, and provider-specific behavior — without that, a technically correct architecture can still misdiagnose where the money is going.