← Community

Cut llm token costs

Reduce an inference bill by measuring where tokens actually go, then applying caching, routing, and output limits in order of payoff. Use when LLM costs are rising, before scaling a feature, or when latency and cost both need cutting.

---
name: cut-llm-token-costs
category: ai-agents
description: >-
  Reduce an inference bill by measuring where tokens actually go, then applying
  caching, routing, and output limits in order of payoff. Use when LLM costs are
  rising, before scaling a feature, or when latency and cost both need cutting.
---

# Cut LLM Token Costs

Inference bills are usually dominated by one or two call sites nobody has looked
at. Guessing which produces confident, useless optimisation — the classic being
someone shortening a prompt by 200 tokens on a path that runs twice a day while a
retry loop burns millions.

Measure, then apply the fixes in payoff order.

## Measure first

Log per call: `feature`, `model`, `prompt_tokens`, `completion_tokens`,
`cached_tokens`, `latency_ms`, and whether it was a retry.

Then produce one table: **cost by feature**, descending. It is almost always
top-heavy, and it is frequently not where anyone expected. Optimising anything
outside the top three is a waste of your time.

Two numbers people forget to include:

- **Retries.** A failure loop can quietly double a bill.
- **Output tokens.** They usually cost several times input tokens, so a verbose
  model can be more expensive than a nominally pricier terse one.

Shared publicly by the author and open for any use. See the Terms.