An inference bill is not a mystery number. It is arithmetic over tokens, and once you can see the arithmetic you can change it deliberately. This is the model I would use to understand a request, measure a real cost per outcome, and find the few levers that matter.
Cost per request is arithmetic
For a chat request, the cost is:
cost = input_tokens * input_rate + output_tokens * output_rateCache hits are priced below fresh input, so any tokens served from cache reduce the first term. At the published Standard rates, input is $0.40 per 1M tokens and output is $1.60 per 1M tokens, so an output token costs four times an input token. That single ratio is the most useful thing to know about a chat bill: output is the expensive half.
Read the usage object, not your own estimate
Do not estimate tokens with a local tokenizer and hope it matches. Read
the usage object from the response and record it. Billing
follows the provider's count, not yours, and the gap between an
estimate and the real count is exactly the kind of drift that makes a
bill feel unpredictable.
For a quick what-if before you build anything, the published calculator at https://webwright.ai/tools/cost-calculator does the same multiplication with the current rates.
The three levers that move the number
- Output length. Because output is priced above input,
the cheapest large win is usually shorter completions. A tighter
system prompt, a clear output schema, and a hard
max_tokenscap on routine calls cut the expensive half directly. - Cache reuse. Repeated context (a long system prompt, a shared document, a fixed instruction block) is where cache hits pay off, because a cached token is billed below a fresh input token. Structure prompts so the stable prefix is identical across calls.
- The right tier for the work. Standard and Premium carry different rates. Match the tier to the workload rather than paying Premium rates for work that does not need them.
Cost per successful outcome, not per call
A per-call number flatters a retry-heavy pipeline. Count the calls that fail, retry, or get thrown away, and divide total cost by the outcomes you actually kept. A wrapper with safe retries and a timeout (see the resilience guide) can lower cost per outcome even when it raises the raw call count, because it stops a failed call from cascading.
A worksheet you can keep
| Line | Value | Where it comes from |
|---|---|---|
| Input tokens per call | observed | usage.prompt_tokens |
| Output tokens per call | observed | usage.completion_tokens |
| Input cost | input_tokens * rate | published rate |
| Output cost | output_tokens * rate | published rate |
| Retries per outcome | observed | your logs |
| Cost per outcome | (input + output) * retries | this worksheet |
Keep input and output separate, keep the tier on the sheet, and re-run the sheet when the workload changes.
Measuring over time
Record tokens and cost per request on every call, then aggregate by day and by feature. The number that matters is cost per outcome trending over a week, not a single request. If it climbs, the levers above tell you where to look first.
When a working session helps
If your bill is growing faster than usage and you are not sure which lever to pull first, that is the exact problem the consulting session is for. In 30 minutes you leave with a prioritized first use case and a rough plan. Book one using the link below.
The WebWright Team