Tokens as a cost model
Turn a feature's request volume into a monthly bill from tokens in, tokens out and cached prefixes, then size the three levers that bring it under budget.
On this page
A model bill has three lines: tokens in, tokens out, and tokens read from a cache. Everything a PM can do about the bill is a change to one of those three numbers or to the price each is multiplied by. The drafting feature — 200,000 requests a day, 1,500 tokens in, 300 out — is worked to $54,000 a month at assumed prices, then brought under its $30,000 budget with three levers, in the order to pull them.
What a token is, for budgeting
A token is a piece of a word: assume 1.3 tokens per English word, about four characters. The 1,500-token prompt is roughly 1,150 words: a 1,100-token system prompt (850 words of instructions, tone and policy that are identical on every call), the ticket, and the five retrieved chunks. The 300-token reply is about 230 words, a long support reply. Prices are quoted per million tokens, so divide by a million once and stay in dollars.
Count tokens; do not estimate them from word counts. Every vendor's API returns the exact input and output counts on each response, and the engineers can log them from day one. The constant is for the plan; the log is for the bill.
The bill, as specified
Input: 200,000 requests × 1,500 tokens = 300 million tokens a day. At an assumed $3 per million, $900 a day. Output: 200,000 × 300 = 60 million tokens a day. At an assumed $15 per million, $900 a day. Total $1,800 a day, $54,000 at 30 days.
Two things in that sum matter. Output is one fifth of the token volume and half the bill, because output tokens are priced five times input, as on seven of the nine published tiers in model selection. And the per-request figure, $1,800 ÷ 200,000 = $0.009, is what the AI-feature PRD's done line tests: under its $0.02 ceiling, and over its $0.005 monthly mean until the levers below bring it to about $0.0046. The monthly figure is for finance; the per-request figure is for the spec.
Against the budget: $54,000 is 1.8× the $30,000 the feature was given, which is itself half the $60,000 of add-on revenue in pricing an AI feature. Per seat, $54,000 ÷ 12,000 seats is $4.50 a month against $5 a seat of revenue. The feature works and keeps 50 cents a seat.
Lever 1: fewer output tokens
The eval says 70% of tickets need a short reply, and when the model is told to be brief those drafts run under 150 tokens. Output becomes 140,000 × 150 + 60,000 × 300 = 21 million + 18 million = 39 million tokens a day, $585. The bill falls to $1,485 a day, $44,550 a month.
It goes first because it is free: a line in the prompt and a cap on the call, no new model, no eval beyond checking the short drafts still pass. It saves $9,450 a month and cuts latency too: a 150-token draft decodes in half the time of a 300-token one.
Lever 2: cheaper tokens
Route those same short tickets to a small model at a tenth of the price, once its golden-set score on the short-reply slice is within 2 points of the mid tier. Input: 60,000 × 1,500 at $3 = $270, plus 140,000 × 1,500 at $0.30 = $63, so $333 a day instead of $900. Output: 60,000 × 300 at $15 = $270, plus 140,000 × 150 at $1.50 = $31.50, so about $300 instead of $585. The bill is about $635 a day, $19,000 a month. The decision doc carries the cautious $27,500, which keeps the routed tickets' output at the mid-tier price as a margin for the share the router escalates.
It goes second because it is not free: it needs the eval run on the slice, a router, and a re-run on every model update. Its saving is the larger one.
Lever 3: the cached prefix
The 1,100-token system prompt is 73% of every input. Vendors sell a discount for sending the same prefix again within a short window, and the discount is published, not negotiated. Anthropic charges a cache read at 0.1 times the base input price as standard, 0.05 times on its current mid and large tiers and 0.025 times on its top tier (source, read 18 Oct 2026), a cache write at 1.25 times the base input price (source, read 18 Oct 2026), and keeps the entry for 5 minutes (source, read 18 Oct 2026) — refreshed on every use, so at 2.3 requests a second the drafting prefix never expires. OpenAI lists cached input on its mid tier at $0.10 per million cached input tokens against $2.00 uncached (source, read 18 Oct 2026). Google lists cached input on its large tier at $0.20 per million cached input tokens against $2.00 uncached (source, read 18 Oct 2026).
At the assumed $3 per million with a read at a tenth of that, one call's input costs (400 × $3 + 1,100 × $0.30) ÷ 1,000,000 = $0.0012 + $0.00033 = $0.00153 instead of $0.0045. The input line falls from $900 a day to about $306, roughly $18,000 a month saved on the unrouted bill, and still $5,000 a month on the routed one. It is counted last because the rate is the vendor's to change, and the engineers must confirm it applies to the way the prompt is assembled: a prefix that changes by one character per tenant is not a prefix.
Two more lines on the vendor's page
Batch pricing: Anthropic's batch API runs at 50% off both input and output tokens (source, read 18 Oct 2026) for work that can wait hours. The drafting feature cannot wait, but the nightly ticket summaries and the weekly eval run can, and half price is half price.
Prompt-length tiers: Anthropic's small model charges $0.50 per million input tokens and $2.50 per million output tokens on prompts over 100,000 tokens (source, read 18 Oct 2026), five times the short-prompt rate. A feature whose prompt grows with conversation history can cross that line on a long thread without anyone deciding to, which is why the PRD caps the history the prompt carries, in tokens, and the engineers log the count.
In an interview
What is being tested is whether you can get from a request volume to a monthly number in under a minute and then say which lever to pull first. Say the three lines, do the arithmetic per day and then per month, state the gap as a ratio, and order the levers by cost to pull: output cap, then routing, then caching. Say what you would read on the vendor's page before counting the third lever. The token bill is the drill.
The mistake that costs money is inventing a cached-token price rather than reading it: the arithmetic looks complete and the bill arrives different. The second is quoting the monthly figure without the per-request one, which leaves the PRD with no ceiling. The third is pulling the cheapest-model lever first, before the eval, which trades a token saving for agent time at a worse rate; model selection has that arithmetic.
Check yourself
1. A new tenant adds 3,000 agents at 40 tickets a day each. What does the drafting feature cost for them a month, as specified and after levers 1 and 2?
3,000 × 40 = 120,000 drafts a day, 60% of the current volume, so scale the bills: $54,000 × 0.6 = $32,400 as specified; about $27,500 × 0.6 = $16,500 after the two levers. Per seat, $11 as specified and $5.50 after — the second still above $5 of revenue, which is the conversation to have with pricing before the contract is signed.
2. The engineers can cut the system prompt from 1,100 tokens to 700 by moving policy text into retrieval. With caching already on, is that worth a sprint?
With a cache read at a tenth of the input price, 400 cached tokens cost 400 × $0.30 ÷ 1,000,000 = $0.00012 a call, $24 a day, about $720 a month. Not a sprint. Without caching, the same 400 tokens at full price are $240 a day, $7,200 a month, and it is. Caching changes which optimisations are worth doing; check whether it is on before sizing the next one.
3. The vendor's cached-read rate turns out to be a quarter of the input price rather than a tenth. Which figures in the decision doc change?
Only lever 3. One call's input becomes (400 × $3 + 1,100 × $0.75) ÷ 1,000,000 = $0.0012 + $0.000825 = $0.002 instead of $0.00153, so the cached input line is about $405 a day instead of $306 — still a saving on $900, just a smaller one. Levers 1 and 2 are unchanged, and the bill was already inside budget without lever 3, which is why it was counted last.