The PM interview · The four questions

Estimation, AI version

Estimate the drafted reply's monthly cost for a new 3,000-agent tenant from three stated assumptions, check it against the fleet bill, and price the contract.

estimatemediumExercise · 3 min · 58 of 59

On this page

Sales has a signature coming and the interviewer wants a number before it lands: what the drafted reply costs a month for one new tenant. The round scores the assumptions, the rounding and the check, which the estimation lesson works through.

The situation

"A new tenant signs for 3,000 agents next month. What will the drafted reply cost us a month for them?" On the whiteboard, from the token bill: the feature serves 200,000 model calls a day across 12,000 seats and costs $54,000 a month over 30 days. Each call is 1,500 input tokens and 300 output tokens. Assume $3 per million input tokens and $15 per million output tokens. The add-on sells at $5 a seat a month. The two levers already planned — capping short replies and routing them to a small model — bring the fleet's bill from $54,000 to about $27,500. When you ask, the interviewer says the tenant's agents each handle about 40 tickets a day, like the largest existing tenant, and that agents across the fleet regenerate about one draft in ten. Say three assumptions before any arithmetic.

The question

Estimate the tenant's monthly cost as specified and after the two levers, check it against the fleet's bill, and say what it means for the contract.

A strong answer · Rubric · Where answers go wrong

Write your answer first, then open

A strong answer

Three assumptions first. One: 40 tickets per agent per day, the interviewer's figure. Two: 1.1 calls per ticket, because one draft in ten is regenerated. Three: today's prompt shape, so a call costs 1,500 × $3 ÷ 1,000,000 + 300 × $15 ÷ 1,000,000 = $0.0045 + $0.0045 = $0.009.

Arithmetic: 3,000 × 40 = 120,000 tickets a day; × 1.1 = 132,000 calls; × $0.009 = $1,188 a day; × 30 = $35,640. Call it $36,000 a month as specified. The levers cut the fleet's bill to about 51% of itself, so about $18,000 after them.

Check against the known bill: 132,000 calls is 66% of the fleet's 200,000, and 0.66 × $54,000 = $35,640. The same figure, which confirms the per-call arithmetic and nothing else. The volume needs its own check, per seat: the fleet runs 200,000 ÷ 12,000, about 17 calls a seat a day; this tenant runs 44, about 2.6 times that. Plausible, since the largest tenant already works at 40 tickets an agent, but the fleet's $4.50 a seat does not apply here.

What it means: $36,000 ÷ 3,000 is $12 a seat against $5 of revenue, $15,000 a month. The tenant loses $21,000 a month as specified and about $3,000 after the levers. The contract needs a usage allowance per seat with a per-call price above it before signature, and the levers must ship before the tenant goes live.

The figure is most sensitive to tickets per agent: at the fleet's 17 calls a seat it is about $13,500, under the $15,000 of revenue before either lever, and the tenant looks profitable. That gap is why the question was worth asking.

Rubric

States three assumptions, including calls per ticket, before any arithmetic
Converts to a cost per call, shows the day and then the month, and rounds the result
Checks against the fleet's bill and says the check confirms the arithmetic, not the volume
Checks volume per seat and finds the tenant at about 2.6 times the fleet average
Compares cost per seat with the $5 price, as specified and after the levers
Names the assumption the figure is most sensitive to, and the contract change that follows

Where answers go wrong

"The fleet's 12,000 seats cost $54,000, so 3,000 seats cost a quarter: $13,500." Scaling by seats assumes fleet-average usage, and this tenant uses 2.6 times it; the estimate is low by about $22,000 a month.

"About $36,000 a month." Correct and unfinished: the question asked what it means for the contract, and the answer never compares $12 a seat with $5.

"Five minutes rebuilding tokens from words per ticket." The token shape is on the whiteboard; those minutes belong to the per-seat check, which is where the finding is.