Product sense
Answer "design X for Y" and "should this product add an assistant" in one shape: one user counted, an option rejected, the failure mode priced.
On this page
A product-sense round hands you a product and a group of people and asks what to build: "design a tool for new support agents", or, in its AI version, "should our support tool add an assistant". The interviewer scores structure, users and trade-offs. The AI version puts two items inside the trade-offs that the classic never needed: what happens when the model is wrong, and which number would say the product works. Without those two the answer is a feature pitch, and the scoresheet has nowhere to put a pitch.
Four rounds, one loop
The PM loop most large companies run has four rounds of about 45 minutes, each with a different interviewer: product sense, execution, estimation and behavioural. Each interviewer writes a scoresheet before the debrief, and the debrief reads the sheets, not anyone's memory of the conversation. Whatever you reasoned silently is not on the sheet.
Every round is the decision loop from how to use this course — restate, numbers, options, decide, what would change your mind — with the weight on a different step:
| Round | Restate | Numbers | Options | Decide | What would change your mind |
|---|---|---|---|---|---|
| Product sense | which user, which job | how many, how often wrong | three solutions | one, with its cost | the metric and its threshold |
| Execution | which number moved | the inputs that multiply into it | the cuts | roll back, fix or accept | the cut that would reverse it |
| Estimation | the unit of the answer | two or three assumptions | a second route | the rounded figure | the assumption it is most sensitive to |
| Behavioural | the question behind the question | before, after, cost | what you rejected | what you did | what you changed afterwards |
Product sense carries its weight in the narrowing and the rejection. Execution, estimation and behavioural take the other three rows.
The classic: design X for Y
"Design a product for support agents in their first month." The weak answer starts building in the first minute: a training mode, a searchable knowledge base, milestone badges, an AI coach. The interviewer can score none of the three dimensions: nothing was narrowed, "new agents" was repeated rather than segmented, and nothing was rejected.
The strong answer spends five minutes narrowing. Which new agent: one at a 14-agent shop with no trainer, or one of the 200 an outsourcer hires each quarter? The outsourcer's, because 200 hires a quarter is where the cost is. The job, stated the way jobs to be done states one: reach the floor's 40 tickets a day without escalating more than the team does. The number: assume a hire clears 20 tickets a day in week one and 40 by week six, a ramp averaging 30. Each hire is 10 tickets a day short for 30 working days, and 200 hires is 60,000 tickets of lost capacity a quarter, about 1,500 agent-days.
Three options: a shadowing rota, a side panel showing the policy article for the ticket's intent, and drafted replies. Reject drafting out loud: the agent least able to catch a wrong draft would lean on it most, and in week one they cannot tell a 14-day refund window from a 30-day one. Choose the panel and say what it costs: retrieval with no generation, so nothing to check, and the hire still writes the reply, which is how they reach 40. The metric is a cohort column — tickets a day in week three, by hiring cohort, read as cohorts and funnels reads retention — with escalations per hire as the guardrail.
McDowell and Bavaro's Cracking the PM Interview works the three classic forms — design a product, improve one, talk through a favourite — and identifies the users before it proposes any feature, which is the order the sheet rewards.
The AI version: should this product add an assistant
The question sounds like yes or no. It hides three questions: for whom, doing what, and how often wrong. Answering the yes or no first — "yes, users expect it", "no, too risky" — skips all three, and both answers score the same.
Worked: should the admin console add an assistant that builds automation rules from a sentence? The user is the tenant's support admin. Assume the 40 tenants have 300 admins between them, one per 40 agents, each writing 5 rules a month: 1,500 rules at 20 minutes each in today's builder, 500 admin-hours. Drafting a rule in 5 minutes returns 375 of them. The token bill is not the question: at ten times the drafting call's $0.009 (assume $3 per million input tokens and $15 per million output), 1,500 calls cost about $135 a month.
The failure mode is where this answer is scored. A wrong drafted reply is one message, and the agent usually catches it. A wrong rule acts on every ticket it matches until somebody notices. Assume a tenant with 10,000 tickets a day and a rule that wrongly matches 2%: 200 tickets a day closed or misrouted, and nobody reads them. The error rate is the same kind of number as drafting's 8%; the blast radius is not, and the blast radius sets the design.
So the assistant drafts the rule and runs it against last week's tickets before it can be saved — "this rule would have matched 214 tickets; here are ten" — and the admin activates it. It will not activate a rule, will not change a live rule without showing the diff, and will not touch SLA or routing rules, which carry contract terms. What it will not do is part of the answer, not a disclaimer after it.
The metric: assistant-built rules still active and unedited after 30 days, with admin minutes per rule beside it. Not "assistant conversations", which rise when the assistant confuses; the North Star lesson has the test — can it rise while the outcome falls? The guardrail: reopen rate on tickets touched by assistant-built rules, against hand-built ones.
Decide: yes, as a draft plus a dry run, activation always manual. What would change it: admins editing more than one rule in five after the dry run, which would mean the assistant spends review time rather than saving build time.
How the round is scored
Structure: narrowed before solving, steps said out loud. Users: one segment, with a reason and a count. Trade-offs: an option rejected with its cost; in the AI version, the failure mode priced by its blast radius, what the product will not do, and a metric that falls when the product gets worse. Earlier lessons supply each sentence: the error count from evals as the feature's test suite, the threshold from the escalation arithmetic in jobs to be done, the metric's shape from the product form.
In an interview
What is being tested is whether you narrow before you build. Open with the narrowing sentence — "I will take the outsourcer's new hires, because 200 a quarter is where the cost is" — and close with the reversal: "and I would stop if admins edit more than one rule in five after the dry run."
The mistake that loses points: answering "should we add an assistant" with yes or no before saying for whom it works and how often it would be wrong. The second is a metric that counts use instead of outcome — sessions, messages, "AI adoption" — which reads as a team that will celebrate a confusing assistant.
Check yourself
1. The dry run adds 3 minutes to every rule the assistant drafts. Does that kill the case?
No. 1,500 rules × 3 minutes is 75 hours a month, so the saving falls from 375 hours to 300. The dry run is what stops a 2% mismatch becoming 200 unread tickets a day; keep it, and quote the saving net of it.
2. The same assistant is offered to all 12,000 agents for personal reply macros. Do you keep the dry run?
No. A personal macro's blast radius is one agent's replies, and the agent reads the text at every insert, so the review already happens at the point of use. A preview for macros, a dry run for tenant-wide rules. What the macro case needs is the error count per insert, because 12,000 agents make it a volume problem.