Behavioural
Tell a STAR story with a number in every letter, and answer "a time the model got it wrong" with the cause in your system and the change it forced.
On this page
The behavioural round asks for a story from your past: a conflict, a failure, a call made on thin evidence. The AI version is "tell me about a time you shipped something the model got wrong, and what you changed". STAR — situation, task, action, result — is the skeleton every candidate knows. What the interviewer scores is what sits inside it: a number in every letter, an "I" that owned a decision, and a change to the system rather than to one output.
STAR, with a number in every letter
The weak answer has all four letters and nothing in them: "Our drafts had quality problems. My task was to improve them. I worked closely with engineering and research, and quality improved significantly." The interviewer has nothing to write on the sheet. Each letter owes a number:
- Situation: the scale. "48,000 drafts a day for one tenant's 1,200 agents."
- Task: what you owned, in one clause. "I owned the done line and the golden set."
- Action: the decision, with the alternative you rejected. "I added ten cases before changing anything, rather than shipping a prompt fix on the field note alone."
- Result: before and after in the same unit, with the denominator. "88.6% to 91.9% on 210 cases."
McDowell and Bavaro's Cracking the PM Interview treats this round as one you prepare for, with stories written before the questions arrive. Write each story with its numbers in it, because a number recalled under pressure is a number invented under pressure.
The AI version: a time the model got it wrong
The interviewer listens for five beats, in order, because the order shows when you found out and what you did about it:
- What shipped, and its done line.
- How you found out, with the count — and whether you found out or a customer told you.
- Containment, and how long it took.
- The cause, named in your system, and the change that would catch the next one.
- The result in the unit you started with, and what reached customers.
Worked, told by the PM of the week in working with an FDE and with research, in about two minutes:
"I owned drafted replies for a tenant with 1,200 agents, 48,000 drafts a day, launched at 92% on a 200-case golden set. On a Monday the FDE's field note showed 60 agent reports in a week, 35 on refunds: drafts quoted a 14-day window to annual-plan customers, who get 30. That afternoon I turned drafts off for refund tickets, about 6% of volume, so agents wrote those by hand. We traced 140 replies sent with the wrong window in the six days before, and the FDE's team wrote to each customer. I added ten refund cases to the set and it scored 88.6%. Research said the model follows the policy when it has it, so the cause was ours: the exception was not in the context, and no case tested it. We retrieved the policy per ticket and the set reached 91.9%, one case short of the line, so I held the rollout to other tenants for a week rather than bend the line. Since then every policy exception becomes a golden-set case before a tenant goes live."
Count what that answer gives the sheet. Fourteen numbers, a containment step taken the same afternoon, a cause that names a gap in the eval rather than a flaw in the model, and a decision against the candidate's own interest — holding a rollout for 0.1 points. That last sentence does more for the ownership score than any adjective could.
What the interviewer is scoring
- Ownership. "I" for your decisions, "we" for the team's work, each where it belongs. A story told in "we" throughout scores no ownership; one told in "I" throughout scores no team.
- The number. Before and after in one unit, with the denominator, and the count that reached customers.
- The cause. Named in the system you owned. "The model hallucinated" describes a symptom and places the cause outside your control; "the exception was not in the context, and no case tested it" is a cause you could have prevented, which is the admission the interviewer listens for.
- The change. Something that catches the next instance — a case type, a blocking class, a gate — and not only the fix for this one. The evals lesson supplies the vocabulary: a class that blocks at a rate.
- Reflection. What you do differently now, as a behaviour with a start date, not a lesson learned.
The story bank
Six stories cover most loops: a failure, a conflict, a call on incomplete evidence, influence without authority, a launch, and the model-got-it-wrong story. Write each in about 150 words with three numbers — before, after, and what it cost — and map them onto the common prompts; one story usually answers two or three. A story told in two minutes leaves room for the follow-ups, which is where the score is settled: "how did you know it was 140?", "whose call was it?".
Earlier lessons in this course hand you ready-made stories if you have run them for real. The one-day prototype is the incomplete-evidence story, because its decision rule — 30 of 60 drafts inserted — was written before the 38 came in. The RICE lesson's honest confidence column is the conflict story, told as the afternoon you scored sales' favourite at 50%.
In an interview
What is being tested is whether you own outcomes measured in numbers and change systems after failures. Tell the story in about two minutes, a number in every letter, and end on the change that would catch the next one. When a follow-up asks for a number you do not have, say so and say how you would have measured it. An invented number survives the first follow-up and not the second.
The mistake that loses points is a result line with no number in it — "quality improved significantly", "customers were happier" — because the interviewer has nothing to write down. The AI-specific second is blaming the model: "the model hallucinated" puts the cause outside the system you owned, which is the opposite of what the round scores.
Check yourself
1. Your strongest story has a clean result, edit-before-send from 14% to 9%, but the decision in it was your manager's. Do you use it for "a time you made a hard call"?
No. Use it for a launch or failure prompt, where your part was execution. For the hard-call prompt choose a smaller story where the decision was yours; "whose call was it?" is a standard follow-up, and the answer sets the ownership score.
2. The interviewer asks whether 140 wrong replies was a lot. What do you answer?
Size it against its slice. Refund tickets were about 6% of 48,000, so 2,880 drafts a day and about 17,000 in the six days; 140 is under 1% of them. Then say why it still mattered: each was a customer told the wrong refund terms, which is why all 140 got a letter. A rate and a reason, not "it was a small number".
3. You remember the error rate fell "from about 7% to under 1%" but not the sample size. Do you say the numbers?
Yes, with the hedge said out loud and the instrument named: "about 7% to under 1%, on a weekly sample of a few hundred." A rounded true number scores; an exact invented one fails "on how many?"