How to use this course
Run the five-step decision loop every exercise is marked against, score your own answer on a rubric, and know why nothing here asks for a definition.
On this page
Thirty-seven lessons and twenty-two exercises, and not one of the exercises asks what something is. Each one hands you a situation with numbers in it and asks for a decision or an estimate, then shows a strong answer, a rubric and the weak answers an interviewer hears most. The loop that produces a strong answer is the same five steps every time, and it is the one thing to carry out of this module.
The decision loop
Restate the question in one sentence. Pull out the two or three numbers that matter. Lay out two or three options and reject at least one. Decide, in a sentence that could be pasted into a message. Then say what would change your mind — the figure that, if it came in differently, would flip the decision.
Worked, on a question from later in the course. A drafting feature writes replies for one tenant's 1,200 support agents, and the FDE asks whether drafts can auto-send for the simplest tickets.
Restate. Should a reply the model wrote ever reach a customer without an agent reading it, and for which tickets?
Numbers. 1,200 agents × 40 tickets a day = 48,000 tickets, and at this tenant every ticket gets a draft: 48,000 drafts a day. On the 200-example golden set, 92% of drafts are acceptable, so 8% are not: 0.08 × 48,000 = 3,840 bad replies a day if every draft went out. One of the 16 failures is wrong on a fact — a price, a policy, a date — exactly the most the launch gate in the evals lesson allows, since it blocks any build above one in 200: 0.005 × 48,000 = 240 factually wrong customer-facing messages a day. One in 200 is not a measured 0.5%, though: one failure in n examples is consistent, at 95% confidence, with a true rate up to about 4.7 ÷ n, the one-failure form of the rule of three, so up to about 2.4%. That is why the slice still needs 600 clean checked drafts of its own.
Options. Auto-send everything; auto-send the simplest 30%; never auto-send, and insert the draft into the composer with one action.
Decide. Never, at this error rate, and not for the simplest 30% either, because the golden set does not yet score that slice on its own — a 30% slice of an 8% error rate is an unmeasured number, not a smaller one.
What would change your mind. Zero factual errors in at least 600 checked drafts on the slice itself, which bounds its rate below 3 ÷ 600 = 0.5%, plus a measured undo rate. Even 0.5% of the slice's 0.3 × 48,000 = 14,400 drafts a day is 72 wrong replies, which is why the undo rate is part of the threshold and not an afterthought.
That answer is under 300 words. Every exercise in the course can be answered in that shape, and the full version of this one adds the screen states and the guardrail. The shape is not special to product work: a design interview runs the same loop with QPS in place of error rates, and the estimation step is the back-of-the-envelope arithmetic this course borrows wholesale.
How a rubric scores an answer
Every exercise carries a rubric of four to six bullets, each an observable thing the answer does. The rubric for the auto-send question reads: turns the error rate into a daily count; makes the auto-send call and names the threshold that changes it, including that the slice must be measured on its own; lists every state the screen needs, including refused, slow and over budget, each with what the agent can do in it; uses streaming against perceived latency, with the first-token time placed against the 1 s limit; names one metric and one guardrail, with the guardrail's number; drops the disclaimer and says why.
Notice what is not on it. "Understands error rates" is not a bullet, because a reviewer cannot see understanding; they can see 3,840. The bullets are written so that two reviewers agree, which is what makes them usable on yourself.
Score each bullet 0 or 1, no halves. The answer above gets 2 of 6 — the daily count and the threshold — and none of the states, the streaming, the guardrail or the disclaimer, because it stopped at the decision. A 2 of 6 is fine for a first pass; the number tells you what to add, which is the value of a rubric over a feeling.
The order matters. Write your answer before you open the strong one, in prose, with the arithmetic shown. Then read the strong answer, then score yourself. Reading the strong answer first feels like learning and is not: you recognise the reasoning next month and still cannot produce it. The DSA course makes the same rule about writing code before reading a solution.
Why no question asks for a definition
A definition is recall, and recall is the cheapest thing to fake. Anyone who has read the RICE write-up can say that reach times impact times confidence over effort gives a score. The question an interviewer asks, and the one this course asks, is which of three candidates ships first when the scores come out 2,133, 6,000 and 2,000 and sales wants the one that scored lowest. That takes the arithmetic, the confidence column read honestly, and a sentence about what evidence would move you — the exercise is in module 2.
So the question stems that open with "define", "list", "what is" or "describe" do not appear. Each question is one decision or one estimate, sometimes with a second sentence that adds the constraint that makes it hard. The frameworks are taught, with worked examples, in the lessons; the exercises ask what you do with them.
The weak answers at the end of each exercise are rarely wrong on facts. "C, because AI is the strategy" is a real answer, and it loses because it was chosen by narrative — it never multiplied anything and never said what would change it.
The numbers, and the units they come in
Every performance and cost claim in the course is grounded in named constants so that no two lessons contradict each other: 1.3 tokens per English word, 86,400 seconds in a day, a one-million-request day at about 12 requests a second. The model prices are assumptions, labelled as such wherever they appear — the worked token bill in module 4 assumes $3 per million input tokens and $15 per million output, and says so each time. Wherever a live vendor figure appears, it carries its source page and the date it was read.
Prices are in USD. To convert, multiply by whatever rate you pay today, and use one rate for a whole answer: a comparison survives a wrong rate but not two different ones.
The order of the modules
Discovery, prioritisation and metrics come first because they did not change when models arrived and they are where most product decisions are won. The tech module sits before design and writing because the AI-feature PRD needs tokens, latency and evals already in hand. The interview module is last because every lesson's closing section feeds it. An engineer moving into product skips the first lesson of the tech module and nothing else.
The next lesson says what changed and what did not. The two after it draw the line between this job and a forward-deployed engineer's.
In an interview
An interviewer running a product question is scoring the loop, not the answer. They are listening for the restatement, because candidates who skip it answer a question that was not asked; for numbers, because a decision without a number is a preference; and for the last step, because "what would change your mind" is what separates judgement from confidence. The product sense lesson shows how each of the four interview rounds maps onto the five steps.
Say the steps out loud as you take them: "Let me restate that", "the number that matters here is", "I'd reject the first option because", "I'd go with", "and I'd reverse that if". The phrases make the structure visible to someone with 45 minutes and a scoresheet.
The mistake that loses points: answering with the name of a framework instead of a number. "I'd use RICE here" scores nothing. "RICE puts B at 6,000 and C at 2,000, and C's score is held down by a confidence we can test in two days" scores on three bullets at once.
Check yourself
1. You have 25 minutes for an exercise and your answer is at the decide step with nothing written for what would change your mind. Do you spend the last five minutes polishing the decision or writing the last step?
The last step. A decision with a reversal condition is worth more on every rubric in this course than a better-worded decision without one. Write one sentence naming a figure and a direction: "if 600 checked drafts on that slice show no factual error, I would revisit auto-send for it."
2. The golden set in the worked example grows from 200 examples to 400 and the pass rate stays at 92%. Does the daily count of bad replies change?
No. 8% of 48,000 is 3,840 whatever the set size; the set size changes how much you trust the 8%, not what it implies. What a larger set buys is a start on stratification, not the answer: 30% of 400 is 120 slice cases, and even with zero failures that bounds the factual rate only below 3 ÷ 120 = 2.5%, five times the 0.5% line. Scoring the slice against 0.5% takes 600 clean examples, from shadow mode rather than the golden set.
3. You score your own answer 5 of 5 on a rubric and then read the strong answer and find it reached the opposite decision. Which do you trust?
Neither, yet. Compare the numbers each used. If both started from the same figures and diverged at the options step, the decision is a bet, which the course says out loud in a few places. If the strong answer used a number you did not, you missed it, and your 5 of 5 was marked by someone who had not seen it.