Discovery interviewing on site
Run a 45-minute interview on the customer's floor that trades opinions for counts and ends in three statements a reviewer could mark true or false.
On this page
A discovery interview on site has one deliverable: three statements about the customer's work that a reviewer who was not in the room could mark true or false, each with a number and a source. Everything else that happens in the 45 minutes is in service of those three lines. The failure mode is not asking too little; it is asking questions that can only be answered with an opinion, and leaving with five of them.
Opinions are free, so they are worth nothing
The kickoff version of discovery goes like this. The engineer describes the idea — a tool that finds the governing policy wording for a claim — and asks the handler whether it would help. The handler says yes. Four more handlers say yes. The engineer writes "strong demand from the floor" in the trip report, and eight weeks later the tool is live and the handlers keep the PDF open in a second window, because the tool finds the wrong clause for commercial policies and nobody measured how often that happens.
Nothing in that interview could have come out any other way. Agreeing with a visitor costs a handler nothing, disagreeing costs a small awkwardness, and the question gave them no way to be specific. Rob Fitzpatrick's The Mom Test puts it as three rules: talk about their life, not your idea; ask about specifics in the past, not generics or opinions about the future; talk less and listen more. The rules are easy to recite and hard to follow with a sponsor in the room who wants to hear the pitch.
The test for any question you are about to ask is whether the answer could be wrong. "Would you use this?" cannot be wrong. "How many claims did you close yesterday?" can be, and the log will say so.
Ask what happened last Tuesday
Specific means a day, not a type of day. "Walk me through a typical claim" gets a tidy story with the mess edited out. "Show me the last claim you closed" gets a screen, three windows, and the pause while the handler scrolls a 340-page PDF for the right section. Sit beside the handler and watch; the watching is the interview.
What you write down while watching is time and place. On a claims floor the record for one claim looks like: opened 10:42, policy PDF opened 10:43, right section found 10:50, route set 10:53. Eleven minutes, seven of them in the PDF. Six claims in an afternoon give you a shape — the search step ran 5 to 12 minutes, the decision 3 to 5 — that no amount of asking would have produced, because the handler experiences it as "a few minutes" and would have told you so.
Last Tuesday also works on the people who do not sit at a screen. The team lead will not know the median triage time, but will know that last Tuesday 1,340 claims came in and 26 handlers were on the floor because two were out. The QA reviewer will not know the error rate, but will remember the three claims sent back last week and why. Each of those is a count with a date on it.
Count it
Every adjective the customer uses is a number you have not asked for yet. "It takes forever" is minutes; "all the time" is times per day; "everyone" is a headcount; "most of the work" is a fraction of the minutes. The discipline is to convert on the spot — "forever: ten minutes, thirty?" — and write down the answer with who said it.
The counts then get checked against each other, which is where an interview earns its time. The team lead gives you 1,200 claims a day and 28 handlers. The handler's screen gives you 11 minutes a claim. 28 handlers at roughly 450 productive minutes each is about 12,600 handler-minutes a day; 1,200 claims at 11 minutes is 13,200. The floor is slightly over capacity before anyone opens a PDF, which means claims queue overnight, which is the thing the VP of Claims actually minds. You have not been told that; you derived it from two people's numbers, and it is the first question for the next interview: how many claims are still open at 18:00, and who clears them. Mark the derived numbers as derived — the VP will want to know which figures are theirs and which are yours.
Counts you cannot get on the floor go on a list for the system log, which is the subject of mapping the workflow. An interview that ends with a log request has a next step; one that ends with a thank-you does not.
Never pitch
At some point the handler, the team lead or the sponsor asks what you are going to build. The honest answer in week 1 is that you do not know yet, and the useful one is "show me one more claim first". The pitch has a time and a document — it is the goal line of the scoping doc — and giving it early does two kinds of damage. It ends the facts: once the handler knows you are building a search tool, every later answer is about search tools. And it commits you to a shape before the counts have said where the minutes are.
Fitzpatrick's other rule applies here: a meeting that ends in a compliment has failed, and one that ends in a commitment has succeeded. Commitment means the customer gives something that costs them — a week of logs by Friday, thirty minutes with the QA reviewer, a copy of the policy set as text. Ask for it before you leave the room, with a date.
Leave with three statements
The output has a format. A statement names a fact, a number, who it came from, and how it will be confirmed. The three from a first day on a claims floor:
- Handlers spend a median 7 of 11 minutes per claim finding the governing wording in the policy set — six observed claims on Tuesday; to confirm from the document system's timestamps for a full week.
- 1,200 claims a day arrive against roughly 12,600 handler-minutes of capacity, so the queue carries into the next day — team lead's counts; to confirm from the open-at-18:00 figure in the claims system.
- The QA reviewer returns about 3 claims a week for a wrong route, mostly commercial policies — reviewer's recollection; to confirm from the return log.
Each one could be false. The log might say 4 minutes, not 7, and then there is no pilot on retrieval. That is the point: the statements are the hypotheses the scoping doc will turn into a goal, and a hypothesis a reviewer cannot mark false is a slogan.
Who to interview, in what order
A day on site holds five or six 45-minute conversations. Take them from the floor upward: two handlers at their screens, the team lead, the QA reviewer, then the person who owns the policy documents, because the data request will go to them. The sponsor is last. Sponsors talk in outcomes — 30% faster — and you want the floor's facts before you hear the outcome, so that you can say what the outcome would cost in minutes rather than nod at it.
Keep the sponsor's number, though. It is the goal; discovery's job is to find the shortest path to it, which is what the kickoff exercise turns on.
In an interview
The case round tests whether you separate what the customer observed from what they believe, and whether you can turn a conversation into something measurable before proposing a build. Say how you would spend the first day — handlers first, sponsor last — and give the three statements you would want to leave with, each with its number and its confirmation. Interviewers listen for the phrase "to confirm from the log"; it signals that you treat interview data as a hypothesis.
The mistake that loses points is arriving with the solution and running the interview to collect agreement with it. A candidate who opens with "we would build a retrieval assistant and I would validate that with handlers" has described a pitch; the interviewer will ask what the handlers could have said that would have changed the build, and there is no answer.
Check yourself
1. A handler tells you the policy search "takes about five minutes". Your six observed claims put it at 7. Which number goes in the statement, and what do you do about the gap?
The observed number, with the discrepancy noted: "7 minutes observed over six claims; the handler's own estimate is 5." Neither is the answer; the document system's timestamps over a week are, and the gap is the reason to ask for them today. If the week's median comes back at 5, the pilot's addressable minutes drop from 7 to 5 and the 30% goal needs more than retrieval.
2. The sponsor joins your first handler interview and asks you to "explain the vision" in the opening minutes. What do you do with the remaining 40?
Give the vision in two sentences — faster triage, measured on their claims — and then ask the handler for the last claim they closed. The sponsor's presence makes the handler's yes even cheaper, so every question for the rest of the session has to be about a specific past claim, where the sponsor cannot supply the answer. Ask the sponsor for 20 minutes afterwards, alone.
3. The team lead says 1,200 claims a day; the claims system's dashboard says 980. Which goes in the scoping doc?
Neither, yet. The gap is roughly 20%, which is a different pilot: 980 a day at 11 minutes is 10,800 handler-minutes against 12,600 of capacity, and the overnight queue disappears. Ask what the dashboard counts — perhaps it excludes claims re-opened after a return — and resolve it from the raw log before any goal is written. A goal built on the wrong denominator is wrong by 20% on the day it is signed.