A week in the role
Say what a forward-deployed engineer's week contains, day by day, in week 3 of an 8-week pilot — and why the outcome, not the ticket, is what you own.
On this page
Week 3 of an 8-week claims-triage pilot at a regional insurer: 1,200 first-notice-of-loss claims a working day, 28 handlers, a median of 11 minutes per claim, and a scoping doc signed in week 0, before the first line of code, that promises a median at or under 7.7 minutes by week 8. The five days below, two on site, one building, one on the eval and half a day writing, are a composite built on the course's running example, the claims-triage pilot that carries most of the exercises; every figure in them is an assumption you can replace with your own. The sentence that defines the job comes at the end, because the week is the argument for it.
Monday: the claims floor
08:30, beside two handlers. 60% of the day's claims arrive between 09:00 and 12:00: 720 in three hours, 240 an hour, against 28 handlers at 11 minutes each clearing 28 × 60 ÷ 11 ≈ 150 an hour. A queue builds at about 90 an hour until noon and drains through the afternoon. Nobody on the floor had said "queue"; they said "mornings are bad".
Six claims watched end to end with a stopwatch, each step counted: 7 minutes finding the governing wording in the 340-page policy set, 4 minutes deciding the route, one of the six sent back by a supervisor. The questions are the ones discovery interviewing on site prescribes — what happened on the last claim you sent back, how often, show me — and by lunch the kickoff's 7/4 split is confirmed on the floor, step by step, the point of mapping the workflow before the model.
14:00, thirty minutes with the champion, the Head of Claims Operations, who needs one number for her Thursday ops review. 16:00, the head of underwriting, described to you as a blocker: the model must never route a claim above the high-value band on its own, whatever it scores. He is right, and what he has given you is a rule, not a prompt — code on Wednesday, and a decision of his in Friday's update.
Tuesday: the data team and the questionnaire
Week 2's prototype ran on a one-off CSV of 200 claims. Tuesday morning turns that into a read-only view the pilot polls every 60 seconds, the freshness same-day triage needs, because getting the data out properly is the pilot's first real deliverable and the shadow run that starts in week 4 reads from it. The export has 40 columns, three date formats, and a policy number reused across policy years, so a join on it alone doubles some claims. The question that takes longest is which columns are personal data and so never leave the customer's tenant.
The security questionnaire lands at lunch: 60 questions, about 20 of them the engineer's to answer on the spot and the rest for your own security team, with the sentence that says so; the questionnaire sorts the 60 into those two piles. At 17:00 the view's first full day lands: 1,183 rows for Monday.
Wednesday: building
A day alone with the door shut. The morning goes on the ETL that replaces the prototype's throwaway plumbing: the 60-second poll of the view, one date format, the policy term resolved from the loss date so each claim is one row, personal-data columns dropped before anything leaves the tenant. The afternoon goes on the prompt, written against the rubric the handlers will mark with.
The sizing is a few lines. Per claim: the 3-page form and adjuster note at about 1,600 words, the retrieved policy section at 900, the instruction prompt at 700 — 3,200 words, about 4,300 tokens at an assumed 0.75 words per token. Output is a 120-word triage record, about 160 tokens. Per day: 1,200 × 4,300 ≈ 5.2 million input tokens and 1,200 × 160 ≈ 0.19 million output. At r per million input tokens and 4r output, that is 5.2r + 0.19 × 4r ≈ 6r a day, and the arithmetic stops there; the customer multiplies by their contract. Sizing the deployment adds the peak. The underwriting rule from Monday is a line of code before the model is called, not a sentence in the prompt, because a prompt is a request and the rule is a guarantee.
Thursday: the eval
By midweek two senior handlers have labelled 100 of the 200 claims, escalate or approve, and agreed on 87: 87% agreement, and the practical ceiling for anything measured on these cases until the 13 disagreements are settled. The first 90 minutes of Thursday is a room with the 13 cases on screen: both handlers; the champion, who rules on each case; the claims QA lead, who owns the set and runs the session; and you, writing down the rule each ruling reveals — the meeting building the eval set describes.
Then the run. The pilot's call is read off its triage record: escalate when the route is fraud_review or needs_handler, approve otherwise. On the adjudicated 100 it agrees with gold on 88, against the 95% line, and agreement is the smaller problem. The head of underwriting signed two lines for this decision: at most 2 missed escalations per 100 gold escalations, and precision on escalate of at least 0.80. Of the 12 disagreements, 4 are missed escalations and 8 are false ones; the 100 hold 20 gold escalations, so recall is 16 ÷ 20 = 0.80 and precision 16 ÷ 24 = 0.67. This is an early run, and it fails both of his lines. The underwriting rule, now in code, adds the third way a claim escalates, any claim above the high-value band, and catches 3 of the 4 misses on a re-run: 91 of 100, 91%. Recall is 19 ÷ 20 = 0.95, still one miss over a line that allows none on 20 escalations, and precision is 19 ÷ 27 = 0.70: the 8 false escalations are a threshold problem the rule cannot fix. Under all three lines, then, and that is Thursday's finding: the shadow run starts next week, and its per-segment gate is three weeks away, at the end of week 6. Why the line is agreement on their cases rather than a public score is an eval is not a benchmark; the scorer behind his two lines is score the eval set.
Friday morning: writing
Four hours on two documents, about two each. The weekly update is one page in the narrative form: the number (91% on the 100 adjudicated so far, with both escalation lines still failing), the delta and its cause (up 3 points on Thursday's re-run; 3 of 4 missed escalations closed by the high-value rule), the decision needed (the head of underwriting to approve the rule's wording as permanent behaviour), from whom, by when (Tuesday). The champion forwards it to the sponsor unedited, which is the test it has to pass. The field note goes the other way, to your own product team: the policy number reused across policy years is a property of the policy system, so every customer on it needs the same term resolution. The weekly update and the field note gives both forms. The afternoon is the internal sync and the train home.
The sentence that defines the job
You own the outcome, not the ticket. An engineer can ship the ETL and the prompt by Wednesday, close every ticket, report "pipeline live, prompt v3 deployed" — and have moved the pilot's number by nothing anyone can see. At week 8 the renewal case is an anecdote.
The week above has the same tickets, and each day exists because of the number: Monday checked the minutes on the floor and found the rule, Tuesday got the data the number is computed on, Wednesday built the thing being measured, Thursday measured it, Friday asked the person who can decide for the decision. The hours: on site about 16, building 8, the eval 8, writing 4, meetings and travel 6 — around 42, of which the model-facing work was 3 or 4. Where the role sits against the five jobs that share pieces of this week is one table in FDE and its five neighbours; the row that separates it is "Success": the customer is live, using it, and renews.
In an interview
The customer-scenario round and the behavioural round both ask for a week like this. "Walk me through a deployment you ran" wants days with numbers in them, and the interviewer listens for whether your week has an eval day at all, whether you can say what number moved and why, and whether a named person at the customer decided something because of what you wrote. In the role, the customer tests the same thing every Friday: a number and a decision needed, or a list of things done.
The mistake that loses points is describing the week as tickets — "built the pipeline, tuned the prompt, fixed some bugs" — with no number that moved and no person who decided. The second-worst answer has the number but no decision, which says you measured and then waited.
Check yourself
1. The champion asks you to spend Thursday building the chat surface the VP wanted, instead of adjudicating the 13 disagreements. Which do you do, and what does each cost in week-6 terms?
Adjudicate. The gold set gates the prompt at 95%, and 13 unsettled cases cap measurable agreement on it at 87%: no prompt change can show 95% until the gold is fixed. The shadow run, which starts next week, gates each segment at the end of week 6, and it should not start on a prompt the gold cannot score, so the 90 minutes with the handlers is the only work this week that can make week 6 passable. The chat surface sits at week 6 in the scoping doc: "not yet", with that date. What reverses it: the sponsor moves the gate.
2. Monday's rows land through the view: 1,183 against an expected 1,200. Stop and investigate, or build on it?
Build on it and open a question, not a stop. 17 short is 1.4%, inside a day's normal variation. What would stop you is a systematic gap — a region, a status or a value band missing — because the shadow run reads from this feed and is gated per segment, and a missing slice makes that segment's reading unrepresentative. Ask the data team for Monday's count at source; if it is 1,183, carry on.
3. From week 4 the sponsor asks you to work remotely to save the travel. Estimate what the week-4 update loses.
Monday's floor time and Tuesday's data-team room, the two inputs Thursday's number was computed from. This week they produced the high-value rule, worth 3 of 4 missed escalations and 3 points (3 of 100); remote, week 4 reports the same 91% with no new rule found. Agree one day on site after the week-6 gate, when the number is being watched rather than moved.