Workflows, state and the harness · State and memory

State for a three-day recruiting screen

Design what each applicant's thread persists, where and for how long, so a rubric change mid-role and a restart mid-send cost nothing.

designhardExercise · 4 min · 24 of 75

On this page

A screen that runs for three days per applicant is only as good as what it writes down between runs. The last hire lost two things the state should have held; memory and state for long-running automations is the lesson this practises. The graph sends declines from code, as the lessons draw it to compare skins; the version to ship has a named person decide every decline, and the recruiting screen designs it.

The situation

The agency is hiring a developer and expects about 150 applicants in the first week; each must hear within three days of applying. The graph is the screen from the lessons. Code checks three rules; the model scores the CV on a five-criterion rubric, about 3,700 tokens in and 250 out a call; code routes 18 or more to an invite, 12 to 17 to the founder and under 12 to a decline. The founder clears her queue at 09:00 and 16:00. An invite carries a booking link, a reminder at 24 hours and a close at 48.

On the last hire, two things went wrong. On day two the founder added a sixth criterion, agency experience, after 60 applicants had been scored, and nobody could say who was scored under which rubric. A worker restart during an invite batch sent 11 candidates two invites. The privacy notice promises deletion six months after the role closes. Assume the large model at $2 per million input tokens and $8 per million output, and a 200k-token window.

The question

Design each applicant's state: the fields its thread persists, which are durable and which are recall, where each lives and for how long, and where the run stops. Then show how your design absorbs a rubric change mid-role and a restart mid-send.

A strong answer · Rubric · Where answers go wrong

Write your answer first, then open

A strong answer

One thread per applicant, keyed by role and application, checkpointed after every step. Durable fields: the rule results and any failing rule; rubric_version; a score and a quoted CV line per criterion; total; route; each person's verdict, who and when; the invite key, sent time, deadline and reminder flag; the booked slot. The CV is a file id, never a copy; the model's reply is working state, dropped once parsed. About 2 KB a checkpoint over eight checkpoints is 16 KB an applicant, 2.4 MB a role.

Recall sits in a store keyed by a hash of the email: earlier applications and outcomes, read by code before scoring, and the founder's overrides of the model's total, kept per rubric version.

The run stops twice, at an interrupt for the founder and waiting on the booking webhook. Both timers derive from deadline in state, so a restart re-arms them.

The rubric change becomes a query. The version covers the criteria and the thresholds, since 18 of 25 is not 18 of 30. Code lists version 1 threads not yet decided and re-scores them, at most 60 × (3,700 × $2 + 250 × $8) ÷ 1M ≈ $0.56. Decided threads keep version 1, so two totals can be told apart.

The restart is absorbed by a key: the send carries the thread id plus "invite", and the mail step refuses a key it has sent. Eleven duplicates become none.

Threads and CV files are keyed by role and recall by email hash, so deletion at close plus six months is three deletes, one per store. If repeat applicants turn out to change email, recall needs a second key.

Rubric

Keys one thread per applicant and lists the durable fields one by one, with the CV held by reference and the model's reply not persisted.
Separates recall from durable state, keys recall on something exact, and has code rather than the model read it.
Names both stopping points and keeps their timers in state, so a restart re-arms them.
Stores a rubric version, covering criteria and thresholds, on every scored thread, and prices the re-score of the threads it affects.
Makes the invite idempotent with a key derived from the thread, and says where the key is checked.
States the retention date and shows deletion reaching every copy.

Where answers go wrong

"One conversation per role, so the model remembers every applicant." 150 applicants at about 3,700 tokens each is 555,000 tokens, more than twice the 200k window, before the model writes a word. The reviewer scores whether memory is a store or a window, and this answer chose the window.

"Copy the CV and the model's full reply into every checkpoint, to be safe." Storage is cheap; deletion is not. 150 applicants × 8 checkpoints is 1,200 copies of personal data to find on the retention date, and no later step reads any of them.

"Add the new criterion to the prompt and carry on." Scores out of 25 and out of 30 then sit in one ranking with nothing marking which is which, and the threshold of 18 now means two different things. The founder compares totals that are not comparable.