The recruiting screen
Design a CV screen where a named person decides every rejection, price that review per role, and test the scoring for bias before a role opens.
On this page
Each role the agency hires for draws about 150 applicants, and the founder has promised every one of them an answer within three days. Module 3 drew the screen to compare two skins, and in that drawing code declined 90 of the 150 with nobody deciding. This page is the full design, and it has one fixed constraint: every rejection carries a named person's decision. The arithmetic requires that before the law does, and where the law applies it requires it too.
The role
The six fields of the virtual employee frame, with the numbers from one idea, two skins: of 150 applicants, 30 fail a hard rule, 24 score 18 or more out of 25, 36 score 12 to 17, and 60 score under 12.
| Field | The recruiting screen |
|---|---|
| Inputs | About 30 applications a week, a form and a CV, about one role a month. The CV is read as text. A voluntary equal-opportunities form is stored apart and never reaches the screen. |
| Outputs | Per applicant, a record: rule results, five scores each with a quoted CV line, a route, who decided. Then an invite with a booking link, or a decline saying how to ask for a review. Never a decline without a named decider; never a web search on an applicant. |
| Done | Within three days of applying, each applicant has an invite or a decline, and the record names the person who decided. |
| Manager | The operations manager owns it and runs the weekly review. The founder owns the rubric and decides the borderline band. |
| Escalation | CV unreadable: the operations manager, within 1 working day. A proposed decline unreviewed after 1 working day: the founder, never sent on a timer. A request for review: the founder, within 5 working days. A system down: the owner, at once. |
| Budget | About $0.0094 an applicant in tokens, at most two scoring calls. People: about 6.7 hours a role, 2.3 of them the founder's. |
The SOP
Trigger An application is submitted.
Steps 1. Store the CV by reference; extract its text. code
2. Redact name, contact details, photo, date of birth,
nationality and address; give the applicant an id. code
3. Apply the three rules. code D1
4. Score five criteria 1-5, each with a quoted CV line. model
5. Check that every quote appears in the CV; retry once. code D2
6. Route on the total. code D3
7. The founder decides 12-17, at 09:00 and 16:00. person
8. The operations manager decides every proposed decline,
daily at 14:00. person D4
9. Send the invite or the decline, keyed. code D5
10. Remind at 24 hours; close the invite at 48. code
Decisions D1 rules = right to work, portfolio link opens, salary inside the band;
a failure is a proposed decline with its evidence; a dead
link first gets one request for a working one, 24 hours
D2 quotes = a quote not found in the CV after the retry sends the
applicant to the founder, never to a decline
D3 route = 18+ invite; 12-17 founder; under 12 proposed decline
D4 decline = a named person confirms it or moves the applicant to
the founder; no other path sets "declined"
D5 send = keyed on applicant id plus invite or decline; a decline
names how to ask for a review
Exceptions CV under 150 words of text. Quotes fail twice. A review is requested.
A proposed decline unreviewed after 1 working day. A system down.
Done Within 3 days of applying, every applicant is in one end state:
invited, declined with a named decider, or held with a reason and owner.
Redaction, the quote check and the decline review, steps 2, 5 and 8, are new since module 3.
The mistake: 90 declines nobody decided
In module 3's drawing, a rule failure or a total under 12 went straight to a decline email. Suppose the founder reads the first role's 90 automatic declines afterwards and finds 9 she would have kept. Five had a portfolio link with a typo or a host that was down that day. Three had two-column CVs whose text came out of extraction in the wrong order, so the model scored half a CV. One lost points on a criterion for a two-year gap the rubric never mentions.
That is 10% of declines wrong. The module 1 exercise put a missed hire at $300, and approval gates gates an item when error rate times cost exceeds the review:
expected loss 10% × $300 = $30 a decline
review 1.5 minutes × $30 ÷ 60 = $0.75 a decline
break-even $0.75 ÷ $300 = 0.25% wrong
The money says gate every decline, forty times over; sampling would pay only below 1 wrong decline in 400.
The law says the same where it applies. Article 22 of the GDPR gives a person "the right not to be subject to a decision based solely on automated processing" that "similarly significantly affects" them (Regulation (EU) 2016/679 of the European Parliament and of the Council (opens in a new tab), Article 22), and the regulation's recital 71 (opens in a new tab) names "e-recruiting practices without any human intervention" as an example. The EU AI Act lists systems used "to analyse and filter job applications, and to evaluate candidates" in Annex III, point 4 (ANNEX III, AI Act Service Desk (opens in a new tab)), and Article 6(2) treats Annex III systems as high-risk (Article 6: Classification rules for high-risk AI systems (opens in a new tab)). A deployer must assign human oversight to people with "the necessary competence, training and authority", keep the system's logs for at least six months, and tell the people it is used on (Article 26: Obligations of deployers of high-risk AI systems (opens in a new tab)).
These bind where your applicants or your hiring are in the EU; elsewhere, check the local rule. Whether building the screen makes you its provider too, and from when each duty applies, are questions for counsel before the first role opens. The design does not wait on the answer: the arithmetic already put a person on every decline.
The rung, and why not the one below or above
This is a rung 3 build: one graph thread per applicant, state saved after every step, and a stop wherever a person decides. In LangGraph the two human steps are interrupt() calls on the applicant's thread, saved by the checkpointer and resumed with the decision, however long the person takes (Interrupts (opens in a new tab); Persistence (opens in a new tab)). A resumed node reruns from its first line, which is why step 9's send carries a key.
Why not rung 2. Module 3 chose the canvas for an owner who cannot write Python; three things overturn that here. The property that must never break, no decline without a decision record, is a unit test run on every change, whereas on a canvas it is a node nothing runs before an edit goes live, and applications arrive at any hour. Other code reads the state by applicant: the deletion job, an applicant's request to see the decision, the bias report. And a threshold change is a fairness decision that belongs in a one-line diff beside its test. The price is a developer on retainer, assumed at 2 hours a month at $80: $160.
Why not rung 4. An agent that "screens applicants" picks its own path: read the CV, open the portfolio, perhaps search for the person, stop when satisfied. Two applicants with the same CV can take different paths, and the record holds a transcript instead of a rule. The tokens are the smaller cost. With the per-turn arithmetic, a 3,100-token start (rubric and job description 2,000, tool schemas 800, form 300) and about 800 more each turn:
6 turns 6 × 3,100 + 800 × (0 + 1 + … + 5) + 100 × 6 = 18,600 + 12,000 + 600
= 31,200 tokens
one call 3,700 in + 250 out = 3,950 tokens, an eighth
That is $9.90 a role against $1.41. The objection is not the $8; it is a selection procedure that differs per applicant.
The graph
Invites need no gate. An invite harms no applicant, and a wrong one costs a 20-minute first call, $33 of the founder's time; at an assumed 15% unsuitable that is $5 of expected loss against $5 to read the invite first.
The gates
The founder's band. 36 applicants at about 3 minutes, in two sittings a day: 108 minutes, $180 a role at the assumed $100 an hour.
Every proposed decline. The operations manager, trained on the rubric and with the authority to move anyone up, reads one card per applicant: for a rule failure, the rule and its evidence, about a minute; for a low score, five scores with five quoted lines and the CV one click away, about 1.5 minutes. That is 30 + 90 = 120 minutes, $60 a role at $30 an hour. A proposed decline unreviewed after one working day goes to the founder. A timeout never declines, the same rule as approval gates' expired state.
Whether the review is a decision. A reviewer who confirms 90 of 90 in four minutes has signed a list. Two numbers tell the difference: the median time per card, which should sit near 1.5 minutes, and two seeded cards per role, strong CVs from the eval set given a total of 10, marked internally and never mailed. A review that confirms a seed is a click-through, and the weekly review hears about it that week.
Review on request. Every decline names how to ask for a person to look again. Assume 3 requests a role at 10 minutes of the founder's time: $50.
The connectors
The applicant system posts each submission by webhook, with a nightly reconciliation poll as the backstop webhooks versus polling recommends.
The portfolio check is a request with a 10-second timeout and two retries with backoff. A 404 is a rule failure, after the applicant's one chance to fix the link; a timeout or server error is "could not check", an exception, not a failure, the split rate limits, retries and backoff draws.
The model provider receives redacted text only. Redaction runs inside the tenant before the call, as the data map prescribes, and the provider's retention goes in the map.
Mail goes from one identity, hiring-bot, through two templates, keyed on applicant id plus kind, so a resumed thread cannot send twice.
Retention. The CV, the checkpoints and the decision log are deleted six months after the role closes: the promise in memory and state, and long enough for Article 26's six-month minimum on logs.
The budget
The two sums from how to use this course, per role, at roughly one role a month.
Sum 1. The founder reads every CV, 150 × 8 minutes = 20 hours × $100 = $2,000. The operations manager sends the answers and books the calls, 150 × 2 minutes = 5 hours × $30 = $150. $2,150 a role.
Sum 2, once. A contractor, 40 hours × $80 = $3,200; the founder on the rubric, 6 hours, $600; an assumed $1,000 for a lawyer to read the design and the applicant notice. $4,800.
Sum 2, run. Tokens: 150 × (3,700 × $2 + 250 × $8) per million = $1.41, about $4 with retries and eval runs. Hosting the graph and its database, assumed $25. The retainer, $160. $189 a month.
Sum 2, people.
founder's band 36 × 3 min 108 min × $100/h $180
proposed declines 30 × 1 + 60 × 1.5 120 min × $30/h $60
review requests 3 × 10 min 30 min × $100/h $50
unreadable CVs 5 × 5 min 25 min × $30/h $13
bias check after each role 30 min × $30/h $15
weekly review 4.33 × 20 min 87 min × $30/h $43
total $361
Sum 2 is $189 + $361 = $550 a month. The saving is $2,150 − $550 = $1,600, and the build pays back in $4,800 ÷ $1,600 = 3 months. A person on every decline is $60 of the $550.
The input it turns on is whose hours Sum 1 counts. Had the operations manager screened by hand at $30, Sum 1 would be $600 + $150 = $750, the saving $200, and the payback 24 months, past the 12-month retirement date. The screen pays because it gives the founder back nearly 18 of her 20 hours a role.
The evals
Routing. The cases are the last role's applications, kept as redacted text for applicants who ticked a box agreeing to it, assume 100 of 150, and deleted at 12 months. The gates produce the labels for free: 36 founder decisions, 90 reviewer decisions, and the 24 invites marked after the first call. The line, as in what "working" means:
founder-labelled invites routed to decline 0 of 100 (must pass)
route agrees with the label 85 or more of 100
every score's quote found in the CV 100 of 100
a run 100 × $0.0094 = $0.94
At 85% on 100 cases the standard error is √(0.85 × 0.15 ÷ 100) ≈ 3.6 points, so 85 against 82 is noise and 85 against 75 is not.
The bias check, before a role opens. Redaction removes the name, but proxies survive it: a two-year gap, a degree from abroad, non-native phrasing, a two-column layout. Take 30 CVs from the set, make one variant of each per proxy, 120 variants, and score both sides of every pair. The line: no variant changes route, and totals move by at most 1 point on at least 28 of 30 pairs per proxy. A run is 120 × $0.0094 = $1.13, so it runs on every change to the prompt, the rubric or the model.
The bias check, after a role closes. Selection rates by group, from the voluntary form, where local law allows collecting it. The US guidelines treat a group's rate below four-fifths of the highest group's as evidence of adverse impact, and warn that small numbers may not be reliable (§ 1607.4 Information on impact (opens in a new tab)). Suppose 60 women and 75 men applied, and 11 and 21 advanced: 18.3% against 28.0%, a ratio of 0.65, under the line. One more woman advanced moves the ratio by 0.06, so a single role cannot settle it. The ratio sends the reviewer to read the 49 declined women's cards before the next role opens, about 74 minutes, and the ratio itself is judged over a year's ten roles.
In practice
Tell the team: "The screen scores and proposes. A person decides every decline, the record names them, and a decline nobody has signed goes to the founder, never out." Ask a vendor selling screening three things: who decides when the system scores an applicant low, what the record shows that person, and the counterfactual results on your own rubric.
The mistake that costs money is letting a score decline an applicant. At 10% wrong and $300 each, every unreviewed decline carries $30 of expected loss against 75 cents of review, and it is the decision Article 22 was written about.
Check yourself
1. Over three roles the reviewer confirmed 270 of 270 proposed declines at a median of 20 seconds, including both seeded cards last role. What do you do?
Treat the declines as unreviewed. Twenty seconds is not five quoted lines, and the seeds show the review confirms whatever it is shown. Stop sending from that queue, re-review the last role's 90 properly, 30 × 1 + 60 × 1.5 = 120 minutes or $60, and change the card so the reviewer records which criterion they checked.
2. The founder wants applicants who answer "no" to the right-to-work question declined automatically, since a no is a no. Decide.
Keep the review. Assume 10 such answers a role at a minute each: 10 minutes, $5. A form answer can be a mis-click or a misread question, and the decline would be a decision about a person made by code alone. If the minutes bother anyone, make the form confirm the answer before it submits.
3. A new prompt passes the routing set, but in the gap pairs 6 of 30 totals drop by 2 points or more and 2 applicants move from the founder's band to a proposed decline. Ship it?
No. The line allows at most 2 of 30 pairs to move more than a point and no route changes; this fails both. The likely cause is a criterion the model reads as continuity of employment. Rewrite it to score evidenced skills and rerun the pairs, $1.13, until the gap stops moving the total.