Evals as the unit test of LLM work
Write an eval case as input, expected and grader, pick the cheapest grader that can tell right from wrong, and run the set on every prompt or model change.
On this page
A unit test pins a function's behaviour so that a change which breaks it fails before it ships. A feature built on a model needs the same guarantee and cannot have it in the same form: output varies between runs, a correct answer is one of many strings, and the model behind the name changes without a commit in your repository. An eval keeps the shape of a test suite — fixed inputs, an expected result, a verdict per case — and replaces the assertion with a grader and the green tick with a pass rate. It runs on every prompt change the way the test suite runs on every commit.
The demo is not a test
The way most teams check a prompt change: paste five tickets that failed last week into the playground, read the answers, ship. The five are the tickets the change was written for, so they pass by construction; nobody looked at the kinds of ticket it could break; and the reading is one person's unrecorded judgment, so the next change cannot be compared with this one.
The ticket classifier in prompt design as interface design shows what the alternative buys. Revision 3 moved the score from 166 to 182 of 200. A control revision that only rewrote the role sentence — "an expert support engineer with ten years of experience" — read like an improvement in review and scored 165. The set said it changed nothing.
A case is input, expected, grader
Three parts, plus labels the report groups by:
{
"id": "tri-0142",
"input": { "ticket": "I was charged twice for March and now I can't log in." },
"expected": { "label": "billing" },
"grader": "exact",
"slice": "billing-account boundary",
"source": "transcript 2026-09-14"
}
The input is exactly what the feature receives — the ticket, the retrieved policy, the tool results a loop would see — frozen, so the case reproduces next month. The expected value is whatever the grader needs: a label, a set of fields, the facts an answer must contain, or a rubric. The grader is chosen per case, not per suite. Slice and source cost nothing and say which kind of input got worse and where it came from.
A suite is a versioned file of cases in the repository, next to the prompt it tests. Two hundred is the working size here; building the golden set covers where they come from and what 200 can see.
Three grader kinds
Exact or schema. The output is parsed and compared: the label equals billing, the date equals 2026-03-31, the object validates against the schema. Microseconds per case, no tokens, and the verdict never varies. Classification, extraction and routing are graded this way.
Code-checked. The output has no single right string but has properties a function can check: the line items sum to the total, every cited clause is in the retrieved set, the generated SQL returns the reference query's rows. Milliseconds, no tokens, deterministic. These are the invariant and provenance checks from reading the response reused as graders, so the eval and the live path agree on what correct means.
Model-judged. A second model call reads the case, the output and a rubric, and returns a verdict in a schema. It is the only grader for what no code can check, such as whether a reply answers the question or a summary is faithful. It costs a call per case per run and has an error rate of its own, so its verdicts count only once it is calibrated against people: LLM-as-judge and its failure modes.
The rule is the cheapest grader that can tell right from wrong for that case. The common mistake runs the other way: a team that has built a judge grades everything with it, labels included, paying a call per case for a noisier verdict than string equality.
One feature, graded three ways
A reply drafter returns a queue label and a draft of about 120 words. One case grades both:
- the label, by exact match
- the draft, by code: contains the order number, contains no email address but the customer's, stays under 150 words, promises no refund above the order total
- the draft, by a judge: answers the question asked, and states the policy correctly
The code checks run first, and a draft that fails one is failed without a judge call. If 10% of drafts fail a code check, the judge sees 180 of the 200 cases.
The cost of a run, in the course's names. The drafter reads about 3k tokens and writes about 160, so 200 cases are 600k × P_in plus 32k × P_out. A judge call reads the rubric, ticket and draft, about 1.2k tokens, and writes a 60-token verdict, so 180 judgments add 216k × P_in and 11k × P_out, roughly a third on top. Exact and code-graded cases are cheap enough for every change; judged cases run when a change is proposed for merge.
The suite runs on every change
A change is wider than the prompt text. The model version, the temperature, a tool description, the chunk size in retrieval, the output schema — any of them can move the pass rate, and the eval is where that shows before a customer sees it.
One change arrives without a commit: the provider updates the model behind the name, as sampling and determinism warns. So the suite also runs nightly with prompt and code held fixed. Pin a dated model version where the API offers one and record the version in every result, so a drop with nothing merged has a suspect.
Speed decides whether people run it. At the course's latency figures (about 1 s to the first token and 50 tokens/s, from the four numbers) a drafter call takes about 4.2 s and a judge call about 2.2 s, so a judged case is about 6.4 s, and two hundred cases with ten in flight take 20 × 6.4 ≈ 130 s, about two minutes. Under ten minutes, the suite runs on every pull request; over thirty, people stop waiting and merge.
What comes out is a report, not a boolean: the pass rate, the per-slice rates, the paired diff against the last baseline — which cases flipped, and which way — and the cost per task and p95 latency from the same run. Turning that report into ship or hold is regression gates.
Where the analogy stops
A suite has a pass rate, not a verdict, and on 200 items its standard error is near 2 points. A case can flip between two runs of one prompt, so the suite needs repeats to know its own noise. Its expected values are people's labels, wrong at a measurable rate. And it costs tokens, so its size and schedule are a budget.
The product manager's view is evals as the feature's test suite. For an automation that runs unattended, the eval set, drift alarms and weekly review are taught in what working means.
In an interview
"How would you know the new prompt is better?" tests whether you turn "is it good" into a number. Say: "a versioned set of cases, each an input, an expected value and a grader — exact match for labels, code checks for invariants, a calibrated judge only for free text — run on every change and nightly, reported per slice with the paired diff against the baseline." Then give the set's size and the smallest change it can see.
The mistake that loses points is "we tested it on some examples and it looked right" — a demo described as a test. Close behind: a model judge for everything, labels included, with no number for how often it is wrong.
Check yourself
1. An invoice-extraction feature returns supplier, date, currency, line items and total. Which grader does each field get?
Schema validation first, then exact match for supplier, date, currency and total, and a code check that the line items multiply out and sum to the total. No judge: every field has a checkable right answer, so a judge adds tokens and error and decides nothing.
2. A team wants to judge all 200 cases with a model on every commit, and merges about 30 commits a day. Estimate the judge's input tokens and say what you would change.
200 × 1.2k = 240k input tokens a run, × 30 = 7.2M a day, 7.2M ×
P_in, to grade outputs that are mostly labels. Grade labels and fields exactly on every commit, judge only the free-text cases, and run those on the pull request and nightly.
3. The nightly run drops from 182 to 170 of 200 with nothing merged since yesterday. What happened, and what does the suite need?
Twelve tickets is far more than the two or three that flip between runs of an unchanged prompt, so it is likely real. With no commit, the suspect is the model behind the name, or the judge's model if the drop is in judged cases. Check the recorded version, pin a dated one, re-run to confirm, and read the paired diff for the slice that moved.