Deterministic first: the harness pattern
Sort every step of a task into code, model or person, put a code check after each model step, and price the wrong split in editor hours.
On this page
A pipeline built deterministic first gives the model one job per step, and only the steps no rule can do. Code fetches, validates, counts, formats, decides whether the run continues and performs every side effect; the model writes, classifies or scores, and code checks what it produced before anything reads it. Why reliability lives in that code rather than in the prompt is derived in why reliability lives in the harness. This lesson applies it to one of the agency's pipelines and prices the wrong split.
Which steps get a model
One question sorts a step: given the same input, could two careful staff disagree about the right output? If not, the step is code. Five kinds of step never get a model:
- Lookups. Reading a brief, finding a contact by email, fetching last month's posts. The answer exists; a model can only reproduce it or not.
- Rules you can write down. Length within 10% of target, a score of 18 or more, a due date at least three working days away. A model applying a rule is right most of the time; code is right every time, and its failures are bugs a test finds once.
- Arithmetic. Totals, tax, word counts.
- Side effects. Sending, publishing, paying, deleting. The model may propose; code acts, once, with a key.
- Whether the run continues. A gate is a condition in code, or a person.
A step gets a model when its input is free text or its output is a judgment no rule states: a draft in the client's voice, a category for an email someone wrote in a hurry, a score against a rubric with a quoted line as evidence.
Anthropic's "Building effective agents (opens in a new tab)" names five workflow patterns built from these pieces: prompt chaining, routing, parallelization, orchestrator-workers and evaluator-optimizer. The pipeline below is the first, a chain, with a code gate between the links.
Running example: the content pipeline
The agency writes 8 blog posts a week for its clients, and an editor spends about 20 minutes on each before it goes out.
| # | Step | Who | What it does |
|---|---|---|---|
| 1 | Read the brief | code | the row from the content calendar |
| 2 | Validate the brief | code | client, topic, audience, keyword, length 600–1,500 words, due at least 3 working days out |
| 3 | Gather sources | code | the client's style guide, product page and last three posts, by fixed query |
| 4 | Draft | model | brief and sources in, markdown out |
| 5 | Check the draft | code | length within 10%; keyword in the first 100 words; every link resolves; every figure and product name appears in the sources; no banned phrase |
| 6 | Headline and meta | model | three headlines and a meta description |
| 7 | Check and format | code | headline at most 65 characters, meta at most 155, keyword in both; convert to CMS blocks |
| 8 | Publish | code | schedule in the CMS with an idempotency key, after the editor signs off |
Six code steps, two model steps, and a person between 7 and 8.
The order is the point. About one brief in eight arrives missing a field, usually the audience or the keyword. Step 2 returns it to the account manager in milliseconds, naming the field, before a token is spent. A model handed the same brief does not refuse: it writes for an audience it guessed, and the miss surfaces twenty minutes into the editor's pass. Step 5 sits directly after step 4 so a defect is caught at the step that made it, and the retry reruns one step with the failure attached, "the draft says 40% faster; the sources say 30%", not the whole pipeline.
The mistake: the model checks its own work
The tempting version of step 5 is a sentence in step 4's prompt: "use only figures from the sources". It holds most of the time, nothing records when it does not, and there is no function to test. The code version is about ten lines: pull every number and product name out of the draft, look each up in the source text, and fail the check naming any that is missing. It cannot say whether the draft is good. It can say, every time, whether a figure was invented, and that is the defect a client notices.
What the error rate does when the split is reversed
The assumptions, labelled so you can substitute your own: each model step produces a defect 5% of the time, measured on a golden set as evals as the unit test of LLM work describes. The code check after it catches 80% of those, and a caught defect gets one retry. Code steps give the same output for the same input, so their failures are bugs fixed once, not a rate.
Code first, with the checks:
escapes one model step 5% × (1 − 0.80) = 1%
escapes either of two 1 − 0.99² ≈ 2% of posts
at 8 posts a week 8 × 0.02 ≈ 0.16 a week, one every six weeks
Now reverse the split. The common shortcut is one agent with a search tool, a CMS tool and a prompt that says: check the brief is complete, gather sources, draft, check the draft, write the headline, format it. Code reads the brief to trigger the run and publishes; the model does the other six steps, and every check is the model reading its own output in the same context, which is not an independent check.
all six right 0.95⁶ ≈ 0.735
posts with a defect 1 − 0.735 ≈ 26.5%
at 8 posts a week 8 × 0.265 ≈ 2.1 a week
Thirteen times as many, and none of them flagged. The brief with no audience no longer stops at step 2 either: the model's "validate" step is one of the six, and it fills the gap. Even code first without the checks, two model steps unguarded, is 1 − 0.95² ≈ 9.8% of posts, 0.78 a week. The split and the checks each do part of the work.
What the wrong split costs
Tokens are not the bill. Assume the large model at $2 per million input tokens and $8 per million output. The draft step reads about 5,300 tokens, a 1,000-token prompt, a 300-token brief and 4,000 tokens of sources, and writes about 1,600, a 1,200-word post at 1.3 tokens a word: 5,300 × $2 ÷ 1M + 1,600 × $8 ÷ 1M ≈ $0.011 + $0.013 = $0.024. The headline step adds about $0.005. Eight posts cost about $0.23 a week.
The editor's time is the bill. A hidden defect turns a 20-minute edit into a 60-minute rewrite. Reversed, 2.1 × 40 extra minutes ≈ 85 minutes a week, 1.4 hours at the assumed $30 an hour for staff: $42 a week, about $185 a month. Code first, 0.16 × 40 ≈ 6 minutes a week, about $14 a month. The defect the editor misses is a wrong product figure on a client's blog, which no hourly rate prices.
When a model step's rule turns up later, the step moves to code without the rest of the pipeline changing, because each step is a function the scheduler cannot see inside, as workflow scripts and deterministic harnesses sets out. The split carries over to either skin of one idea, two skins: a model node between two code nodes.
In practice
Ask for the pipeline drawn with every step labelled code, model or person. For each model step ask three things: what check runs after it, what that check catches, and which golden-set number the error rate came from. Then ask of each model step what rule would replace it; if the answer comes quickly, it is a code step.
The mistake that costs money is a model step where a rule would do. Each one adds a defect rate a check can only partly remove, and a model doing validation turns a millisecond rejection at step 2 into an hour of rewriting at the editor's desk.
Check yourself
1. The account manager asks to drop step 2 because the model "handles missing fields fine". Estimate what that costs a week, and decide.
One brief in eight, one a week, is now drafted on a guess. If half the guesses miss what the client wanted, an assumption to replace with your own count, that is 0.5 rewrites a week at 40 minutes over the normal edit: 20 minutes, $10 a week at $30 an hour, and a wrong audience on a client's blog whenever the editor misses it. Step 2 is a dozen lines of code. Keep it.
2. Step 5's link check takes 4 seconds a post because it requests every URL. A teammate proposes asking the model whether the links look valid. Decide.
No. The model cannot know whether a URL returns a page; it can only judge whether it looks plausible, which is exactly the failure the check exists to catch. Four seconds at eight posts a week is 32 seconds a week. Keep it in code.
3. Three additions are proposed: pick three internal links from the client's last 50 posts; convert the headline to the style guide's title case; flag any post that names a competitor. Which get a model?
Only the first. Relevance is a judgment, so picking links is a model step, followed by a code check that each link is one of the 50. Title case is a rule, so code. A competitor flag is a string match against a list the client supplies, so code; a model would add a miss rate to a check that the list makes exact.