Five eras, one ladder
Place your own codebase on the ladder of five ways to put a model in a product, ordered by how much of the loop lives in code rather than in the prompt.
On this page
Every product that calls a model has a loop somewhere: something decides what to send, what to do with what comes back, and when to stop. The five ways of building that loop are usually told as history — prompting, then agents, then skills, then graph automation, then harnesses — but the useful order is not the dates. It is how much of the loop lives in code. Prompting puts none of it there; a harness puts all of it there and leaves the model one job per step. Read the ladder that way and the question for your codebase becomes "where does our loop live, and what failure is about to push it up a rung".
The same feature on every rung makes the ladder concrete: triage of inbound support email. A ticket arrives, something labels it — billing, bug, how-to, refund, spam — routes it to a queue, and in the refund case decides whether a person has to sign. Assume 2,000 tickets a day and a ticket of about 300 words, roughly 400 tokens in the units defined in the four numbers.
Rung 0 — prompting: the app was the prompt
Version one is a single call. The system prompt holds the five categories, the routing rules, twelve worked examples and the refund policy — about 3k tokens — and the ticket goes in after it. One call per ticket, roughly 3.5k tokens in and 60 out: a label and a one-line reason. There is no loop; the code around the call is a parser and a switch. Prompt design as interface design is the engineering discipline of this rung, and it is not a small one.
It works for two weeks. The failure that ends it is a billing ticket that says "my order 48213 was charged twice". The prompt has no order data, so the model guesses whether a refund is due, and nobody notices until finance counts: on tickets that need a fact the prompt does not contain, assume one label in eight is wrong. The fix is not a better prompt. The model needs to fetch something, and a fetch is a loop.
Rung 1 — agentic flows: the model picks the next call
Version two gives the model three tools — lookup_order, lookup_customer, label_ticket — and lets it decide what to call. The reason-act pattern of Yao et al., "ReAct", is the source: read the ticket, call lookup_order, read the result, label. Three calls per ticket is typical, and each call resends everything before it, so the count is a triangle, not a line: about 4k on the first call, 4.6k with the order result, 5.2k with the labelling call — roughly 14k tokens per ticket where rung 0 paid 3.5k, on the tickets that go well.
The failure is the ticket with no order number in it. lookup_order fails; the model tries the customer's name, then their email, then the order number with a digit changed, and the forty-first call is still a lookup. That ticket costs 200k tokens and four minutes. The loop had no exit, and agent loops and stop conditions gives it three. The quieter failure: every procedure a ticket might need — chargeback steps, fraud checklist, enterprise-account rules — has gone into the system prompt, now 9k tokens and paid on every call of every ticket, including the nine in ten that needed none of them.
Rung 2 — skills: procedures packaged as files
Version three takes the procedures out of the prompt. Each becomes a folder with a one-line description and a body, and the loop loads a body only when the model decides the description applies. The system prompt drops to 2k plus the descriptions — six skills at forty tokens each, always paid — and the 1.2k-token chargeback body loads on the one ticket in ten that mentions a chargeback. Tokens per ticket average about 12k across the same three calls — 3.2k, 3.8k and 4.4k, plus the occasional chargeback body — against 32k with the 9k prompt; skills as packaged procedures does the arithmetic properly.
Note what did not change: the loop still belongs to the model. The prompt still says "retry a failed lookup once, then escalate", and that line holds most of the time. One Friday it does not, and a ticket reaches the wrong queue with no record of why. A rule that must hold every time cannot be a sentence the model usually follows.
Rung 3 — graph automation: the model is one node
Version four moves the control flow into a graph. In a visual orchestrator such as n8n the trigger is a new-ticket webhook, a code node fetches the order before any model runs, a model node classifies with the order data in hand, a branch node routes on the label, the HTTP node carries its own retry, and the refund branch ends in a human-approval node. The model is one box among forty. Back to one call per ticket, sometimes two, at about 3.5k tokens, and the lookup cannot loop because the graph decides what runs next. This rung is older than the one before it — graph tools had model nodes before anyone packaged a procedure as a file — which is why the ladder is ordered by where the loop lives, not by date.
The failure is the fortieth node. The prompt is a string inside a JSON export; no test runs when it changes, and renaming a category breaks three branches silently. When the second feature arrives — draft a reply, not only a label — the graph doubles, and the team is maintaining a program with no diff, no unit test and no types. Graph tool or code for a given procedure is the subject of the fourth rung of the automation ladder in the agents course.
Rung 4 — harness workflows: code owns the loop
Version five is a script. A harness is a DAG of steps, each either code or one model call, with typed inputs and outputs, a token budget per step, one retry on a failed check, a resume file so a crash at step 4 restarts at step 4, and a stop point where a person signs off on a refund. The model owns the judgment inside a step — "which of these five labels, given this ticket and this order" — and nothing else. Calls and tokens per ticket match the graph. What is new is testability: the classify step is a function that runs against a 200-ticket golden set on every prompt change, and the retry, the budget and the ordering are code a test can exercise with no model present. Why reliability lives in the harness shows the arithmetic; workflow scripts and deterministic harnesses shows the shape.
There is no rung above this one, but there is a failure that pushes a team back down: a harness built for a feature one prompt would have done. A 200-line scheduler around a single call with nothing to retry is rung 0 with ceremony.
Where your codebase sits
Three questions place a codebase, and the answers are usually mixed. Who decides the next call — the prompt, the model, a graph, or a script? Where do the procedures live — in the system prompt, or in files loaded on demand? What runs when a step fails — a sentence the model reads, or code that fires every time? A codebase can be rung 2 for procedures and rung 1 for control flow; that mix is the normal state.
Mark the rung for each question; the course then reads as a route. Modules 1 to 3 are what every rung needs: counting, tools, cost. Module 6 is the move from rung 1 to rung 2, plus hooks, the first piece of rung 4 most teams adopt. Modules 7 and 8 are rung 4 in full. The failure about to push you up is the one to read first.
In an interview
"How would you build support triage?" is a ladder question wearing a feature's clothes. The interviewer wants to see you start at rung 0, say what breaks it, and climb one rung per failure, with numbers — one call at 3.5k tokens, then three calls at 14k, then the uncapped loop at 200k. Each step up costs tokens or code, and each is justified by a named failure, not by fashion.
The mistake that loses points is starting at rung 4. A candidate who opens with a DAG, retries and a resume file for a feature nobody has shipped once has skipped the question of whether one call would do. Start simple, break it on purpose, climb.
Check yourself
1. Your triage feature is one call per ticket, and 12% of refund tickets are mislabelled because the prompt cannot see order data. Give the model a lookup tool, or fetch the order in code before the call?
Fetch in code. The model needs one fact per ticket, a regex can find the order number, and a tool call would add a round trip on every ticket — roughly 4k tokens more — to let the model decide what you already know it needs. Give the model tools when the next call depends on the last result in a way code cannot predict.
2. A prompt line says "never issue a refund over the limit without approval". It has held for 3,000 tickets. Which rung is that?
Rung 1 or 2: the rule is a sentence, and 3,000 successes do not change its kind. A rule that must hold every time is a check in code before the refund tool runs — a hook. The prompt line can stay as an explanation; it stops being the mechanism.
3. A team has 40 nodes in a graph tool and is about to add reply drafting. Rewrite as a harness now, or add the nodes?
Count what the rewrite buys: a test on the prompt, a diff on every change, a resume point. If a model node's prompt changed three times last month with no eval, the rewrite is the eval, and it pays on the first regression it catches. If the graph is stable and drafting is one more node behind its own approval gate, add the node and write the 200-ticket golden set first — the set is what you carry either way.