Agent loops and stop conditions
Give an agent loop three exits, size its turn and token caps from real runs, detect a stuck loop, and record which exit fired as the trace's first field.
On this page
An agent loop is the tool-call loop run until the model stops calling tools. The model decides when it is finished, and most of the time it is right. The loop you ship is the one for the other times: the model that calls the same tool with the same arguments for forty turns, the task that needs 300 round trips because every step reads a page, and the run that is still going when the monthly budget is gone. Three exits cover those, and which exit fired is the first thing the trace records.
The naive version is the one everybody writes first:
while True:
reply = model(messages)
if reply.stop_reason == "end_turn":
return reply.text
messages += run_tools(reply.calls)
It has one exit, and the model owns it. When the model does not take it, the loop runs until the context window is full and the API rejects the send, and the cost of getting there is not linear in the number of turns.
Why cost is quadratic in turns
Every turn resends the whole conversation. With a 3,000-token prefix and 1,500 tokens added per turn (a call and its result), the input on turn n is 3,000 + 1,500 × (n − 1), and the cumulative input over N turns is
N × 3,000 + 1,500 × N × (N − 1) / 2
| Turns | Input on that turn | Cumulative input |
|---|---|---|
| 10 | 16,500 | 97,500 |
| 25 | 39,000 | 525,000 |
| 40 | 61,500 | 1,290,000 |
A loop that runs four times longer than it should costs thirteen times more, before output. That is the arithmetic behind each cap, and the reason the token cap is a cost cap and not a context-size check: the context only tells you the size of the next send, not what the run has already spent.
Exit 1: the turn cap
Count round trips and stop at a number. Choose it from the distribution of successful runs, not from a guess: if most tasks that finish well take 4 to 9 turns and the p95 is 12, a cap of 25 catches runaways while cutting almost no legitimate run short. Two times p95 is a reasonable first setting; the trace will tell you whether it is right.
The cap is blunt, and that is its merit. It needs no model and no interpretation. Its weakness is that 25 turns of a runaway loop is still 25 turns of cost, which is why it is not the only exit.
Exit 2: the token cap
Sum the input and output tokens of every call in the run and stop when the sum crosses a budget, with the budget expressed in tokens because that is what the bill is in. A budget of 400,000 input tokens in the table above fires on turn 22 (cumulative input 412,500), three turns before the turn cap of 25; on a run with large tool results it fires much earlier, which is the point. Each run reads the counts from the usage block of each response, the same counters where the money goes uses to build the cost sheet, so the cap costs nothing to compute.
Set it per task class. A task worth a few cents to the business gets a budget of a few tens of thousands of tokens; a task that replaces an hour of a person's time can spend a million. The budget is where the loop meets the product decision, and it should be a parameter, not a constant.
Exit 3: the no-progress check
A stuck loop looks like this in the trace: search_tickets("refund delay"), result; search_tickets("refund delay"), same result; search_tickets("refund delay"). The model has run out of ideas and is repeating the last one. Neither cap catches it quickly, because each turn is cheap, and a cheap turn forty times is still forty turns.
The check is simple: hash each call as the tool name plus its arguments in canonical form (keys sorted, whitespace normalised) and stop when the same hash appears twice in a row. The same call with different arguments is progress; the same call with the same arguments and the same result is not. A two-call cycle (A, B, A, B) slips past a strict "twice in a row" check, so a window of the last four or six calls is worth the extra lines; the no-progress detector exercise builds exactly that.
Firing this exit on the second identical call rather than the third is a judgment call: a model that retries once after a timeout is behaving well, so exempt a call whose previous result was an error, or wait for the third.
Which exit fired is the first field in the trace
A run ends in one of five states: done, turn_cap, token_cap, no_progress, or error for a send the API rejected. Record the state as the first field of the run's trace, before the answer and before the token counts. Everything useful about the loop in production is an aggregate over that field.
"14% of runs ended in no_progress, and 11 of those 14 points were on search_tickets" is a tool-description bug, and the fix is in writing tool descriptions. "3% ended in token_cap, all on tasks that fetched more than ten pages" is a context-management bug, and the fix is in memory and context management. Without the field, both show up as "the agent sometimes fails", which nobody can fix. The observability lesson puts the field in the trace schema beside the per-call counters.
What to do when an exit fires
Return. The loop's output is a status and whatever the model had produced so far, and the code around it, the harness, decides what happens next: hand the partial answer to a person, retry the task once with a fresh context and a note about what went wrong, or fail. The loop never retries itself, because a loop that retries itself on turn_cap is a loop with a turn cap of twice the number. Why reliability lives in the harness takes this split further; the founder-facing course covers the same decision as deterministic first.
Put together, the loop is twenty lines:
def run(messages, tools, max_turns, token_budget):
spent, seen = 0, []
for turn in range(max_turns):
reply = model(messages)
spent += reply.usage.input + reply.usage.output
if reply.stop_reason == "end_turn":
return "done", reply.text
if spent > token_budget:
return "token_cap", None
key = canonical(reply.calls)
if seen[-1:] == [key]:
return "no_progress", None
seen.append(key)
messages += run_tools(reply.calls) # errors become results, never raise
return "turn_cap", None
In an interview
The interviewer wants the three exits named, sized and justified, and the sentence "which one fired is the first field in the trace". Give the quadratic arithmetic, because it is what makes the caps matter: a loop that overruns by 4× costs 13×. Then say where each cap's number comes from: p95 of successful runs for turns, the task's worth for tokens, a hashed window for no-progress.
The mistake that loses points is while True with the model as the only exit, or its disguise, "the context window is the limit". The window is a limit on the last send, not on the run's cost, and a loop that reaches it has already spent the cost of every turn before it. The second mistake is retrying inside the loop when a cap fires, which doubles the cap and hides the failure from the trace.
Check yourself
1. Most successful runs take 3 to 7 turns, p95 is 9, and the trace shows 6% of runs hitting a turn cap of 10. Raise the cap or not?
Not yet. Look at the 6%: if their last five calls repeat the call on turn 5, they are stuck loops that a no-progress check should have caught at turn 6, and raising the cap makes them cost more. If they are making distinct calls and finishing well when rerun with a cap of 15, raise it to 18, twice the p95, and watch the share fall below 1%.
2. A run's prefix is 5,000 tokens and each turn adds 2,500. Where does a 400,000-token input budget fire, and is that before or after a turn cap of 20?
Cumulative input after N turns is 5,000N + 2,500 × N(N − 1)/2. At N = 16: 80,000 + 300,000 = 380,000; at N = 17: 85,000 + 340,000 = 425,000. The token cap fires on turn 17, three turns before the turn cap, which is the right order: cost caps should bite first on large-result tasks.
3. The model alternates between get_order(123) and get_shipment(123) for twelve turns, never repeating a call consecutively. Which exit should catch it, and how?
The no-progress check, with a window: hash the last six calls and fire when a hash of the sequence (A, B, A, B, A, B) shows a cycle of length two. A consecutive-only check misses it, and the turn cap ends it only after all twelve turns are paid for; the window check fires on turn 6.