Multi-agent patterns
Price fan-out and gather, generate then verify, and a judge panel against one call, and pick the one that catches the error you actually have.
On this page
Three patterns cover most multi-agent designs: fan-out and gather, generate then verify, and a judge panel that votes. Each multiplies the cost of one call by a known factor, and each catches one class of error while missing the others. The design question is never whether more agents are more reliable. It is which error the eval shows, and which pattern is the cheapest one that catches it — often a single call with a checklist.
Fan-out and gather
Split the input into independent pieces, run one call per piece in parallel, and gather the results in one more call or in code. Thirty supplier contracts of 5k tokens each, and the question is which ones renew automatically with less than 60 days' notice. One call reads 150k: it fits the course's 200k window, from the four numbers, but contracts 10 to 20 sit in the middle of a long context, where recall drops — the Liu et al. result in tokens and the context window. Fanned out, it is 30 calls of 5k plus 1k of instructions, 180k input in all, and a gather in code over 30 small JSON objects: 1.2× the single call's input, at the wall time of one call.
What it catches is omission: items missed because attention was spread across a long context. What it misses is anything that needs two pieces at once, such as a renewal clause overridden by an amendment filed as a separate document. No worker sees both, and the gather sees only their answers. The independence test in subagents and orchestration is the check to run before splitting.
Generate then verify
One call produces, a second checks, and a failure goes back to the generator with the verifier's finding. Anthropic's "Building effective agents (opens in a new tab)" calls the looped form evaluator-optimizer. The cost is (1 + v) × (1 + r), where v is the verifier's cost relative to the generator and r the share regenerated and verified again.
The cheapest and strongest verifier is code: a schema, an invariant, a lookup. That is the check and retry of why reliability lives in the harness, and its v is close to zero. A model verifier earns its cost only when the check needs reading — does every claim in this summary appear in the source? For a 1,000-token summary of a 5k source, counting weighted tokens as input plus five times output:
generator 5,000 in + 1,000 out = 10,000
verifier 6,000 in + 100 out = 6,500
regenerate 10% × (10,000 + 6,500) ≈ 1,650
total ≈ 18,000 (1.8× one call)
It catches errors visible against something the generator did not check: an unsupported claim, a total that does not add up. It misses errors the verifier shares. The same model reading the same source with a similar prompt tends to accept what the generator believed, so give the verifier a narrower question — "list every number in the summary and the source sentence it came from" — and it becomes a different check rather than a second opinion.
A judge panel with a vote
The same output goes to three judge calls and code takes the majority, at three times the cost of one judge. If one judge is right 90% of the time and its errors are independent, the majority is right whenever at least two are:
0.9³ + 3 × 0.9² × 0.1 = 0.729 + 0.243 = 0.972
97.2% for three times the cost looks cheap. It assumes three independent judges, and three samples of one model with one prompt are not that. Suppose 7 of the judge's 10 points of error are systematic: cases it gets wrong every time, such as preferring the longer answer, one of the biases Zheng et al. measured and LLM-as-judge and its failure modes covers. The vote cannot touch those 7 points, because all three samples agree, wrongly. It fixes only the random 3:
systematic error 7% (all three wrong together)
random error per judge 3 ÷ 93 ≈ 3.2% on the other cases
majority wrong on those 3 × 0.032² × 0.968 + 0.032³ ≈ 0.3%
panel accuracy 1 − (0.07 + 0.93 × 0.003) ≈ 92.7%
A panel catches inconsistency, a verdict that flips between runs. It does not catch a blind spot the judges share. Diversity helps only when it is real: different models, different prompts, or different evidence.
When one call with a checklist beats a panel
Measure which error you have before choosing. Run the single judge five times on the same 50 cases. If verdicts flip between runs on many of them, the error is inconsistency, and a lower temperature (free) or a panel (3×) helps. If verdicts are stable and wrong against human labels, the error is systematic, and no number of copies fixes it.
For systematic error, change the question. "Rate this answer 1 to 5" invites length bias; a checklist of five binary criteria, each answered with the quoted evidence, makes the judge look for specific things. Criteria that code can check — a required field is present, a price matches the catalogue — leave the judge entirely. One call with a checklist costs about 1.1× a holistic judge, for the extra tokens of evidence, and attacks the part of the error a panel cannot. Calibrate it against the human labels in the golden set before trusting it.
| Pattern | Cost against one call | Catches | Misses |
|---|---|---|---|
| Fan-out and gather | N + 1 calls, about 1.2× the input | items lost in a long context | anything needing two pieces at once |
| Generate then verify | about 1.8× with a model verifier | claims or totals checkable against a source | errors the verifier shares |
| Judge panel of three | 3× | verdicts that flip between runs | biases the judges share |
| One judge with a checklist | about 1.1× | holistic-scoring bias, skipped criteria | criteria not on the list |
In an interview
What is tested is whether you price a pattern and name its error class. Say: "three judges triple the cost; at 90% per judge they reach 97% only if the errors are independent, and with 7 points of shared bias they reach about 93%". Then give the measurement that tells you which case you are in: rerun one judge on the same cases and see whether verdicts flip.
The mistake that loses points is treating agent count as reliability: proposing a panel, a debate or a verifier without naming the error it catches, or without checking that this is the error the eval shows.
Check yourself
1. A summariser's eval shows 12% of summaries contain a claim not in the source. A model verifier, or a judge panel?
A verifier with a narrow question: list each claim and the source sentence that supports it, and regenerate on any unsupported claim. An unsupported claim is a fact to check, not an opinion to vote on. Expect about 1.8× the cost of one call; keep it if the eval then shows unsupported claims below 2%.
2. A three-judge panel and a single judge both agree with human labels on 88 of 100 cases. What do you do?
Drop to one judge. Identical agreement means the vote changed nothing, so the disagreements are systematic and shared; the panel was spending two extra calls per case for no gain. Read the 12 disagreements for a common cause — length, a missing criterion — and turn it into a checklist item or a code check.
3. A fan-out over 50 documents returns 50 answers, and the gather writes a wrong conclusion because two documents contradict each other. Which pattern failed?
Fan-out did what it can do; the error was cross-document, which fan-out cannot catch by construction. Have each worker return its claim with the supporting quote, and make the gather — code, or one call that sees only the claims — look for contradictions before it concludes.