Reading the response
Read the stop reason before the text, size the output cap to the reply you asked for, and run the four checks that stand between a model output and the system that acts on it.
On this page
A response is a structure, not a string: a list of content blocks, a stop reason that says why generation ended, and usage counters that say what it cost. The first production bug most teams ship is reading the string and ignoring the rest — parsing a reply that was cut off at the output cap, or treating the preamble of a tool call as the answer. Four checks, run in order before any output is trusted, catch the whole class.
The anatomy of a response
Content is a list of blocks. A block is text, or a tool call carrying a name and arguments, and a response may hold several of either — or none. The stop reason is one of a small set: end of turn, the model finished; output cap, generation hit the maximum you set; tool call, the model wants a tool run before it continues; stop sequence, a string you asked it to stop at appeared; refusal, the model declined. Usage carries the input tokens, the output tokens and the cache read and write counts — the ledger that proves or disproves the arithmetic from tokens and the context window.
The stop reason decides the next step
End of turn: run the checks below, then use the output. Output cap: the reply is truncated — a JSON object cut mid-string, a summary that ends in half a sentence — and nothing parses it; the decision is one retry with a larger cap, or splitting the task. Tool call: run the tool, return the result and call again, the loop in tool use and the tool-call loop; a text block beside the call, if there is one, is a preamble and not the answer. Refusal: route to a person and log the input hash; the same input retried gives the same refusal, so a blind retry burns a call and learns nothing.
Two outages, both from reading the string first. A team parsed content[0].text and one night a response held a tool call and no text block; the index error took the feature down at 2 a.m. A summariser capped output at 1,000 tokens; a long contract's summary hit the cap, a lenient parser closed the cut-off string and the missing brace and accepted the truncated JSON, and the summary field shipped holding half a sentence. Both are one if on the stop reason.
Truncation and the output cap
The cap is a budget for the reply you asked for, not a safety margin. A 400-word summary is ~550 tokens at 1.3 tokens per word; set the cap at 800, not 4,000. The reason is that the timeout scales with the cap: 4,000 ÷ the course's decode speed (50 tokens/s, from the four numbers) is 80 seconds of worst case to wait for, and a runaway reply that uses it costs 4,000 × P_out. A cap near the expected length makes truncation a signal — the model wrote more than the task should need — and keeps the timeout honest, which is the arithmetic in retries, timeouts and rate limits.
Detect truncation by the stop reason, never by the length of the text; a reply of 790 tokens is complete if the stop reason says end of turn and truncated if it says output cap. And remember the cap counts against the window: the output reserve in the budget planner is the cap.
Empty content
A response with no text block is not "no result". It happens when a stop sequence matches at the first position, when a prefilled assistant turn is simply ended, when a refusal carries no text, and when the reply is a tool call alone. Each is a fact about the request's shape, and each is handled by the stop reason — except the stop-sequence case, which is a bug in the stop sequence. Log the stop reason and the input hash, raise a failure to the caller, and never return an empty string to a user or store one as the answer.
The four checks before trusting output
First, the stop reason is the one you expected: end of turn for an answer, tool call during a loop. Second, schema validity: types, enums, required keys, no extra keys. Third, business invariants: totals that add up, ranges, ids that exist, dates in order — code you write, in the constructor of the object you build. Fourth, provenance of any cited fact: a quoted passage must appear verbatim in the source you sent, a cited clause number must be in the set you retrieved, a figure must trace to a line. The implementation is a substring match and a set-membership test, microseconds each. The mechanism it defends against is plain: the model produces a plausible citation whether or not one exists, because a citation is the likely next token after a claim. The check is the only thing between "the contract says" and the contract; the model does not know your data is the longer argument.
Each check writes its result into the trace for that call, beside the tokens, the latency and the stop reason — the record observability and tracing builds on. The cost is nothing against a model call. A failing check costs one retry — budget exactly one — and a second failure goes to a person. At 10,000 calls a day with 3% failing a check, that is 300 retries of ~3k tokens: 900k input tokens a day, 900k × P_in, about 3% on the bill; if a fifth of those fail again, 60 items a day reach a review queue, two minutes each, two hours of someone's day. Those two hours are the price of never acting on an unchecked output.
Usage counters are the ledger
Reconcile the counters with the estimate. You sized the request at 29k input tokens and the counter says 31,400: the 2,400 are the message framing and the tool definitions you did not count, and now you know their size. The cache-read counter says 0 when you expected 12k: the prefix changed, and prompt caching says what moved. The output counter against the cap says whether the cap was sized right. These numbers, per call, are the alarm inputs for cost per task and p95 — the same counters the cost-control lesson in the sibling course, cost control: token budgets, caching and routing, reads once the feature runs unattended.
In an interview
What is being tested is whether you treat model output as untrusted input. The phrasing that scores: "stop reason first — end of turn, cap, tool call, refusal each go somewhere different — then schema, then invariants in the constructor, then provenance against the sources I sent, each logged to the trace, one retry, then a person." Name the output cap as a budget sized to the reply, and say what the usage counters are reconciled against.
The mistake that loses points is parsing the text before checking why generation stopped. The second is saying "we validate the JSON" and having no answer for the cited clause that does not exist in the retrieved set.
Check yourself
1. A feature returns 400-word summaries. What cap, what timeout, and what alarm?
~550 tokens expected; cap at 800. Timeout: 800 ÷ the decode speed, plus a few seconds of prefill — around 20 seconds, not the minute and a half a 4,000 cap would need. Alarm when the stop reason is output cap on more than 1% of calls: either the prompt started producing longer replies or the inputs changed shape.
2. 2% of answers cite a clause number that is not in the retrieved set. Which check, and what then?
The provenance check: the set of clause ids you sent, membership test per citation. On failure, one retry with "cite only from the clauses provided" appended; on a second failure, return the answer with the citation removed and flagged, or route it, depending on what the feature promises. The 2% becomes a tracked rate with an alarm at 1%, because a rising rate means the retrieval or the prompt changed.
3. A loop's response has stop reason tool call and a text block reading "Here is the answer: the total is 1,240." Is that the answer?
No. The model is mid-turn; the text is a preamble it wrote before deciding to call a tool, and the number in it has no tool result behind it. Run the tool, return the result, and take the answer from the end-of-turn reply that follows. Reading the preamble as the answer is the
content[0].textbug in a second costume.