Tool use and the tool-call loop
Run the send-call-execute-return loop by hand, batch parallel calls, return failures as results, and account for every thousand tokens a four-turn task reads.
On this page
A model cannot call anything. It emits text, and one shape of that text is a request: the name of a tool and a JSON object of arguments. Your code runs the tool, puts the result back into the conversation, and sends the whole conversation again. Tool use is that loop and nothing else; everything a vendor markets as an agent is this loop with budgets around it.
The loop has four moves per round trip, and every one of them is yours except the second:
- You send the system prompt, the tool definitions and the messages so far.
- The model answers with either text (stop reason: end of turn) or one or more tool calls (stop reason: tool use).
- You run the calls and append one result block per call.
- You send everything again, including the results.
The model never sees your function. It sees a name, a description and a JSON Schema for the arguments, and it sees whatever string you return. That is the whole interface, which is why the description carries so much weight and why the schema rules from structured outputs apply unchanged.
A tool is a schema plus a description
A definition is three fields: a name, a description and an input_schema in JSON Schema. The schema is the contract the model's arguments are validated against; the description is the only documentation the model gets.
{
"name": "find_free_slots",
"description": "Return free 30-minute slots in the organiser's calendar between two ISO timestamps. Use before proposing a meeting time. Does not check other attendees.",
"input_schema": {
"type": "object",
"properties": {
"from": { "type": "string", "description": "ISO 8601 start, organiser's timezone" },
"to": { "type": "string", "description": "ISO 8601 end" }
},
"required": ["from", "to"],
"additionalProperties": false
}
}
Three tools of that size cost roughly 150 tokens each, 450 in all, and they are sent on every round trip. That is the first thing to know about tool cost: definitions are prefix, resent every turn, and the number of tools multiplies the floor of every call. A tool surface of 40 tools at 150 tokens is 6,000 tokens before the user has typed anything, which is the argument the MCP lesson makes for loading tools on demand.
The worked example: a three-tool calendar assistant
The assistant has find_free_slots, check_availability(attendee, slot) and create_event. The system prompt is 300 tokens, the three definitions 450, and the user asks: "Book 30 minutes with Priya and Tom this week." Call that 50 tokens. The fixed prefix is therefore 800 tokens per turn.
Turn 1. Input 800 tokens. The model returns one call, find_free_slots(from, to), about 60 tokens of output.
Turn 2. We run it and get eight free slots as JSON, around 400 tokens. Input is now 800 + 60 + 400 = 1,260. The model returns two calls in one message: check_availability("priya", slot_3) and check_availability("tom", slot_3), about 120 tokens together.
Turn 3. Both calls are independent, so we run them concurrently and return both results in one message, in the order the calls arrived, each tagged with the id of the call it answers. Tom's calendar service timed out. That result is not an exception in our loop; it is the string error: calendar service timed out after 5 s returned as the result of that call. Say 600 tokens for the two results. Input is 1,260 + 120 + 600 = 1,980. The model could retry Tom or try another slot; here it books slot 3, which Priya has free, and calls create_event, about 80 tokens.
Turn 4. The event id comes back in 60 tokens. Input is 1,980 + 80 + 60 = 2,120. The model writes "Booked Thursday 14:00 with Priya; Tom's calendar was unreachable, so I invited him without checking." Stop reason: end of turn. About 80 tokens.
Where the tokens went:
| Bucket | Tokens read | Share |
|---|---|---|
| Prefix (system + tools + user), sent on all 4 turns | 3,200 | 52% |
| Tool results, each resent on every later turn | 2,460 | 40% |
| The model's own earlier calls, replayed | 500 | 8% |
| Input total | 6,160 | |
| Output (calls + final text) | 340 |
Cost is 6,160 × P_in + 340 × P_out. Two things follow. Input dominates output by nearly twenty to one, so the loop is an input-cost problem, which where the money goes takes further. And half the input is a prefix that never changed, which is exactly what prompt caching charges at C_r instead of full price, provided the definitions are byte-identical on every turn.
Parallel calls arrive in one message
When two calls are independent, a model returns them together, in one message, with one id each. The loop must handle a list, not a single call, and must return one result block per id, all in the next message. Returning the first result and waiting for the model to ask for the second costs an extra round trip of 1,500 tokens or more at this point in the conversation, and the latency of one more prefill and decode.
Run them concurrently when the tools are side-effect free. When they are not, serialise: two create_event calls for the same slot are a double booking if they race. Reads fan out, writes queue.
A failed tool returns an error result, not an exception
The loop's job is to keep the conversation well formed. A tool that raises, times out or returns garbage becomes a result block whose content says so, and the model decides what to do: retry, fall back, or tell the user. If the loop raises instead, the conversation ends with a dangling call and no result, and the next send is rejected as malformed.
The error string is a prompt. "error: calendar service timed out after 5 s" tells the model it can retry. "error: attendee not found" tells it not to. "error" on its own produces a retry loop until the turn cap in agent loops and stop conditions fires. Write errors the way you write descriptions: for a reader who cannot ask what went wrong.
The same discipline covers the arguments. Validate every call against the schema before running anything, and send a validation failure back as the result ("error: to must be ISO 8601, got 'Thursday'"); a tool that ran on a bad argument may have done something a retry cannot undo. A write that reaches a payment or booking system also needs an idempotency key per call id, for the same reason any retried API does; the idempotency lesson has the pattern. The model chose the tool and the arguments; validation, dispatch, concurrency, error formatting and the decision to stop are code, and the harness lessons are about keeping them there.
In an interview
The interviewer is testing whether you can describe the loop as code, with the model as one function in it, rather than as a magic agent. Draw the four moves, say who owns each, and show the token count growing turn by turn. Mention the two details that mark someone who has run this in production: parallel calls come back in one message and go out in one message, and a tool failure is a result the model reads, not an exception the loop throws.
The mistake that loses points is treating output tokens as the cost. A candidate who prices the task at 340 output tokens has missed the 6,160 input tokens, and with them the reason prefix caching and context management exist. The second-worst answer runs the two availability checks one per round trip, doubling the turns for no information.
Check yourself
1. The calendar assistant gains a fourth tool, send_invite_email, with a 200-token definition. By how much does the four-turn task's input grow, and what should you check before adding it?
The prefix rises from 800 to 1,000 tokens, resent on all four turns: +800 input tokens, 13% on a 6,160-token task, before the model ever calls it. Check two things: whether the tool is used often enough to earn its place on every call, and whether its definition is byte-stable so the cache still hits.
2. Tom's calendar times out on turn 3. The loop catches the exception and skips the result block for that call. What happens on the next send?
The conversation now has a call with no result, and the API rejects the message as malformed, so the task ends with no answer. The fix is a result block containing "error: timed out after 5 s"; the model then retries or books without him and says so.
3. The model returns three independent read calls in one message. Serial execution takes 1.2 s each; concurrent takes 1.3 s total. The results are 300 tokens apiece. Which do you choose, and what changes if one of them is a write?
Concurrent: 1.3 s against 3.6 s, and either way all three results go back in one message, so the token cost is identical. If one is a write, run the reads concurrently and the write after them, and give the write an idempotency key, because a retry of the whole turn must not create a second event.