Cruxly › AI engineering › Prompting, Structured Output & Tool Calling

Function Calling Basics: The Loop, Parallel Calls, Errors and Forced Choice

the model asks, your code runs, the history grows: the loop, parallel results, is_error, tool_choice and the token bill

Prompt: "A restaurant assistant has two tools, get_forecast and book_table. A user says: book dinner for four in Pune tonight if it will not rain. Write the loop. What happens when the booking tool times out, how do you let the model call tools in parallel safely, and what does this conversation cost in input tokens?"

40 minutes first. Write the loop with a stop condition, say what goes into each tool result, and do the token arithmetic across the turns. The trap is thinking the model "calls" the function: it only writes a request, and everything after that is your code.

The loop: the model asks, your code runs

The model never executes anything: it emits a structured request, and your program decides whether and how to run it. A tool definition has three parts: a name, a description, and a JSON Schema for the arguments. The description is the main thing the model reads when choosing; Anthropic's documentation calls it the most important factor in tool performance and suggests at least three or four sentences saying what the tool does, when to use it and when not to, and what each parameter means. Design quality is the subject of Designing Tools an Agent Can Actually Use; here we need the mechanics.

The loop, in the Claude API's vocabulary (OpenAI's Responses API has the same shape with different names):

@@ 1 | Send the request with tools and the messages so far The tool definitions are tokens: the tools parameter, plus a special system prompt the API adds, all count as input. Anthropic documents the added prompt at 286 tokens for Claude Sonnet 5.5 and Opus 5.5 with auto choice (retrieved October 2026).

@@ 2 | Read stop_reason tool_use means the model wants one or more tools run; end_turn means it has finished and the last text block is the answer. Other values (max_tokens, refusal, and so on) must be handled, not assumed away.

@@ 3 | Run each requested call in your code Each tool_use block has an id, a name and an input object. Validate input against your schema (do not trust it, even with strict schemas), check that this user may do this, then execute.

@@ 4 | Append two messages and go back to step 1 Append the assistant message exactly as received, then a user message holding a tool_result block per call, each carrying the matching tool_use_id. OpenAI's equivalent is a function_call_output item with the matching call_id.

@@ 5 | Stop on end_turn, or on your cap Always have a maximum iteration count and a wall-clock deadline. A model that keeps calling a failing tool will do so until you stop it; the stop conditions and the planning layer above this loop are in The Agent Loop in Depth: ReAct, Plan-and-Execute, Reflection and Stop Conditions.

messages = [user_msg]
for i in range(MAX_STEPS):                  # e.g. 8
    resp = client.messages.create(model=M, tools=TOOLS, messages=messages)
    messages.append(assistant(resp.content))
    if resp.stop_reason != "tool_use":
        return final_text(resp)
    results = [run_one(b) for b in resp.content if b.type == "tool_use"]
    messages.append(user(results))          # all tool_result blocks, first in the message
raise StepLimit

Strict tool schemas fix the shape of the arguments, not their wisdom. Setting strict: true makes tool inputs match the schema (the contract and its limits are in Structured Outputs: Schemas, Validation Loops and What 'Guaranteed' Means). A strict book_table(restaurant_id, time, party_size) call can still ask for party_size 400 or a restaurant that does not exist. That is why the model does not get your database handle: tools are the narrow, validated, authorised door, and "why not let the model query the database directly?" is answered by everything the door checks, which Guardrails for an Agent That Takes Real Actions develops.

Parallel calls, errors and timeouts: return a result for every call

A single assistant turn can contain several tool requests, and you owe the model one result for each, all in the next message. Anthropic's docs say the API does not prescribe an execution order: run them concurrently, sequentially, or any mix. Whichever you choose, return one tool_result per tool_use, matched by tool_use_id, all together in the next user message, with the tool_result blocks before any text (text first returns a 400). OpenAI's flow likewise ties each function_call_output to a call_id.

Run independent reads in parallel and side-effecting writes one at a time. The latency gain is concrete. Assume the model takes 1.5 s per turn and each tool call 0.4 s. For a forecast lookup and a restaurant-availability lookup that do not depend on each other, sequential turns cost 1.5 + 0.4 + 1.5 + 0.4 + 1.5 = 5.3 s; one parallel turn costs 1.5 + 0.4 + 1.5 = 3.4 s, a saving of 1.9 s (the timings are assumptions for the arithmetic). But a pair of book_table and charge_deposit calls has an ordering, and running them concurrently can charge for a table that was never booked. If you need to prevent parallel calls, Anthropic's tool_choice accepts disable_parallel_tool_use and OpenAI has parallel_tool_calls: false (at most one call per turn); also make each tool safe to retry and consider whether the model is reasoning about dependencies you should instead encode in a single composite tool.

Failures are information; return them, do not hide them and do not drop the call. There are four distinct cases.

The tool raised an error. Return a tool_result with is_error: true and an instructive message: not "failed" but "no table for 4 at 20:00; free slots: 20:30, 21:00". Anthropic's documentation recommends exactly this, so the model can recover without guessing.
The model sent a bad call (missing or invalid argument, unknown tool). Return an error result naming the problem. Anthropic notes Claude will typically retry two or three times with corrections, and that strict tools remove the missing-parameter class entirely. For missing information the better behaviour is to ask the user; Anthropic's docs say Opus is more likely than Sonnet to ask for a missing required parameter, while Sonnet may infer a value, a behaviour they say is not guaranteed. So test your own tools for it.
You skipped a call (the batch ran sequentially and an earlier call failed). Still return a result for it with is_error: true and "Not executed: the preceding call failed", as the docs show. A missing result for a tool_use id is a malformed conversation.
The tool timed out. Set a timeout shorter than your request budget and return an error result that states what is unknown: "book_table timed out after 10 s; the booking may or may not exist; call get_booking with the request key before trying again". For writes, give every call an idempotency key derived from the conversation and call, so a retry of an unfinished booking returns the existing booking instead of creating a second.

tool_choice: auto, none, forced, and what changed on current models

tool_choice decides whether the model may, must, or must not call a tool, and the "must" setting is no longer available everywhere. Anthropic's four settings are auto (the model decides; the default when tools are supplied), any (it must use one of the tools), tool (it must use a named tool) and none (no tools). OpenAI's Responses API has the analogues auto, required, a named function, allowed_tools (restrict to a subset of the tools you passed) and none.

:::key Verified October 2026 Anthropic's "Define tools" page (retrieved October 2026): any and tool return a 400 error on Claude Opus 5.5, Claude Sonnet 5.5, Claude Fable 5.1 and Claude Mythos 5.1; auto and none still work. The page recommends auto with strict tool use to guarantee schema-valid inputs, or structured outputs when you need a response in a fixed JSON shape, and says prompting still influences which tool auto picks. Manual extended thinking (thinking: {type: "enabled"}) also rejects any and tool. On models that accept forced choice, a forced call skips thinking (the response starts with the tool call) and the API prefills the assistant turn, so no explanatory text precedes the call. Changing tool_choice between requests invalidates cached message blocks (tool definitions and system prompt stay cached). OpenAI's function-calling guide (retrieved October 2026) lists auto, required, a forced function, allowed_tools and none. Model support lists change with releases; check the page for the model you actually run. :::

So on current models steer with auto, a strict schema and an instruction, and verify that the call happened. The old reasons to force a tool were (a) getting structured output from a model that would otherwise chat and (b) making sure the lookup happens. For (a), use structured outputs, which is the purpose-built feature. For (b), write the instruction in the system prompt ("Always call get_forecast before answering anything about weather"), which Anthropic's docs describe as a lever of increasing strength ("use the tools to investigate before responding" up to "always call a tool first"), and then check in code: if the answer cites a forecast and no get_forecast result exists in the transcript, reject it. A cheap programmatic post-condition is stronger than any amount of instruction. Test the instruction on your eval set across a few phrasings, since the docs themselves note the behaviour is steerable, not guaranteed.

If you can't change the model and need the lookup to happen, you can also do the call yourself outside the model: when the intent is certain (the user asked for the forecast by name), call the forecast API in your own code and insert the result as context. Deterministic work should be deterministic code.

The token bill: every turn re-sends the history

A tool conversation is billed turn by turn, and each turn re-sends everything so far, so the input you pay for is far larger than the input you wrote. Model the dinner conversation with assumed sizes. The first request holds the system prompt, tool definitions and the user message: 600 tokens. The tool calls are: get_forecast (the result is 250 tokens), then book_table which fails with the free slots listed (350 tokens), then book_table again which succeeds (350 tokens). Each model call that requests a tool emits about 50 tokens. The assistant's final answer is 150 tokens.

Request	History in the input	Input tokens
1	600	600
2	600 + 50 + 250	900
3	900 + 50 + 350	1,300
4	1,300 + 50 + 350	1,700
Total		4,500

Four requests bill 600 + 900 + 1,300 + 1,700 = 4,500 input tokens, 7.5 times the 600 you might expect (4,500 / 600). Output is 50 + 50 + 50 + 150 = 300 tokens. At an assumed $3 per million input and $15 per million output (illustrative prices, not a quote), that is 4,500 x 3 / 1e6 = $0.0135 plus 300 x 15 / 1e6 = $0.0045, so $0.018 per conversation.

Caching changes the arithmetic; result size changes it more. With a cache breakpoint at the end of each request, the next request reads the previous prefix instead of recomputing it. Assume a cache read costs 0.1x the input price and a write 1.25x (assumed multipliers; take the real ones from the provider's pricing page, and see Prompt Caching: Paying for the Same Tokens Once). The billed-token equivalents are: request 1, 600 x 1.25 = 750; request 2, 600 x 0.1 + 300 x 1.25 = 435; request 3, 900 x 0.1 + 400 x 1.25 = 590; request 4, 1,300 x 0.1 + 400 x 1.25 = 630. Total 2,405 token-equivalents against 4,500, which is 53 percent, so input cost drops from $0.0135 to about $0.0072 and the conversation from $0.018 to about $0.0117. The saving is real but it is not the first thing to reach for, because the bigger lever is what you put in the history. The bill is quadratic in turns times result size: ten turns that each add a 2,000-token result send 600 + 2,050k summed over k = 0 to 9, a total of 98,250 input tokens, against 21,750 if each result is 300 tokens. That is 4.5 times more, from the same ten calls.

So shape results before they enter the context. Return the fields the model needs and no more, paginate lists with a stable cursor and a short first page, truncate long text with a marker that says it was truncated and how to get the rest, and return identifiers the model can pass to a follow-up call. Anthropic's tool guidance makes the same recommendation: bloated responses waste context and make it harder for the model to extract what matters. "How many results can I keep?" has no fixed answer: the hard limit is the context window minus the output budget, but cost and answer quality degrade well before it, so keep only the results the next step needs. When a conversation gets long, old results are the first thing to clear or summarise, which Multi-Turn Conversation State and Context-Window Limits covers. And remember that changing tool_choice or the tool list mid-conversation breaks the cached prefix.

How to say it in the interview

"Function calling is a loop that my code owns. I send the tool definitions with the prompt, the model replies with a request to call a tool by name with JSON arguments, I validate and run it, and I send the result back in the next message, and I repeat until the model stops asking, with an iteration cap so it cannot spin. Parallel calls come back together: I run independent reads concurrently, run writes one at a time, and return one result for every call in one message, using an error result for a call that failed or was skipped rather than dropping it. A timeout becomes an error result that says what is unknown, plus an idempotency key so a retried booking cannot book twice. Every turn re-sends the whole history, so on my assumed sizes a four-request conversation bills 4,500 input tokens, not the 600 of the first request, and I cut that with a cached prefix and short, paginated tool results."

Checklist

The loop has a maximum step count and a wall-clock deadline, and logs every stop_reason
Every tool_use id gets exactly one tool_result in the next message, with results before any text
Tool inputs are validated and authorised in your code even when strict schemas are on
Independent reads run in parallel and writes run one at a time
Failures return is_error results with what happened and what to try next
Writes carry an idempotency key and a status-check tool for timeouts
Skipped calls still get an error result such as "Not executed: the preceding call failed"
Forced tool_choice is not assumed to exist; auto + strict + instruction + a code post-condition is the default
Tool results are trimmed, paginated and truncation-marked before entering context
The token bill was computed as the sum over turns, with caching assumptions labelled
Changing tool_choice or the tool list mid-conversation is treated as a cache break

Practice this interactively