Cruxly

Cruxly › AI engineering › Agentic Systems

Designing Tools an Agent Can Actually Use

a tool is an interface whose user can't ask you what you meant

Open the interactive lesson

Prompt: a team gives its calendar agent one tool per REST endpoint — list_users, list_events, get_event, create_event, update_event — each returning the raw JSON the API returns. The agent books meetings at the wrong time, runs out of context on busy calendars, and sometimes calls the same endpoint thirty times in a row. Redesign the tool layer.

30 minutes first. The naive fix is a better system prompt. The actual problem is that the tools were designed for a programmer reading documentation, and the user here is a model reading a tool description once, mid-task, with limited context.

Fewer, bigger tools — not one per endpoint

list_users, list_events, get_event, create_event, update_event
→ the agent must chain 3-4 calls, and hold every intermediate result in context
calendar_find_slot(attendees, duration, window)
calendar_book(slot, title, attendees)
→ two task-shaped tools; the chaining happens in your code, not in the model's head

Return meaning, and cap the size

What a tool returns goes straight into the model's context, so treat the response as part of the prompt.

:::warn Errors are instructions too "Error 400" teaches the model nothing. "date must be YYYY-MM-DD, got '7 Oct'" lets it fix the call on the next try. Write tool errors for the reader who has to recover from them. :::

Separate tools that read from tools that act. A search can be retried freely; calendar_book cannot. For any tool that changes state:

The description is a prompt

Dated note (Anthropic tool-use docs, read 2026-10-22): forced tool choice is not universal. On Claude Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1, tool_choice of any or tool returns a 400 error, and manual extended thinking also rejects it. The documented pattern is auto plus strict tool use (so inputs are schema-valid) plus an instruction in the prompt naming the tool to call. Check your model's current restriction before designing around a forced call.

Evaluate tools the way you test code

You can't tell whether a tool design is good by reading it. Measure it on tasks:

@@ 1 | Write realistic tasks Use real requests that need several steps, with a known correct outcome — "move Tuesday's design review to the first free slot on Thursday with the same attendees."

@@ 2 | Run the agent and record more than pass/fail Track task success, number of tool calls, tokens used, and tool errors. A design that succeeds but needs thirty calls is a cost and latency problem waiting to happen.

@@ 3 | Read the transcripts The numbers show that something is wrong; the transcript shows why — a description the model misread, a field it kept asking for, a result too long to use.

@@ 4 | Change one thing, keep a held-out set Improve descriptions or merge tools, then re-run on tasks you didn't tune on, so you know the gain is real and not overfitting to your test set. Anthropic's write-up describes pasting the evaluation transcripts into Claude Code so it can propose improvements to the tools and their descriptions, and relying on held-out test sets to make sure the gains weren't overfitting.

:::key The interview signal Saying "I'd evaluate the tools on held-out tasks and read the transcripts" separates someone who has built an agent from someone who has only prompted one. :::

How to say it in the interview

"An API is designed for a developer who reads the docs once and writes code against it. A tool is read by a model on every call, inside a limited context window, with no chance to ask what a field means. So I would design fewer, higher-level tools, return meaning instead of raw IDs, cap how much each call can return, and then evaluate the tools on real tasks the same way I would test code."

Checklist

Practice this interactively
Share this lessonWhatsAppLinkedInX

Related lessons