Tool calling lets AI models do real work — query databases, call APIs, search files, send messages. But tool calling that works in demos often breaks in production.
The failure modes are predictable: the model calls the wrong tool, sends bad parameters, misinterprets results, or gets stuck in loops. This guide covers the patterns that prevent these failures.
Quick answer
Design tools with clear, specific descriptions and schemas. Validate parameters before execution. Return structured results the model can understand. Handle errors gracefully and set step limits to prevent loops.
- You are building an AI application that needs to interact with external systems.
- Your tool-calling workflow works in testing but fails with real inputs.
- You want to prevent common tool-calling failure modes before they happen.
Why tool calling breaks
The model decides which tool to call based on the tool's description and the conversation context. If the description is vague, the model guesses. If the parameters are ambiguous, the model fills them incorrectly. If the result is unstructured, the model misinterprets it.
Most tool-calling failures are design failures, not model failures. Fix the tool design and the model's behaviour improves.
Designing reliable tools
Each tool should do one thing. A tool that 'searches and filters and sorts' is three tools pretending to be one. Split it.
Write descriptions from the model's perspective. The description should answer: What does this tool do? When should I use it? What will I get back?
- One tool, one action — split compound operations
- Use specific parameter names with type constraints
- Include examples in the description when the use case is ambiguous
- Return structured results with consistent formatting
Parameter validation
The model will sometimes send parameters that are technically valid JSON but semantically wrong — an empty string where a file path is expected, a negative number for a count, or a date in the wrong format.
Validate parameters before executing the tool. Return a clear error message that tells the model what was wrong and how to fix it.
Error handling that helps the model recover
When a tool fails, the error message goes back to the model as context. A good error message helps the model try a different approach. A bad error message ('Error 500') gives the model nothing to work with.
Include what went wrong, why it went wrong, and what the model could try instead.
# Good error response
{"error": "File not found: /data/report.csv",
"suggestion": "Use list_files to find available files in /data/"}
# Bad error response
{"error": "FileNotFoundError"}Preventing tool-calling loops
Without limits, a confused model can call the same tool repeatedly with slight parameter variations, burning tokens and time. Set a maximum number of tool calls per turn (5-10 is usually enough).
If the model hits the limit, return a message explaining what happened and suggesting a more specific question.
Testing tool-calling workflows
Test with realistic inputs, not just happy-path examples. The edge cases that break tool calling are: ambiguous queries, missing data, tools that return empty results, and multi-step tasks where early steps return unexpected results.
Log every tool call in development. The sequence of calls tells you exactly where the model's reasoning went wrong.
Worked example: customer support tool chain
You build a support agent with tools for searching knowledge base articles, looking up customer accounts, and creating support tickets. Each tool has specific parameter validation, clear error messages, and structured return formats. The agent reliably handles queries like 'Find the article about password resets and create a ticket for customer #1234.'
Common mistakes
- Vague tool descriptions that make the model guess.
- No parameter validation — the tool crashes on bad inputs.
- Unstructured error messages that give the model nothing to work with.
When to use something else
If you need to set up MCP servers for tool delivery, see MCP servers for AI agents. For structured outputs without tool calling, see structured JSON outputs.
Frequently asked questions
Why does tool calling break in practice?
The model chooses tools from their descriptions and fills parameters from context. Vague descriptions make it guess the tool, ambiguous schemas make it fill parameters wrong, and unstructured results make it misread the output.
How should I design a tool?
One job per tool. A tool that searches and filters and sorts is three tools wearing a trenchcoat: split it, and give each a precise description and a tight schema.
Why validate parameters if the JSON is valid?
Because valid JSON can still be semantically wrong: an empty string for a file path, a negative count, a mis-formatted date. Validate before executing and return a clear error so the model can correct itself.
How do I stop infinite tool loops?
Set a step or iteration limit, detect repeated identical calls, and return structured errors instead of silent failures so the model changes course rather than retrying forever.
What should a tool return to the model?
Structured, minimal results rather than raw dumps, with a clear success or error shape. The model acts on what it can parse, and noise or ambiguity causes misinterpretation.
How do I handle a tool that fails?
Return an explicit, structured error the model can reason about (such as 'file not found, check the path'), not a raw exception or an empty result. Graceful, descriptive failures keep the workflow recoverable.
Related guides on this site
These guides cover related tool delivery, structured output, and agent patterns.