Error handling for tool calls

Tool calls are not guaranteed to work

Everything up to this lesson has assumed a tool call goes through cleanly: the model asks, your code runs the function, the result comes back, the model continues. Real tools don’t cooperate that reliably. A flight-search API can time out under load. A weather service can hit its rate limit and reject your request. A database query can return a result in a shape your code didn’t expect. None of these are exotic edge cases — at any real volume of calls, they are the normal background rate of things going wrong, and an agent that has no plan for them either crashes outright or, worse, proceeds as if a failed call had succeeded and gives the user a confidently wrong answer built on bad or missing data.

Not every failure is the same failure

Lumping every error into one generic try/catch that just retries is a common shortcut, and it hides important differences. A timeout usually means the service is slow or temporarily overloaded — worth retrying, probably after a short wait. A rate limit means the service is telling you explicitly to slow down — retrying immediately only makes it angrier; backing off is not optional here, it’s the documented contract. A malformed or unexpected response — a field missing, a type that doesn’t match what you expected — often means retrying the exact same call will fail the exact same way, so the useful response is to log it and stop, not to hammer it. And a result that parses perfectly but is simply wrong — a tool returning a plausible but incorrect flight price due to a bug on its end — is the hardest case, because nothing about the shape of the response tells you anything failed at all; catching this one usually depends on validation logic specific to the tool, not on generic error handling.

A worked scenario: the flaky API

Picture an agent that calls a flight-pricing API as part of answering “what’s the cheapest flight to Lisbon next week?” The first call times out — the pricing service is momentarily overloaded. A well-built agent catches this specific failure, waits a short interval, say one second, and retries. The retry succeeds, returns pricing data, and the agent answers the user normally. From the user’s side, nothing unusual happened; the delay was perhaps a second longer than usual. The entire failure and recovery happened invisibly, exactly because the agent treated a timeout as something worth a bounded retry rather than either crashing immediately or looping on it forever.

Now contrast that with a naive agent that has no backoff and no retry limit. The first call times out, so it retries immediately — but the service is still overloaded, so the second call also fails, immediately, and so does the third, and the agent keeps retrying as fast as it can, as often as it can, potentially for the full length of the conversation. Each attempt still costs an API call and real wall-clock time, so the user is left waiting indefinitely while the agent burns cost against a tool that was never going to succeed on attempt four if it didn’t succeed on attempt one through three. This is the exact failure mode that a backoff schedule and a hard retry ceiling exist to prevent.

Backoff and ceilings, concretely

A reasonable retry policy for a transient failure like a timeout looks something like: wait one second before the first retry, two seconds before the second, four seconds before the third, and give up after three attempts total — doubling the wait each time, which is what “exponential” backoff means in practice. That ceiling matters as much as the backoff itself. Without it, a permanently broken tool — a dead endpoint, an expired API key, a service that’s been shut down — produces the same symptom as a momentarily slow one, and the agent has no way to tell the difference except by eventually giving up. Three attempts with growing delays between them is usually enough to ride out a genuine transient blip while staying cheap and fast when the failure turns out to be permanent.

What happens after you give up

Giving up on a tool shouldn’t mean crashing the agent process. The better pattern is to treat the failure itself as information and hand it back to the model: tell it, in effect, “the flight-pricing tool failed after three attempts,” and let the model decide what to do next within the conversation — try a different tool if one exists, narrow the request and try again, or tell the user directly that it couldn’t get pricing right now rather than making something up. This keeps the failure contained and visible instead of silent, which is the whole point of treating error handling as a first-class part of tool use rather than an afterthought bolted on once something breaks in production.