Catching tool errors at the right layer
The mental model is the easy part - handle each kind of tool error where it can actually be fixed. The build is where the detail lives: telling failures apart when you can't trust their wording, retrying in the harness, and stopping a model that won't stop itself.
The first piece laid out the mental model: what’s worked for me is every tool failure is classified into one of four kinds, and each gets handled at the layer that can act on it. A fresh transient error the harness retries. Bugs and not-found errors go to the model, which can fix an argument or look elsewhere. Permission, and a transient that has run out of retries, end the run. The rule underneath all of it: only raise an error to the model if the model can actually reason about it.
That’s the what. This is the how. And it starts with the question I skated over: when an exception lands in your lap, how do you even know which of the four kinds you’re holding?
Classify by what the error is, not what it says
Back in that first version I classified by running a pile of regexes over the error message string. It worked in the demo and bit me in production. The word “access” in my permission pattern matched cannot access host: connection timed out, so a transient network error got filed as a permission problem and never retried.
The error string is written by whatever service threw it, and it will use whatever words it likes. You have no contract over it. What you do have a contract over is the exception type the language gives you, and the HTTP status code a well-behaved client hangs off the exception. Those are facts about what happened, not prose about it. Classify by what the error is, not by what its message happens to say. Look at the structured signals first, and fall back to string matching only when there’s genuinely nothing structured to read.
"""
Tool error handling for an agent loop, in four layers:
1. Harness retry - transient errors retried in code with backoff. The model never sees them.
2. Classification - structured signals first (exception type, status code), regex last resort.
3. Surfacing - model-fixable errors returned as is_error tool_results: short, actionable, sanitized.
4. Circuit breaker - a per-tool consecutive-failure counter in the loop. Suggestions don't stop loops; counters do.
Plus a terminal path: permission and exhausted-transient errors stop the run in code.
"""
import json
import random
import re
import time
# Structured signals first; regex is the last resort.
TRANSIENT_STATUS = {408, 429, 500, 502, 503, 504}
PERMISSION_STATUS = {401, 403}
NOT_FOUND_STATUS = {404, 410}
# Anchored so a stray word can't misclassify ("cannot access host: timed out"
# must not read as permission).
TRANSIENT_RE = re.compile(r"\b(timed? ?out|connection (reset|refused|aborted)|rate.?limit)\b", re.I)
PERMISSION_RE = re.compile(r"\b(permission denied|access denied|forbidden|unauthorized)\b", re.I)
NOT_FOUND_RE = re.compile(r"\b(not found|does not exist)\b", re.I)
def _status_code(exc: Exception) -> int | None:
"""Pull an HTTP status off common exception shapes (httpx, requests, SDKs)."""
for attr in ("status_code", "status", "code"):
v = getattr(exc, attr, None) or getattr(getattr(exc, "response", None), attr, None)
if isinstance(v, int):
return v
return None
def _retry_after(exc: Exception) -> float | None:
"""Honour the server's Retry-After header instead of guessing."""
resp = getattr(exc, "response", None)
headers = getattr(resp, "headers", None) or {}
try:
return float(headers.get("retry-after") or headers.get("Retry-After"))
except (TypeError, ValueError):
return None
def categorise(exc: Exception) -> str:
# 1. Exception type, the strongest signal.
if isinstance(exc, (TimeoutError, ConnectionError)):
return "transient"
if isinstance(exc, (FileNotFoundError, KeyError, LookupError)):
return "not_found"
if isinstance(exc, PermissionError):
return "permission"
# Code bugs are never transient and never the model's fault, so don't retry
# them and don't let regex on their message misclassify them.
if isinstance(exc, (ValueError, TypeError, AttributeError)):
return "bug"
# 2. HTTP status code, if the exception carries one.
status = _status_code(exc)
if status in TRANSIENT_STATUS:
return "transient"
if status in PERMISSION_STATUS:
return "permission"
if status in NOT_FOUND_STATUS:
return "not_found"
# 3. Message regex, last resort only.
msg = str(exc)
if TRANSIENT_RE.search(msg):
return "transient"
if PERMISSION_RE.search(msg):
return "permission"
if NOT_FOUND_RE.search(msg):
return "not_found"
return "unknown"The message is a string an external service chose to send you. The exception type is a fact about what actually happened. When the two disagree, trust the fact.
Retry transient errors in the harness, never in the prompt
Transient errors never reach the model, so the harness has to deal with them on its own. The mechanics are old and boring and exactly right. Retry the same call a few times. Back off exponentially between attempts so you’re not hammering a service that’s already struggling. Add jitter so that a hundred agents which all failed at the same instant don’t then all retry in the same instant. And if the failure came with a Retry-After header, honour it instead of guessing. Three attempts is a sensible default.
def execute_tool(fn, *args, max_attempts: int = 3, base_delay: float = 0.5, **kwargs):
"""Run a tool, retrying transient failures with exponential backoff + jitter.
Re-raises a non-transient error immediately, and re-raises the transient one
once attempts are exhausted, at which point the caller treats it as terminal."""
for attempt in range(1, max_attempts + 1):
try:
return fn(*args, **kwargs)
except Exception as exc:
if categorise(exc) != "transient" or attempt == max_attempts:
raise
delay = _retry_after(exc) or (base_delay * 2 ** (attempt - 1))
time.sleep(delay + random.uniform(0, delay * 0.25))Most of the time that’s the end of the story. The second attempt succeeds and the model never knew anything happened. But when the retries run out and the call is still failing, the meaning changes. It is no longer “should I retry this?”, because the harness has already answered that. It is now a settled fact: the service is down. And a dead service is not something the model can fix by reasoning, so it doesn’t go back to the model as a decision. It graduates into a terminal error, which is the next thing to build.
Stopping a run that can’t continue
Two of the four buckets are terminal: permission, and a transient that has exhausted its retries. In both you know for a fact there is nothing left to try, and the cleanest thing is not to involve the model at all.
You’re already holding the error at the moment you catch it, before it ever goes back to the model. So that’s where you stop it. Raise an exception straight out of the tool loop, catch it at the top, and the model never sees the error or gets the chance to spin on it.
class ToolRunTerminated(RuntimeError):
"""A failure with nothing left to try: permission, or a transient that has
used up its retries. Raised straight out of the tool loop so the model never
sees it; the loop catches it and tells the user."""
def __init__(self, category: str, detail: str):
self.category = category
self.detail = detail
super().__init__(f"{category}: {detail}")
TERMINAL = {"permission", "transient"} # a "transient" reaching the caller means retries were exhausted
USER_MESSAGE = {
"permission": "I couldn't finish that. Access was denied, so the credentials probably need checking.",
"transient": "I couldn't finish that. The service stayed unavailable after several retries.",
}
# At the top of the agent loop:
# try:
# run_agent_turn(...)
# except ToolRunTerminated as stop:
# reply_to_user(USER_MESSAGE[stop.category]) # a plain, fixed message, no extra model turnWhat the user gets is up to you, and for a hard terminal error the simple option is usually the right one: catch the exception and return a plain message saying what happened (“I couldn’t access your calendar, the credentials look to have expired”). No extra model turn, no chance of another loop. If you want a warmer, written explanation you can hand that final phrasing to the model instead, but a fixed message is the safer default once the run is already over.
That handles the errors you can call terminal the moment you see them. But most stuck states aren’t that clean. A bug the model “fixes” with an equally broken argument, a not-found it keeps re-querying with the same wrong path, a tool it simply likes calling, none of these announce themselves as terminal. They only reveal themselves by repeating. You can’t hand-write a special case for every one, and you wouldn’t want to. What you need is one general mechanism that doesn’t care which tool or which category, and doesn’t depend on the model agreeing to stop.
The circuit breaker: stopping that doesn’t need the model’s permission
A circuit breaker in your house watches how much current is being drawn, and when something keeps pulling too much, it cuts the circuit itself. It doesn’t ask the faulty appliance to please stop, and it doesn’t trust the appliance’s opinion about whether it’s on fire. It just breaks the connection. The point of the thing is that the safety mechanism doesn’t depend on the part it’s protecting you from behaving well.
That’s exactly the gap we’re left with. A surfaced error is advice, and the model can ignore advice, so you need something that isn’t advice. In an agent loop the faulty appliance is the model and the current is failed tool calls, and the breaker itself is almost embarrassingly simple: a counter.
class CircuitBreaker:
"""Per-tool consecutive-failure counter. Resets on success, trips at the threshold.
A suggestion string asks the model not to loop; this guarantees it."""
def __init__(self, threshold: int = 3):
self.threshold = threshold
self.failures: dict[str, int] = {}
def record(self, tool_name: str, ok: bool) -> bool:
"""Returns True once the breaker has tripped for this tool."""
if ok:
self.failures[tool_name] = 0
return False
self.failures[tool_name] = self.failures.get(tool_name, 0) + 1
return self.failures[tool_name] >= self.thresholdThree things make that little counter work, and none of them are obvious until you’ve watched a loop spin without one.
It counts consecutive failures, per tool, and resets on success. Not total failures. If a search tool fails twice and then works, the count drops back to zero, because a success means the model found its way out and the earlier stumbles were just normal fumbling. The breaker only trips when one tool fails several times in a row, which is the signature of a model that’s stuck rather than one that’s exploring. Three in a row is the threshold most people settle on, and it’s the one the 12-factor agents writeup uses: a per-tool consecutive-error count, reset on success, break and escalate when it crosses the line (Factor 9).
The counter lives in your loop code, not in the context window. This is the part that actually matters. Everything you put in a tool result is something the model can read and talk itself out of. The counter is not in the tool result. It’s checked in your own code before the next call to the model even happens, so when it trips, there is no next attempt, no matter what the model would have chosen. That’s the whole difference between asking and enforcing. The suggestion field asks the model to stop. The circuit breaker makes stopping not depend on the model agreeing.
Tripping doesn’t mean crashing, it means stopping on purpose. When the breaker trips you end the run the same way you would for a terminal error: stop in code and return a message to the user, or escalate to a person. The wording of that message isn’t the point. The point is that the decision to stop was made by your counter, not left to the model, which by now has already proven it won’t stop on its own.
I learned the worth of this the way most people do, by not having one. A tool in one of my early agents kept failing on a call the model was convinced was fine. It tried again, same failure, again, same failure. The logs filled with the identical line, the cost counter kept ticking, and the run would have gone until it hit the context limit if I hadn’t killed it by hand. Nothing in the system had the authority to stop it, so it didn’t.
One question always comes up here: how long does the counter live? Keep it to one run. You create the breaker when the loop starts, let it accumulate across every turn of that task, and throw it away when the task ends. The thing you’re guarding against is this model, stuck on this task, right now. When the next task starts it deserves a clean slate, because it might call the very same tool with completely different and perfectly valid arguments. Three lines of bookkeeping, sitting outside the model’s reach, is the entire defence, and it’s the one piece of this whole system the model cannot talk its way around.
Putting the layers together
This is the whole thing assembled, and it’s roughly what I reach for by default now whenever I build an agent that calls tools. Not because it’s the one true way, but because it’s the version that covered most tool errors and stopped surprising me in production. Four layers, one function: handle_tool_call is where they meet. It runs the tool with retries folded in, records the outcome on the breaker, and routes whatever comes back. Terminal failures raise straight out of the loop. The breaker gets a vote before anything reaches the model. And only the errors the model can act on, bugs and not-found, come back as is_error results carrying a short, sanitized suggestion.
# Surfacing: short, actionable, sanitized. Only model-fixable categories get here.
SUGGESTIONS = {
"not_found": "The resource does not exist. Don't retry with the same arguments. "
"Check the identifier for typos, or use a search/list tool to find the right one.",
"bug": "The tool failed on these arguments. Check them against the tool's schema "
"and try once with corrected arguments.",
"unknown": "Unexpected error. Don't retry with identical arguments. Try a different "
"approach or tool, or tell the user an unexpected error occurred.",
}
MAX_ERROR_CHARS = 300 # external error strings are untrusted input: truncate, never include tracebacks
def wrap_tool_error(tool_use_id: str, exc: Exception) -> dict:
category = categorise(exc)
return {
"type": "tool_result",
"tool_use_id": tool_use_id,
"is_error": True,
"content": json.dumps({
"category": category,
"message": str(exc)[:MAX_ERROR_CHARS],
"suggestion": SUGGESTIONS.get(category, SUGGESTIONS["unknown"]),
}),
}
def handle_tool_call(tool_use, registry, breaker: CircuitBreaker) -> dict | None:
"""Returns a tool_result for the model, or None to tell the loop to stop and
answer. Raises ToolRunTerminated for terminal failures."""
fn = registry[tool_use.name]
try:
result = execute_tool(fn, **tool_use.input) # Layer 1: retries transient inside
breaker.record(tool_use.name, ok=True)
return {"type": "tool_result", "tool_use_id": tool_use.id, "content": result}
except Exception as exc:
category = categorise(exc) # Layer 2
if category in TERMINAL: # terminal: stop in code, model never sees it
raise ToolRunTerminated(category, str(exc)[:MAX_ERROR_CHARS])
if breaker.record(tool_use.name, ok=False): # Layer 4
return None # caller forces a final answer / escalates
return wrap_tool_error(tool_use.id, exc) # Layer 3: bug / not_found / unknown -> modelRead the except block top to bottom and it’s the whole piece in one place: name the failure, stop if there’s nothing left to try, stop if you’re looping, otherwise hand it back with advice.
One level further out is the agent loop itself, and it’s worth seeing because it’s what gives the handler’s two stop-signals their meaning. A None coming back means the breaker tripped, so the loop stops and answers with what it has. A ToolRunTerminated raised through it is a terminal failure, caught once at the top. And the breaker is created fresh at the start of each run, which is what “keep it to one run” actually looks like in code.
def run_agent(user_msg, registry, client, max_turns: int = 20):
breaker = CircuitBreaker() # fresh per run
messages = [{"role": "user", "content": user_msg}]
try:
for _ in range(max_turns):
reply = client.create(messages=messages, tools=registry.specs)
messages.append({"role": "assistant", "content": reply.content})
tool_calls = [b for b in reply.content if b.type == "tool_use"]
if not tool_calls:
return reply.text # model gave its answer, we're done
results = []
for tool_use in tool_calls:
result = handle_tool_call(tool_use, registry, breaker)
if result is None: # Layer 4 tripped: stop looping
return "I kept hitting the same failure and stopped. Here's what I have."
results.append(result)
messages.append({"role": "user", "content": results})
return "I ran out of turns before finishing." # turn cap, another deterministic stop
except ToolRunTerminated as stop: # terminal failure: tell the user, end the run
return USER_MESSAGE[stop.category]It’s a completely ordinary ReAct loop. The only thing the error handling adds is that every way a tool can fail now has exactly one place it’s dealt with, and three of the four ways out of the loop, the breaker, the terminal stop, and the turn cap, are decided in your code rather than left to the model.
That’s the recovery layer I keep coming back to. It isn’t the only way to slice it, and the thresholds, the messages, the exact categories are all mine to tune and yours to change, but with something like this in place a tool failing becomes a normal event the loop absorbs and moves past, rather than the thing that quietly ends your run at 3am.