Catching tool errors at the right layer

The first piece laid out the mental model: every tool failure gets handled at the layer that can act on it, and there are only three outcomes. The harness retries what a retry can fix. The model sees only the errors its reasoning can move, the bugs and the not-founds. The failures nothing can fix, permission and a transient that has used up its retries, end the run.

That model hides the load-bearing decision. Before anything can be retried, surfaced, or stopped, something has to decide which kind of failure it’s holding. When building out any agent harness, there were two candidate homes for that decision, and they produce systems that fail in very different ways. This is the one I picked, why I picked it, and what it cost. The full runnable implementation - real search tool, harness, breaker and all - lives in the companion repo; the listings below are slices of it.

The design question: who names the failure?

The first candidate is a central classifier. Tools run naked and throw whatever their libraries throw - a raw Tavily rate-limit error, a ValueError, an httpx timeout - and the harness inspects whatever escapes: check the exception type, pull a status code off it if one exists, and as a last resort match patterns against the message string. All the intelligence lives in the centre; the tools stay dumb.

The second inverts it. Each tool catches its own native exceptions at the boundary, while it still has full context on what they mean, and re-raises them as one of a small set of harness types. By the time an error leaves the tool it’s already classified. The harness just dispatches on type; the tools carry the intelligence.

The central version was the tempting one, because it promises that a new tool needs no error-handling code at all. But price out what it actually demands. The harness has to recognise every error shape any tool could ever produce, for services it knows nothing about. It has to infer meaning from clues, and the clues degrade as you go down the stack: exception types don’t cover HTTP failures, status codes aren’t always attached, and the message string is prose a vendor chose, with no contract behind it. String matching is the sharpest hazard - a permission pattern containing “access” will happily match cannot access host: connection timed out, and now a transient network blip is filed as a credentials problem and never retried. A provider rewording an error message can silently break classification in production without a single exception raised.

What settled it for me wasn’t the fragility, though. It was noticing that the central classifier does detective work on a question that was never in doubt. Whoever wrote a web_search tool knows exactly what a Tavily 429 means. A central classifier is an inference engine built to recover knowledge the system already had, three layers away from where it lives. So classification went where the knowledge is. The tool declares; the harness never guesses.

Three exceptions are the whole vocabulary

If tools declare their failures, they need a language to declare them in, and my first cut of that language enumerated failure kinds: transient, permission, not-found, bad-input. It read sensibly and it was oversized, because the harness never branched on most of the distinctions. The number that matters isn’t how many ways a tool can fail. It’s how many things a harness can do about a failed call: retry it itself, hand it to the model as an observation, or end the run. There is no fourth thing. Any taxonomy finer than the actions it maps to is decoration, categories you maintain without ever branching on them. So the whole language is three exception types:

class Retryable(Exception):
    """The world blinked. The harness retries; the model never sees it."""

    def __init__(self, message, retry_after=None):
        super().__init__(message)
        self.retry_after = retry_after  # seconds, if the API told us


class ToolInputError(Exception):
    """The model can fix this: bad argument, missing resource. Returned as is_error."""


class FatalToolError(Exception):
    """Nothing left to try. Ends the run."""

Each tool owns a translation block: catch native errors, raise harness vocabulary. For a simple web search tool it’s a few lines sitting right next to the call that produces the errors:

async def web_search(query: str) -> dict:
    try:
        resp = await tavily_client.search(
            query=query,
            include_answer="basic",
            search_depth="advanced",
            max_results=5,
        )
    except UsageLimitExceededError as exc:                   # rate / quota -> retry
        raise Retryable(f"tavily usage limit: {exc}") from exc
    except (InvalidAPIKeyError, MissingAPIKeyError) as exc:  # misconfig -> fatal
        raise FatalToolError(f"tavily auth: {exc}") from exc
    # ...result shaping omitted — full tool in the repo

A few lines, and they’re statements, not inferences. I’m not hoping a status table somewhere agrees with me. I’m recording a fact I already knew when I wrote the call, in the one place it can’t drift away from the code it describes.

The other tool in my registry is a compound-interest calculator, and it’s what convinced me the vocabulary was right. Pure local maths: it cannot rate-limit, has no HTTP status, and will never produce a Retryable. Its only realistic failures are bad inputs, so its entire error handling is raising ToolInputError from its validation checks. Under the central design it would have been paying for retry machinery, status tables, and header parsing it cannot physically use. Under this one, each tool carries exactly the handling its failure modes require, and nothing else.

The harness gets dumb, and that’s the point

With classification pushed to the edge, the harness collapsed into two small functions, and neither of them inspects anything. The retry loop catches the one type that asks to be retried:

async def execute_tool(fn, tool_input, max_attempts=3, base_delay=0.5):
    """Run a tool, retrying ONLY Retryable failures with jittered backoff.

    ToolInputError / FatalToolError propagate untouched. Exhausting retries is
    treated as fatal here — a deliberate choice. To degrade instead (e.g. let the
    model carry on without search results), catch FatalToolError per-tool in
    handle_tool_call and return an is_error result rather than re-raising.
    """
    for attempt in range(1, max_attempts + 1):
        try:
            return await fn(**tool_input)
        except Retryable as exc:
            if attempt == max_attempts:
                raise FatalToolError(
                    f"{fn.__name__} still failing after {max_attempts} attempts: {exc}"
                ) from exc
            delay = (
                exc.retry_after
                if exc.retry_after is not None
                else base_delay * 2 ** (attempt - 1)
            )
            delay += random.uniform(0, delay * 0.25)
            print(f"[RETRY] {fn.__name__} attempt {attempt}/{max_attempts} failed ({exc}); sleeping {delay:.1f}s")
            await asyncio.sleep(delay)

The mechanics are old and boring and exactly right. Back off exponentially so you’re not hammering a service that’s already struggling. Add jitter so a hundred agents that failed in the same instant don’t all retry in the same instant. And the server’s Retry-After gets honoured when one was given - it arrives as a field the tool set on the exception, not something the harness sniffs off a response object it hopes exists.

When the attempts run out, my harness raises FatalToolError and the run ends. I made that choice knowing the alternative: for a search tool, degrading is often the better experience - hand the model an is_error result saying “search is unavailable, answer from what you know” and let the run limp home. I kept abort because I’d rather a run die loudly than answer thinly, but I left the seam visible: the swap is two lines in handle_tool_call. The thing I was determined to avoid was inheriting the choice from the code layout without noticing I’d made one.

Dispatch, one level up, is just the try/except structure itself:

MAX_ERROR_CHARS = 300  # tool error text is untrusted input: truncate, no tracebacks

# ...

async def handle_tool_call(tool_use, registry, breaker):
    fn = registry.fns.get(tool_use.name)
    if fn is None:  # model invented a tool name — recoverable and able to be reasoned upon, inform model
        return _error_result(tool_use.id, f"Unknown tool: {tool_use.name}")
    try:
        result = await execute_tool(fn, tool_use.input)
        breaker.record(tool_use.name, ok=True)
        return {
            "type": "tool_result",
            "tool_use_id": tool_use.id,
            "content": json.dumps(result),
        }
    except ToolInputError as exc:
        if breaker.record(tool_use.name, ok=False):
            print(f"[BREAKER] {tool_use.name} tripped after {breaker.threshold} consecutive failures")
            raise FatalToolError(
                f"{tool_use.name} failed {breaker.threshold} times in a row — breaking the loop"
            ) from exc
        return _error_result(tool_use.id, str(exc))
    except FatalToolError:
        raise  # abort the run — caught in run_agent
    except Exception as exc:  # untranslated: degrade safely, don't crash, don't retry
        if breaker.record(tool_use.name, ok=False):
            print(f"[BREAKER] {tool_use.name} tripped after {breaker.threshold} consecutive failures")
            raise FatalToolError(
                f"{tool_use.name} failed {breaker.threshold} times in a row — breaking the loop"
            ) from exc
        return _error_result(tool_use.id, f"Unexpected error in {tool_use.name}: {exc}")

The harness file now imports no httpx, no tavily, no regex module, no status-code tables. It cannot be broken by a vendor rewording an error message, because it never reads one. The catch-all at the bottom is what makes the whole convention survivable: when a translation block gets forgotten (usually by me), the untranslated exception goes back to the model as an ordinary error result, no retry, no crash. That default is also what let me move tools onto the new vocabulary one at a time instead of in one big migration. And the truncation stays in any design, because an external service’s error string is untrusted input wherever you classify it.

The circuit breaker doesn’t care who classified what

Everything above handles a single call failing. It does nothing about the loop failing: a model that “fixes” a bad argument with an equally broken one, re-queries the same missing path, or keeps reaching for a tool that keeps failing. The retry loop never sees any of this, because each new model attempt is a fresh execute_tool call and the retry counter starts from one every time. Loop failure only reveals itself across calls, and it needs its own guard.

Mine is borrowed from the fuse box. A circuit breaker doesn’t ask the faulty appliance to please stop, and it doesn’t trust the appliance’s opinion about whether it’s on fire. It cuts the circuit itself. In an agent loop the appliance is the model, the current is failed tool calls, and the breaker is embarrassingly simple:

class CircuitBreaker:
    """Per-tool consecutive-failure counter. Resets on success, trips at the threshold."""

    def __init__(self, threshold: int = 3):
        self.threshold = threshold
        self.failures: dict[str, int] = {}

    def record(self, tool_name: str, ok: bool) -> bool:
        if ok:
            self.failures[tool_name] = 0
            return False
        self.failures[tool_name] = self.failures.get(tool_name, 0) + 1
        return self.failures[tool_name] >= self.threshold

It counts consecutive failures per tool and resets on success, because two stumbles followed by a success is a model exploring, while three identical failures in a row is a model stuck. That’s the same shape the 12-factor agents writeup lands on: count consecutive errors, reset on success, break and escalate at the line. The placement is the part I care about most. Everything you put in a tool result is something the model can read and talk itself out of; the counter lives in my harness code and is checked before the next model call ever happens, so when it trips there is no next attempt, whatever the model would have chosen. A suggestion asks the model to stop; the breaker makes stopping not depend on the model agreeing. I learned that distinction by not having one: an early agent of mine retried a call the model was convinced was fine until I killed the process by hand, logs full of the identical line, cost ticking. Nothing in the system had the authority to stop it, so it didn’t.

I scope the breaker to a single run and throw it away after, because the next task deserves a clean slate: it might call the very same tool with perfectly valid arguments. And it never asks which kind of error occurred. It guards the loop, not the call, and it keeps working even when every translation block in the codebase is wrong.

That’s every component in place, and this is the whole system on one page:

The trade-offs I’m accepting

Two costs came with this design, and I’d rather name them than pretend the choice was free.

The first is duplication, and it’s the reason I wouldn’t call the design finished. At two tools, translation at the edge is a few lines each. At ten tools wrapping HTTP APIs it becomes ten near-identical try/except blocks encoding the same status-code knowledge - exactly the sprawl a central classifier exists to prevent.

The second cost is discipline. Edge translation only holds if the translation blocks actually get written, and my catch-all means forgetting one doesn’t fail loudly: the tool mostly works, it just never retries what it should. That’s the honest weakness of what I’ve built, and the only mitigation I’ve found is keeping the vocabulary small enough that translating is easier than skipping it. Three types, five lines per tool, is about as low as I can get that bar.

That’s the recovery layer as I actually built it: classification at the edge because that’s where the knowledge lives, a three-type vocabulary because that’s how many actions exist, a harness that dispatches and never inspects, and one counter with the authority to stop the whole show. Next in the series I’ll take the same lens up a level: not a single call failing, but the whole plan going wrong, and how an agent decides when to abandon an approach instead of a call.