TOOL CALLBACKS · WHAT THE HOOK RETURNS
Tool callbacks: block before execution
Gate six tool calls with a before-tool hook, and see how its return value decides whether the model explains, retries, lies or the whole turn dies.
Your experiment
A support turn requests six calls: a charge lookup, a 40 refund, an order search with page_size 500, an 1,800 refund that needs a manager, an account deletion that is never allowed and a summary email. The before hook already blocks both forbidden calls, but returns a bare “ERROR”. Change what it returns, where it runs and how it treats arguments.
Play replays the computed run over sixty inspection seconds; Next event steps, Restart rewinds, and Inspection time scrubs. Any input change recomputes the run and rewinds it without autoplay.
Event trace
Computed data
Final metrics
Simplified callback model following the linked article: returning None lets the tool run, returning a dict skips it and becomes its result. How the model reacts to each result (explain, retry twice, report success) is a fixed fixture, and an exception is modeled as ending the turn. No model or tool is called.
A hook, its position and its return value
A before-tool hook sees the tool, its arguments and the tool context before anything runs. Returning nothing lets the call proceed; returning a dict skips the tool and hands that dict to the model as the result; changing the arguments in place and returning nothing rewrites the call. Hook position moves the same check after the tool, or removes it. Denial returned by the before hook picks what the model receives for a blocked call. Oversized page_size rewrites, denies or ignores a search for 500 orders. Requester is a manager and the typo option test the approval lookup.
The opening run: blocked, but blind
Both forbidden calls are stopped before they run, so no money moves beyond the 40 refund and no account is deleted. But each denial reaches the model as the bare string “ERROR”. With no reason to act on, the model retries the identical call twice, is denied twice, and tells the user something went wrong: 4 wasted retries and no explanation. The search is clamped to 50 rows and answered. Enforcement worked; the conversation did not.
Changing the return value
Set Denial returned by the before hook to {status: denied, reason}: both calls are denied once, the model explains approval_required and offers escalation, and the gate holds. {status: ok}, tool skipped is worse than an error: nothing runs, yet the user is told twice that it is done. Raise an exception and the turn ends at the 1,800 refund with a generic error; the deletion and the summary email are never attempted. Deny the call for page_size leaves the order question unanswered, and Not checked fetches all 500 rows.
Failure boundary: position and the key you read
Set Hook position to After the tool has run: the 1,800 refund and the deletion execute, 1,840 leaves in refunds, and the hook can only replace the result, so the user is told no while the effect stands. No hook gives the same effects. With the legible denial, tick Requester is a manager and the 1,800 refund is allowed. Now tick Hook reads user:isManager (typo): it only matters for a manager, nothing writes that key, so the manager is wrongly denied and nothing raises. A gate must run before the effect and read the key that is actually written.
Reading the run and its limits
Play replays the six calls, each hook decision and the outcome over sixty inspection seconds; the table shows what the hook decided, the effect and what the user gets. Next event, Restart, Speed and Inspection time control the replay. The model’s reaction to each result is a fixed assumption: a real model may retry once or five times, or recover from a bare error. Keep the gate deterministic and make every denial say why and what to do next.