LLMAGENT · TOOL CALLS · HUMAN APPROVAL
LlmAgent release gate: approve a high-impact tool action
An agent proposes a lookup, a refund and an email, plus one injected refund; choose which calls run alone and which wait for a person.
Your experiment
The user asks for a 40.00 refund on order 7731 and a confirmation email. A support note retrieved with the order asks for a 500.00 goodwill refund to another account. Start with every call auto-run, then require approval above a threshold and for every write, move the threshold past 500 and let the approver rubber-stamp.
Play replays the computed run over sixty inspection seconds; Next event steps, Restart rewinds, Speed changes the pace and Inspection time scrubs. Any input change recomputes the run and rewinds it without autoplay.
Event trace
Computed data
Final metrics
Teaching model, not an ADK LlmAgent or a payment system. The model is assumed to follow the injected note; the careful approver rejects a refund whose destination is not on the order and approves the rest; amounts are fixed. Real approval flows also need timeouts, audit of who approved, and binding the approval to the exact arguments.
One request, four proposed calls
The user writes: order 7731 arrived broken, refund the 40.00 and email me. The agent looks up the order and retrieves its support notes; one note contains text written as an instruction to refund 500.00 to account ACCT-9 as goodwill, and the model adds that as a fourth call. With Approval policy set to Run every proposed call, all four execute: the lookup, the 40.00 refund, the injected 500.00 refund and the email. The user got what they asked for and the business lost 500.00: UNAUTHORIZED EFFECT with 0 approvals asked.
Approve above a threshold
Choose Approve large refunds with the threshold at 100.00. Reads, the email and the 40.00 refund run automatically; the 500.00 refund waits for a person, who sees that ACCT-9 is not on the order and rejects it: APPROVAL BOUNDARY HELD with 1 approval. Raise the threshold to 500.00 or above and the injected refund no longer exceeds it, so it runs unattended. Refund threshold needing approval applies only to this policy, and a threshold is only as good as its distance from the amounts an attacker would choose.
Approve every write
Approve every write asks for 3 approvals: both refunds and the email. The careful approver passes the two legitimate writes and rejects the injected one, so all three requested tasks are done and nothing unauthorized happens. The cost is three interruptions for one request. Clear A retrieved support note carries an injected refund and even Run every proposed call is safe, which is why a clean demo proves little about the policy.
A human who approves everything
Approver approves whatever is shown is tested under Approve every write. The approval requests still happen, 3 of them, but every one is granted and the 500.00 refund executes: UNAUTHORIZED EFFECT. An approval step protects only if the reviewer sees the exact arguments and has time and reason to say no; frequent, low-information requests train people to click approve.
Reading the run and its limits
Play, Next event, Restart, Speed and Inspection time step through the request, the retrieval, the policy and every proposed call across sixty inspection seconds; the table shows each call with its kind, whether it needed a human, and its origin. The model assumes the injected note always becomes a tool call and that the careful approver always spots the wrong destination. Real deployments should also bind approval to the exact arguments, log the approver and expire stale requests.