RETRIES · SIDE EFFECTS
Retry safety: backoff and side effects
Retry one payment tool call through a lost response, a temporary outage and a permanent error, and count the charges that really happened.
Your experiment
The payment server commits the charge but the response never reaches the agent, so the client times out after 3 seconds and retries. Count the charges, then add an idempotency key, remove the retries, tighten the deadline, and switch to an outage and to a permanent error.
Play replays the computed run over sixty inspection seconds; Next event steps, Restart rewinds, and Inspection time scrubs. Any input change recomputes the run and rewinds it without autoplay.
Event trace
Computed data
Final metrics
Deterministic fixture: no payment service, network or SDK is called. Successful and error responses take 200 ms; a lost response costs the full client timeout. Backoff doubles from the base without jitter, so timings are exact; real clients add jitter to avoid synchronized retries. An attempt is started only if it can finish (200 ms) before the overall deadline. The idempotency key is modeled as a server that stores the first result and replays it.
A timeout is not a failure report
The agent calls a payment tool to charge order o-17 for 49.00 (fictional). A retry is safe only if repeating the request cannot repeat its effect. The dangerous case is a timeout: the client stops waiting, but it does not know whether the server did the work. In the opening fixture the server commits the charge on attempt 1 and the response is lost, so the client sees only a timeout after 3,000 ms. Server behaviour picks one of four fixed scenarios; every other input is a client-side retry policy.
The opening run: two charges for one order
With 3 retries, a 500 ms base backoff, a 3,000 ms client timeout, a 10,000 ms deadline and no idempotency key, attempt 1 runs from 0 to 3,000 ms and commits charge #1 before the response is lost. The client waits 500 ms (500 × 2^0) and sends attempt 2 at 3,500 ms; it succeeds at 3,700 ms and commits charge #2. The agent reports success, and the customer has been charged twice. Retrying was right; retrying a non-idempotent effect without a key was the bug.
Failure boundaries: key, retry count and deadline
Turn on Send an idempotency key the server honours: attempt 2 is answered from the stored result, so there is exactly one charge after 3,700 ms. Set Maximum retries to 0 instead: one charge exists but the agent reports a timeout, an unknown outcome that only a status lookup can resolve. The Overall deadline is the third boundary: at 4,000 ms attempt 2 still fits (it ends at 3,700 ms) and charges twice; at 3,500 ms it would end at 3,700 ms, so it is never sent and the outcome is unknown again. A shorter Client timeout per attempt of 500 ms moves the duplicate earlier, to 1,200 ms, without preventing it.
Outages and permanent errors behave differently
Choose HTTP 503 twice, then succeeds: nothing is committed by a 503, so retries are safe even without a key. Attempts run at 0-200, 700-900 and 1,900-2,100 ms after waits of 500 and 1,000 ms, and the third commits exactly one charge. With Maximum retries 1, or a 2,000 ms deadline, the run ends cleanly at 900 ms with no charge. Choose HTTP 400 on every attempt: with Retry on Timeouts and 5xx only, the agent stops after one 200 ms attempt; switch Retry on to Every failure, including 400, and it repeats the invalid request three more times, ending at 4,300 ms with nothing gained.
Reading the run and its limits
Play replays the computed attempts over sixty inspection seconds; Next event steps, Restart rewinds, Speed changes pace and Inspection time scrubs. Any input change recomputes the run and rewinds it. The trace gives each attempt’s exact start and end in milliseconds, the table lists every server answer and side effect, and the final metrics report attempts, charges, stored-result replays and elapsed time. The fixture uses fixed 200 ms responses and no jitter, so it shows ordering and effects, not realistic latency, and it does not model retries inside the SDK or the model layer.