RELIABILITY · REQUEST WORKSHOP
Rate limiting: inspect the 429 response
Fill the waiting room, watch workers process real model requests, and inspect every admission decision.
Your experiment
Exhaust the bucket. Inspect the 429 and rounded Retry-After estimate, then lower offered traffic. Distinguish rate-limit rejection from a 503 overload rejection.
- Play the default run and inspect the queue.
- Pause and capture a baseline.
- Change one parameter, then compare the same model time.
- Account for every offered unit: completed + pending + rejected + held back.
Admission trace
Teaching model, not an ADK/A2A runtime or measured infrastructure benchmark. Fixed service time, one FIFO, no retries, deadlines or network overhead; 50 ms ticks. Backpressure pauses a cooperative producer at 75% queue occupancy and resumes at 25%. Held-back demand is not transmitted or retained for later delivery. Early shedding uses half of queue capacity. Token bucket admission consumes one token per request, including requests subsequently rejected by the queue. The local Retry-After illustration is not a protocol-wide guarantee. Throttling reference · Load shedding reference.
Distinguish throttling from saturation
Choose the limiter and set refill to two tokens per second under twelve desired arrivals. Inspect 429 events and their Retry-After hints. This teaching policy rounds the estimated token wait up to whole seconds with a minimum of one; it is not a negotiated A2A contract or a promise that the next attempt succeeds. Switch to bounded FIFO with the same workers to inspect queue-overload responses instead. A client should not diagnose exhausted admission tokens and saturated worker queues solely from one total rejection counter.
Set offered work and service capacity
Desired requests per second sets deterministic arrivals, not measured customer demand. Workers sets the number of concurrent service slots. Service seconds is the fixed processing time for one request. Queue capacity bounds waiting requests; active work occupies separate slots. Compare desired rate with workers divided by service time before interpreting a growing queue. Completions are processed before admission in each fifty-millisecond model tick, so equal-time arrivals can reuse a newly freed slot.
Choose an admission policy
Policy selects bounded FIFO, producer backpressure, token-bucket rate limiting or early load shedding. FIFO rejects arrivals after the waiting queue fills. Shedding uses half the configured queue capacity, with a minimum of one waiting slot. Backpressure pauses the modeled producer at the high watermark and resumes below the low watermark: deferred requests are counted but are not retained for later replay. Token refill per second and Burst tokens apply only to the limiter. An initially full bucket permits a burst, but tokens do not add workers or increase service capacity.
Read outcomes and replay
Completed work, active requests, waiting requests, rejected work and deferred demand describe different outcomes. The limiter can spend a token before a later queue-capacity rejection. Inspect the event log for 429 versus overload responses instead of treating every rejection as the same fault. Play runs sixty modeled seconds; Pause holds the board, Step advances one second, Restart rewinds, Speed changes the inspection pace and the timeline reproduces the same state when scrubbed backward. Changes rebuild the scenario. Latency statistics concern this fixed-duration synthetic workload; there are no retries, variable service distributions, multiple tenants or production HTTP calls.