RELIABILITY · REQUEST WORKSHOP
Load shedding: reject before the queue fills
Fill the waiting room, watch workers process real model requests, and inspect every admission decision.
Your experiment
Compare bounded FIFO and early shedding at the same traffic rate. Measure completed throughput, rejections and latency; fewer queued requests are not proof that more work succeeded.
- Play the default run and inspect the queue.
- Pause and capture a baseline.
- Change one parameter, then compare the same model time.
- Account for every offered unit: completed + pending + rejected + held back.
Admission trace
Teaching model, not an ADK/A2A runtime or measured infrastructure benchmark. Fixed service time, one FIFO, no retries, deadlines or network overhead; 50 ms ticks. Backpressure pauses a cooperative producer at 75% queue occupancy and resumes at 25%. Held-back demand is not transmitted or retained for later delivery. Early shedding uses half of queue capacity. Token bucket admission consumes one token per request, including requests subsequently rejected by the queue. The local Retry-After illustration is not a protocol-wide guarantee. Throttling reference · Load shedding reference.
Compare useful work, not just queue length
Set desired arrivals to twelve per second, three workers and one-second service. Compare bounded FIFO with load shedding at queue capacity sixteen. Shedding begins rejecting earlier and may keep waiting latency lower; that does not mean it serves every offered request. Read completed work and rejection counts together. Increase workers to six while holding arrivals constant and compare the outcome again. This experiment exposes the trade-off between bounded waiting and discarded demand, rather than claiming that a small queue alone proves a healthy service.
Set offered work and service capacity
Desired requests per second sets deterministic arrivals, not measured customer demand. Workers sets the number of concurrent service slots. Service seconds is the fixed processing time for one request. Queue capacity bounds waiting requests; active work occupies separate slots. Compare desired rate with workers divided by service time before interpreting a growing queue. Completions are processed before admission in each fifty-millisecond model tick, so equal-time arrivals can reuse a newly freed slot.
Choose an admission policy
Policy selects bounded FIFO, producer backpressure, token-bucket rate limiting or early load shedding. FIFO rejects arrivals after the waiting queue fills. Shedding uses half the configured queue capacity, with a minimum of one waiting slot. Backpressure pauses the modeled producer at the high watermark and resumes below the low watermark: deferred requests are counted but are not retained for later replay. Token refill per second and Burst tokens apply only to the limiter. An initially full bucket permits a burst, but tokens do not add workers or increase service capacity.
Read outcomes and replay
Completed work, active requests, waiting requests, rejected work and deferred demand describe different outcomes. The limiter can spend a token before a later queue-capacity rejection. Inspect the event log for 429 versus overload responses instead of treating every rejection as the same fault. Play runs sixty modeled seconds; Pause holds the board, Step advances one second, Restart rewinds, Speed changes the inspection pace and the timeline reproduces the same state when scrubbed backward. Changes rebuild the scenario. Latency statistics concern this fixed-duration synthetic workload; there are no retries, variable service distributions, multiple tenants or production HTTP calls.