SERVING SCALE · FIXED OR ELASTIC
Agent serving scale: compare fixed and elastic capacity
Serve an hour of agent traffic with a load spike and compare fixed replicas with an autoscaler that needs time to start new ones.
Your experiment
Traffic runs at 20 requests per second, jumps to 80 at minute 20, drops to 40 at minute 35 and returns to 20 at minute 45. Start with the elastic autoscaler and a three-minute cold start, then compare cold starts, limits and fixed capacity.
Play replays the computed run over sixty inspection seconds; Next event steps, Restart rewinds, and Inspection time scrubs. Any input change recomputes the run and rewinds it without autoplay.
Event trace
Computed data
Final metrics
Minute-step capacity model, not Vertex AI Agent Engine or any autoscaler API. Each minute the autoscaler asks for ceil(load / (per-replica rate x target)) replicas within the limits; new replicas serve after the cold start; scale-down is immediate; requests above capacity are dropped rather than queued. Replica-minutes count starting replicas too.
An hour of traffic with a spike
Traffic runs at 20 requests per second, jumps to 80 at minute 20, falls to 40 at minute 35 and returns to 20 at minute 45: 138,000 requests in the hour. Requests per second one replica serves sets the capacity of each replica. The elastic autoscaler asks every minute for enough replicas to keep utilization at Target utilization, within Minimum replicas and Maximum replicas, and new replicas serve only after Cold start for a new replica.
The opening run drops the start of the spike
At 20 requests per second the autoscaler runs three replicas. At minute 20 it asks for twelve, capped at ten, but with a three-minute cold start the new replicas serve from minute 23. For minutes 20 to 22 capacity is 30 against 80, so 9,000 requests are dropped, 6.52 percent, for 315 replica-minutes. Set the cold start to 1 minute and only minute 20 drops, 3,000 requests or 2.17 percent; at 0 nothing is dropped.
Fixed capacity is a cost decision
Set Capacity mode to Fixed replicas: four replicas give 40 requests per second, the whole spike overflows and 36,000 requests, 26.09 percent, are dropped for 240 replica-minutes. Eight fixed replicas drop nothing but cost 480 replica-minutes, against 315 for the elastic run. Fixed replicas only matter in fixed mode; the elastic limits and cold start only matter in elastic mode.
Limits and headroom
In elastic mode, Maximum replicas at 7 caps capacity at 70 during the spike and 11.74 percent is dropped. Minimum replicas at 5 keeps 50 requests per second ready before the spike, so the cold-start gap drops 5,400 requests, 3.91 percent, at 385 replica-minutes. Raising Target utilization to 0.9 runs fewer replicas, 290 replica-minutes, with the same 9,000 dropped, because the cold start, not the target, causes the drops.
Reading the run and its limits
Play, Next event, Restart, Speed and Inspection time step through each change in capacity and the verdict across sixty seconds; the chart overlays offered load and ready capacity per minute, and the table lists every minute. The model has no queueing, retries, request latency, stabilization windows or per-instance concurrency; it does not describe how Vertex AI Agent Engine or any specific platform scales, only why warm capacity and start-up time matter.