EDGE DEPLOYMENT · MEMORY BUDGET
Edge deployment: fit the model inside the worker budget
Place an edge classifier, a JVM agent and its sessions across a Worker isolate, a container and a region, and see what fits, what sleeps and what is lost.
Your experiment
A Worker in front of the agent should answer two FAQ requests with a small classifier, reject an expired token and a request flood, and route model turns to an ADK Java agent in a container. The chosen 90 MB classifier needs 130.5 MB in a 128 MB isolate, so nothing is served. Shrink it, then follow user A’s conversation across the quiet spell from 22 to 40 seconds.
Play replays the computed run over sixty inspection seconds; Next event steps, Restart rewinds, and Inspection time scrubs. Any input change recomputes the run and rewinds it without autoplay.
Event trace
Computed data
Final metrics
Model assumptions, not platform measurements: a 128 MB isolate budget; Worker memory = 18 MB + 1.25 × the artifact; the agent needs a 1,100 MB heap and gets 75% of instance memory; a 1.8 s model call, 0.12 s extra round trip to the region, containers sleeping after 15 s idle and 6 s cold starts. Check your plan’s current limits before relying on any number.
Three places to run code
The Worker runs at the edge in a V8 isolate with, in this model, a 128 MB memory budget; it can host a small classifier, authentication and rate limiting, but not a JVM. Agent runtime places the ADK Java agent in a container next to the Worker, inside the isolate, or in a regional service. Edge classifier artifact sets the Worker’s memory (18 MB plus 1.25 times the artifact), Agent instance memory sets the JVM heap (75% of it, against a 1,100 MB need), and Session store decides whether conversation history survives a sleeping container.
The opening run: the classifier does not fit
A 90 MB classifier needs 130.5 MB in a 128 MB isolate, so the Worker is terminated as soon as it loads and every request fails, even though the agent itself would run. Set Edge classifier artifact to 40 MB: the Worker uses 68 MB, answers both FAQ requests without a model call, rejects the expired token with a 401 and 19 of 20 flood requests with a 429. A 300 MB artifact (393 MB) is far over; with no classifier the Worker needs only 18 MB.
Failure boundary: sleep and sessions
With the 40 MB classifier, the container serves user A’s first two turns, then nothing arrives between 22 and 40 seconds. It sleeps after 15 seconds idle, so the turn at 40 s pays a 6-second cold start and, with In-memory sessions, finds that turns 1 and 2 are gone. Set Session store to External session store and the conversation survives: the verdict becomes DEPLOYABLE. Untick Authenticate and rate-limit in the Worker and the expired token and all 20 flood requests reach the JVM, 25 requests in total.
JVM placement and memory
Set Agent runtime to Inside the Worker isolate: the agent cannot start, because an isolate runs JavaScript, TypeScript, WebAssembly and Python, not a JVM. Set Agent instance memory to 1 GiB: 75% is 768 MB, below the 1,100 MB heap the agent needs, so it fails on its first turn; 2 GiB gives 1,536 MB. Set Agent runtime to Regional service: there are no cold starts, and the median model turn is 1.95 s instead of 1.83 s, because the model call dominates either way.
Reading the run and its limits
Play replays the deployment checks and eight requests over sixty inspection seconds, and the table gives each request’s outcome and latency. Next event, Restart, Speed and Inspection time control the replay, and any input change recomputes it. Every number is an assumption of the model, not a measurement: memory limits, cold starts and container features depend on the plan and change over time. What carries over is the shape: the edge layer is for fast rejection and small models, the JVM lives in a container or region, and state lives outside both.