ADK deployment architecture — Cloud Run vs Agent Engine, sessions, identity, rollout

What actually ships — the agent package

The unit of deployment in ADK is a Python package that exposes a root_agent. The tooling — adk web, adk api_server, and the adk deploy subcommands — discovers agents the same way: it scans a directory for sub-packages whose __init__.py imports an agent module, and takes the module-level root_agent as the entry point. Get that convention wrong and every tool fails identically with ‘no agent found’.

agents/
  support_agent/
    __init__.py        # from . import agent
    agent.py           # root_agent = LlmAgent(...)  <- entry point
    tools/orders.py
    .env               # LOCAL ONLY. never in the image
  requirements.txt     # google-adk pinned
server.py              # the FastAPI wrapper
Dockerfile

Two disciplines pay for themselves. Keep agent.py free of import-time side effects — no network calls, no credential fetches, no files that only exist on your laptop; the module is imported during container startup, so anything slow or absent there becomes a startup-probe failure. And treat the package as the whole behavioral bundle: instructions, tool descriptions, and the model string are part of the artifact, not knobs you tweak in a console. That is what makes the rollout story later possible.

From adk api_server to a container image

The dev loop and the production server run the same runner; they differ in what wraps it. adk api_server starts a FastAPI app over your agents directory, and get_fast_api_app() gives you that same app as an object you can mount routes on, add middleware to, and configure. That object is the thing you containerize. Build it in a small server.py so the deployable surface is explicit and reviewable:

import os, pathlib
from google.adk.cli.fast_api import get_fast_api_app

app = get_fast_api_app(
    agents_dir=str(pathlib.Path(__file__).parent / "agents"),
    session_service_uri=os.environ["SESSION_SERVICE_URI"],
    artifact_service_uri=os.environ["ARTIFACT_SERVICE_URI"],
    allow_origins=os.environ["ALLOWED_ORIGINS"].split(","),
    web=False,          # no dev UI in production
)

Two version notes worth checking against your pinned release: the parameter names moved during ADK’s 1.0 stabilization (older code passes agent_dir and session_db_url), and web=True serves the developer UI — convenient in staging, an unauthenticated debugging console in production. Turn it off, or gate it behind your own auth middleware. Everything else about the app is ordinary FastAPI, which is exactly why this is the escape hatch when the managed runtime does not fit.

Three targets, one agent

ADK deliberately keeps the agent code constant across hosting choices; what changes is how much of the runtime you own. The decision is not about capability — all three run the same agent — but about which operational surface you want to be responsible for.

The honest default: if your differentiation is the agent rather than the platform, start on Agent Engine and move only when a requirement forces it. Move to Cloud Run for custom middleware, non-agent routes on the same service, private VPC access, or a container standard. Move to self-hosted only when a compliance boundary, an existing Kubernetes platform, or a non-Google cloud makes it unavoidable — each step down the table hands you work the step above was doing silently.

Agent Engine: hand over the runtime

Agent Engine inverts ownership. Instead of building an image and operating a service, you deploy the agent itself and the managed runtime supplies sessions, scaling, an authenticated query API, and identity integration. The CLI path is the one to reach for first because it packages the agent directory, resolves requirements, and creates the remote resource in one step:

adk deploy agent_engine \
  --project=$PROJECT \
  --region=us-central1 \
  --staging_bucket=gs://$PROJECT-agent-staging \
  ./agents/support_agent

There is also an SDK path: the Vertex AI SDK provides an ADK application wrapper you hand your root_agent, then create as a remote resource with an explicit requirements list and any extra local packages. Prefer it when deployment is a step inside a larger Python pipeline rather than a CI shell command. Either way the constraints are the same, and they are what to check before committing: dependencies must be declared rather than inferred from a working laptop, egress to private networks is more constrained than on Cloud Run, and the surface is the managed query API rather than any HTTP route you feel like adding. In exchange you stop operating a session database.

Cloud Run: the knobs that actually matter

On Cloud Run you are deploying a container that must listen on $PORT (8080 by default) and you own the settings. The ones that change agent behavior in production, rather than merely the bill:

gcloud run deploy support-agent \
  --image=$REGION-docker.pkg.dev/$PROJECT/agents/support-agent:$SHA \
  --service-account=support-agent@$PROJECT.iam.gserviceaccount.com \
  --set-env-vars=GOOGLE_GENAI_USE_VERTEXAI=TRUE,GOOGLE_CLOUD_PROJECT=$PROJECT \
  --set-secrets=ORDER_API_KEY=order-api-key:latest \
  --concurrency=40 --min-instances=1 --max-instances=50 \
  --cpu=2 --memory=2Gi --timeout=900 \
  --no-allow-unauthenticated --no-traffic --tag=canary

Concurrency is the one people get wrong. An agent turn is mostly waiting on model calls, so it is IO-bound and tolerates high concurrency — until a tool does real CPU work, at which point 80 concurrent turns on two vCPUs means everyone is slow. Start around 40 and tune on observed CPU. Timeout must exceed your worst-case multi-turn agent invocation, not your median. min-instances buys away the cold start of importing the SDK and your tools. And if you stream responses or do any work after the response, use always-allocated CPU — with throttling, an instance loses CPU between requests and background continuations quietly stall.

Self-hosted: what you take back on

The same image runs anywhere OCI containers run — GKE, another cloud’s container service, or your own hardware. The build itself is unremarkable, and it should be: a slim base, pinned requirements, a non-root user, and no development artifacts in any layer.

FROM python:3.12-slim
ENV PYTHONUNBUFFERED=1 PYTHONDONTWRITEBYTECODE=1
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
RUN useradd -m app && chown -R app /app
USER app
CMD ["sh", "-c", "uvicorn server:app --host 0.0.0.0 --port ${PORT:-8080}"]

What you take back on is everything the managed targets did for free: autoscaling policy, a load balancer, TLS, workload identity so the pod can call Vertex without a downloaded key file, secret injection, log and trace shipping, and the session database with its backups. Two build gotchas recur. Add a .dockerignore excluding .env, .git, and virtualenvs — a stray .env in a layer is the most common way credentials leave a laptop. And build with --platform linux/amd64 on Apple silicon, or the image refuses to start with an exec-format error.

Configuration: the environment, not the .env file

ADK loads a .env beside your agent package as a developer convenience, and that convenience is a production trap. The file exists to make adk run work on a laptop; it is not a deployment mechanism. In a deployed environment every value comes from the platform at deploy time, so the same image is promoted unchanged from staging to production and only the configuration around it differs.

Draw the line deliberately: behavior belongs in the image, wiring belongs in the environment. Instructions, tool sets, and the model string are behavior — version them with the code so a CI eval run tests what ships. Endpoints, bucket names, and project IDs are wiring. Fail fast on missing wiring: read required variables at import time and let the container refuse to start, which surfaces a misconfigured deploy as a failed revision rather than a runtime error on a user’s third turn.

Secrets and identity — one service account per agent

A tool-calling agent is a principal. It reads databases, calls internal APIs, and issues refunds, so the blast radius of a prompt-injected agent is exactly the IAM policy of the identity it runs as. Give each agent deployment its own service account granted only what its tools need — not one platform-wide account, and never a downloaded key file when workload identity is available.

Secrets follow the same rule: they live in a secret manager and are injected at deploy time as environment variables or mounted files, referenced by version — --set-secrets on Cloud Run, a secret volume on Kubernetes. Never put a credential in an instruction, a tool description, or session state; all three are model-visible, and session state is persisted and often replayed into evaluation datasets.

Keep the two layers of authorization distinct. The platform authenticates the caller: deploy with --no-allow-unauthenticated and put your gateway or IAM in front. Your callbacks authorize the action: a before_tool_callback that checks the end user’s entitlement, carried in session state, before a refund tool executes. A deployment that does only the first has an agent any authenticated caller can drive as anyone.

Sessions in a deployed environment

The in-memory session service is not a starting point you upgrade later — it is the one thing that makes horizontal scaling impossible, because a conversation only continues if the request lands on the replica holding it. Once state lives in a shared service, replicas are interchangeable and autoscaling and rolling deploys stop being conversation-ending events. What follows is only what changes at deploy time.

Pooling is per instance, not per service. A database-backed session service opens a pool inside each replica, so your real connection ceiling is pool size times max-instances. Fifty instances at a default pool of five is 250 connections against a database that may cap out at 100. Size the pool small, cap max-instances deliberately, and put a connection pooler in front at any real scale.

Migrations precede the revision that needs them. During a traffic split, old and new revisions read the same tables at once, so schema changes must be compatible in both directions for the length of the rollout. And a warning worth repeating: Cloud Run’s session affinity is best-effort routing, not persistence. Teams enable it, see conversations work, and believe the problem is solved — until a scale-down drops the instance and the history with it.

Health checks: startup, liveness, and what 'ready' means

The default health check answers the least interesting question. A container that has bound $PORT will happily return 200 while its session database is unreachable, its secrets are missing, and every conversation is about to fail. Terminology matters here: Cloud Run gives you a startup probe (has this instance finished booting?) and a liveness probe (is it wedged and in need of a restart?); a separate readiness probe that gates traffic is a Kubernetes concept, so on Cloud Run the startup probe is where the readiness question has to be asked.

Ask it honestly. A startup probe should check dependencies the agent cannot function without; a liveness probe should stay cheap and dependency-free, or a database blip will restart every healthy instance at once and turn a degradation into an outage.

from sqlalchemy import create_engine, text

_engine = create_engine(os.environ["SESSION_SERVICE_URI"], pool_pre_ping=True)

@app.get("/startupz")          # startup probe: can we actually serve a turn?
def startupz():
    with _engine.connect() as c:
        c.execute(text("SELECT 1"))
    return {"ok": True}

@app.get("/livez")             # liveness: cheap, no dependencies
def livez():
    return {"ok": True}

Set the startup probe’s failure threshold generously — importing the SDK, your tools, and any model client takes seconds, and an impatient probe kills instances mid-boot in a loop that looks exactly like a crash.

Rollout: revisions, traffic splits, and eval gates

Agents fail differently from services. A bad deploy usually does not throw; it answers slightly worse — routes to the wrong sub-agent more often, escalates less, hallucinates a policy it used to look up. No exception counter catches that, so the rollout has to be gradual and the signals behavioral.

The mechanism on Cloud Run is revision-based. Deploy with --no-traffic --tag=canary so the new revision gets a stable URL and zero live users; run your eval suite against that tagged URL; then move traffic in steps and watch.

# 10% to the canary revision
gcloud run services update-traffic support-agent --to-tags=canary=10

# healthy after a soak -> promote
gcloud run services update-traffic support-agent --to-latest

# regression -> instant rollback, no rebuild
gcloud run services update-traffic support-agent \
  --to-revisions=support-agent-00041-abc=100

The gate that belongs in the pipeline is an eval suite: recorded cases with expected final responses and expected tool trajectories, run against the built artifact before any traffic moves, with a score floor that fails the build. During the soak, watch agent-shaped signals rather than HTTP ones — tool-error rate, escalation rate, tokens per resolved conversation. On Agent Engine the shape is the same with different nouns: deploy a new resource version, point a fraction of clients at it, keep the previous one alive until you are confident.

What breaks first

A pre-deploy checklist, ordered by how often each item is the actual cause of a first production incident.

Notice the pattern: none of these is a bug in the agent. Each is a deployment decision that was left at its development default, and each is cheap to fix before the first deploy and expensive to diagnose after. Pin the model explicitly, pin google-adk, put the eval suite in CI before you need it, and write down which service account each agent runs as. Those four habits eliminate most of the table.