ADK Java Runtime — Anatomy of the Execution Loop

Phase 0: pre-loop setup

Before the loop starts, the runtime resolves three things: the compiled system instruction, the tool list, and the initial session context. All I/O for these must have happened during onStart(). Once the loop begins, latency budget is committed.

Phase 1: model call

Every iteration begins with a model invocation. The runtime packages the current message history + tool descriptions + instruction into a single prompt, submits it, and awaits either a text response or a tool-call decision.

// Inside the runtime — simplified
ModelResponse resp = model.generate(
    session.history(),
    agent.getTools(),
    agent.getInstruction()
);
if (resp.isToolCall()) {
    dispatchTool(resp.getToolCall());
} else {
    // final text — exit loop
}

Phase 2: tool dispatch

If the model returned a tool call, the runtime looks up the tool by name, validates arguments against its schema, and invokes it. The result is fed back into the message history as a tool_result role message.

What this page owns, and what it deliberately does not

This page walks a single iteration of the ADK Java agent loop from the inside: the ordered stages one pass runs through, the branch at the end that decides whether there is another pass, and the JVM-level behaviour each stage drags along with it. The scope is narrow on purpose. The ADK runtime architecture page covers the Runner as an object and the shape of a whole invocation. The core module overview maps the components onto each other. The streaming article owns the subscriber-side contract of the event stream - backpressure, cancellation, time to first token.

Nothing below re-argues any of that. It assumes you already know what a Runner and an Event are, and asks a different question: if you froze the JVM halfway through the third pass of a turn, what exactly would be on the stack, what would already be persisted, and what would still be garbage waiting to happen?

What one iteration actually is

Call runner.runAsync(userId, sessionId, msg, cfg) once and you get back a stream. Between that call and the last event on the stream, the runtime may travel the same circuit twice, five times, or twenty. Each trip is an iteration, and every iteration is bounded by exactly one model call. It starts when the runtime begins building the payload for that call. It ends when the runtime has decided whether to build another one.

Everything that makes agent behaviour interesting - a tool firing, a sub-agent taking over, session state changing beneath you - happens at the seams between iterations rather than inside them. That is what makes the iteration the right unit for reasoning about both latency and failure. A turn that "took nine seconds" was never nine seconds of any one thing; it was four passes of roughly two seconds each plus a slow persist. If your dashboards only measure the outer call, you cannot see which of the four misbehaved, and in practice exactly one of them did something the other three did not.

Numbering matters for the rest of this page. The pass under discussion is iteration N. Iteration N+1 is the pass that runs if the terminal branch says continue. Two sibling pages go deeper on individual pieces - model call orchestration on the provider client, the runtime thread model on scheduling in general - so this page stays on the sequence.

Stage 1 - building the payload for iteration N

The first thing a pass does is materialise the request the model will see. Three inputs get folded together: the agent's compiled instruction, the current tool declarations, and the conversation so far as a list of Content parts.

The property worth internalising is that this is not incremental. Iteration N does not hand the provider a delta over iteration N-1. It constructs the entire payload again, from the session's accumulated events. A pass that happens to be fourth in a chain rebuilds the original user message, the three prior model outputs, and the three tool results that came back - all of them, as fresh Java objects, on every single pass. Tool declarations are usually stable and are often the largest fixed cost in the payload; the history is the part that grows underneath you.

This is also where a beforeModel hook fires. It sees the assembled request for this pass specifically, which makes it the correct place to trim history, substitute an instruction, or short-circuit the pass with a cached answer. It is the wrong place to open a socket, for reasons that turn concrete in the threading section further down. Whatever the hook hands back is what the provider receives; there is no second assembly step waiting to correct it.

Stage 2 - the model call, and what the pass is waiting on

With the payload built, the pass hands it to the model client and stops. This is the only stage in the iteration whose wall-clock cost is not yours to control, and in nearly every production trace it dominates: the assembly above and the parsing below are microseconds to milliseconds of CPU, while the call itself is hundreds of milliseconds to tens of seconds of network.

Two properties of this stage catch people out. First, a retry inside the model client does not open a new iteration. If the client retries a 429 three times with backoff, the pass has performed one logical model call and four physical HTTP requests, and any per-iteration counter you wrote reports one. That is usually the behaviour you want, but it means iteration latency and provider latency are genuinely different metrics and you need to record both, or a provider having a bad afternoon will look to you like an agent that suddenly grew slow.

Second, streaming does not move the iteration boundary. When the provider streams, partial events reach your subscriber while the pass is still parked inside this stage. The pass cannot reach its terminal branch until the response is complete, because until the final chunk arrives it does not know whether a function call is coming. A user watching text appear on screen is watching stage 2 of a pass that has not yet decided whether it is the last one - which is why a confident-looking partial answer can still be followed by a tool round.

Stage 3 - turning one response into events

What comes back from the provider is one response. What the rest of the system consumes is events. This stage performs the conversion, and it is where a single reply can fan out.

A reply carrying only text becomes one event carrying content. A reply carrying function calls becomes an event whose parts are those calls - and when the model requests three tools at once, all three arrive inside that one event rather than as three separate ones. That distinction propagates everywhere downstream: fan-out is decided here, and any code that quietly assumes one function call per event will drop two of the three without an error.

Events produced here are handed to the subscriber as they are produced, not batched until the pass finishes. The practical consequence is ordering. Your subscriber can observe iteration N's function-call event before the corresponding tool has executed, which is precisely what makes a "checking your order" progress indicator possible, and precisely what makes any "the last event I received is the answer" logic wrong. Whether an event is the final response is a property the event carries, not a property of its position in time.

Stage 4 - the branch that decides whether this pass is the last

Here is the entire control flow of the loop, and it is smaller than most people expect. After stage 3 the pass asks one question: does this response require the runtime to go do something before the model can continue?

If the response is text with no function calls, the pass is terminal. The loop exits, and the event that carried that text is the final response of the invocation.

If the response contains function calls, the pass is not terminal. Each call resolves to a tool, its arguments are checked against the declared schema, and the tool is invoked. Results are converted into function-response parts and appended to the session's event history. Control then returns to stage 1, and iteration N+1 assembles a payload that now includes those results. The model asked a question about the world; the loop answered it and asked the model again.

Two variants sit on this same branch. A control signal - an escalate, or a transfer to another agent - ends the current agent's participation without being a plain text answer. A long-running tool suspends the loop outright, leaving the invocation open until something external resumes it. Both decisions are made at this exact point, which is why this is the one sensible place for a guard: if you want a ceiling on tool rounds, it belongs here and not inside any individual tool.

State the corollary plainly, because it surprises people during incidents: nothing in this branch is time-based. A loop that runs away is a loop whose model kept asking for tools. There is no built-in iteration budget unless you install one, and installing one is very cheap insurance against a prompt regression that turns a two-pass turn into a forty-pass turn.

Stage 5 - the write-back point, and what the next pass can see

State changes do not take effect where you wrote them. They take effect when the event carrying them is committed to the session.

A tool that mutates state during stage 4 is expressing an intention. That intention rides along on the event the runtime produces for the tool's result, and it lands on the session when that event is persisted. Until then, a concurrent reader of the session still sees the old value. This is a feature rather than an accident - it keeps the event log the single ordered source of truth, so replaying the log reconstructs state exactly - but it manufactures one specific bug over and over: a tool writes a key, reads the same key back a few lines later from a snapshot taken at entry, and gets the previous value.

The rule that follows is short enough to memorise. Anything written during iteration N is guaranteed visible to iteration N+1, because N+1's stage 1 reads a session that has already absorbed N's events. Nothing is guaranteed visible any earlier than that. And if two tools invoked in the same pass write the same key, the ordering of their events decides the winner, not the order in which their bodies happened to run.

Which thread each stage lands on

Now the JVM half. Stages 1, 3, 4 and 5 are ordinary synchronous Java - object building, parsing, a switch, a persist call. Stage 2 is the one that crosses into asynchrony, and the boundary is not where most people draw it.

The stream returned by runAsync is a cold reactive stream, an RxJava Flowable of events. Cold means that describing the pipeline performs no work at all; the loop does not begin until something subscribes. The thread that then executes the stages is whichever thread drives that subscription, unless a client inside a stage explicitly hands off. So in the simplest configuration, all five stages of every iteration run sequentially on the one thread that subscribed, with the model call in stage 2 blocking it for the duration.

That default is not a defect. On a virtual thread it is exactly the arrangement you want: the carrier is released while the model call parks, and the code reads top to bottom with no callback inversion. It becomes a defect the moment the subscribing thread belongs to a small shared platform pool - a servlet worker, a Netty I/O thread, an RxJava computation scheduler - because then stage 2 is occupying a resource sized for CPU work while doing nothing but waiting on a socket. The general argument lives in virtual threads for Java agents; the loop-specific version is one sentence. Whichever thread subscribes is the thread that pays for stage 2, once per pass.

Where a blocking call poisons the pass

Given the above, the failure mode states itself: any blocking work you insert into a stage runs on whatever thread is currently driving the iteration, and the iteration cannot move past it.

The two places people insert it are hooks and tools. A beforeModel hook that fetches a policy document over HTTP adds its full latency to every pass of every invocation, multiplied by however many passes a turn happens to require. A 60 ms lookup on a four-pass turn is 240 ms nobody budgeted, and it scales with the thing you least control - how chatty the model decides to be. A tool that blocks is at least expected to block, but it blocks that same driving thread, so a tool without an enforced timeout does not merely fail slowly; it pins the loop with no upper bound on the invocation at all.

The genuinely poisonous case is blocking on a bounded shared scheduler. Those pools are sized to core count. Block one from inside a hook and you have not slowed a single agent down - you have stalled every other subscription sharing that pool, and the symptom presents as unrelated agents timing out with nothing wrong in their own traces.

Two defences cover most of it. Give every tool a timeout enforced by the runtime rather than trusting the tool's own client to have one, and treat hooks as pure functions over the request and response objects. When a hook truly needs remote data, fetch it before you subscribe and close over the result.

Allocation and GC pressure, measured per pass

Each iteration allocates a payload proportional to the entire conversation so far, then discards it. That is the shape of the problem: garbage that is short-lived and large, produced once per pass.

Short-lived is the good half. Almost all of it dies before the pass ends, which is exactly the case a generational collector is built for - nothing gets copied, and the young collection stays cheap. Large is the bad half. An agent with a long history building a few hundred kilobytes of payload per pass, at four passes a turn, at a few hundred concurrent turns, produces garbage fast enough that young collections become frequent enough to see in a flame graph, even though each individual one is quick.

Three forces push it the wrong way. History growth compounds with pass count rather than adding to it, so the fourth pass serialises strictly more than the first and a turn needing more passes costs superlinearly. Serialising the payload roughly doubles peak footprint, because the object graph and its encoded form both exist for a moment. And if any of it survives a young collection - a hook that caches the request, a logger that retains the full prompt string, a metric label built from message content - that large graph gets promoted, and big garbage in the old generation is the expensive kind.

The lever with the best ratio here is not collector tuning. It is trimming history at stage 1, because a shorter history shrinks the payload, the token bill and the allocation rate in a single move.

When a stage throws

Exceptions do not stay inside the loop. Because the whole thing is a reactive stream, a throw in any stage terminates the pass and reaches your subscriber as an error signal rather than as an event.

That last clause is the one that bites. Your onNext handler will never see it; your onError will. Code that accumulates events into a StringBuilder and treats "the stream stopped" as "the answer is complete" will happily report a truncated response as a successful one unless the error path is wired. And since an error terminates the stream, there is no resuming from stage 4 - the invocation is finished.

Stage 4 deserves the emphasis. A tool exception that propagates kills the invocation outright. The same exception caught and returned as an error message inside the function response becomes information the model can act on, and the loop continues. Which you want depends entirely on whether the model can plausibly recover - a malformed date is recoverable, a revoked credential is not - but the accidental default in most codebases is the first, because nobody wrote the catch.

Stage 5 is the one to alert on. An event emitted to your subscriber but never persisted means the user saw an answer the next iteration will not know about, and no replay will reconstruct it.

Instrumenting a single pass end to end

None of the above is diagnosable from an invocation-level metric. You need per-iteration granularity, and a small set of hooks provides it without reaching into runtime internals.

Give every pass an index. Increment a counter in beforeModel, carry it in the logging context or as a span attribute, and now every log line and every span emitted during that pass is attributable to it. The beforeModel/afterModel pair brackets stages 1 and 2; the beforeTool/afterTool pair brackets stage 4. Subtracting them gives assembly-plus-model time and tool time separately, and whatever remains is stages 3 and 5.

// The pass counter is incremented in beforeModel, so every signal
// emitted while the loop is running carries the index of its pass.
AtomicInteger pass = new AtomicInteger();

runner.runAsync(userId, sessionId, msg, cfg)
    .doOnSubscribe(s  -> log.info("invocation begin user={}", userId))
    .doOnNext(ev      -> log.info("pass={} event={}", pass.get(), ev))
    .doOnError(err    -> log.error("pass={} aborted", pass.get(), err))
    .doOnComplete(()  -> log.info("invocation end passes={}", pass.get()))
    .subscribe(this::render);

Four numbers per pass are worth recording and almost nobody records them: the index, the history length that went into the payload, the model latency, and whether the pass turned out to be terminal. With those four you can answer the questions that actually arrive during an incident. A turn that got slower with no code change usually shows a rising average pass count rather than rising per-pass latency. An agent stuck in a tool cycle shows the index climbing with a terminal rate near zero. A cost spike with a flat pass count and a rising history length points straight at a trimming policy that quietly stopped trimming.

One caveat on carrying that context. Because a client inside a stage can hand off to another thread, anything you stash in a ThreadLocal during stage 1 is not guaranteed to still be there at stage 5. Prefer context that travels with the object graph, or a scope you propagate explicitly across the handoff, and verify it in a test that actually crosses the boundary rather than one that runs everything inline.