Phase 0: pre-loop setup

Before the loop starts, the runtime resolves three things: the compiled system instruction, the tool list, and the initial session context. All I/O for these must have happened during onStart(). Once the loop begins, latency budget is committed.

Advertisement

Phase 1: model call

Every iteration begins with a model invocation. The runtime packages the current message history + tool descriptions + instruction into a single prompt, submits it, and awaits either a text response or a tool-call decision.

// Inside the runtime — simplified
ModelResponse resp = model.generate(
    session.history(),
    agent.getTools(),
    agent.getInstruction()
);
if (resp.isToolCall()) {
    dispatchTool(resp.getToolCall());
} else {
    // final text — exit loop
}
Advertisement

Phase 2: tool dispatch

If the model returned a tool call, the runtime looks up the tool by name, validates arguments against its schema, and invokes it. The result is fed back into the message history as a tool_result role message.

Phase 3: stopping condition

After each turn the runtime asks: are we done? Three signals end the loop:

  • Model returned pure text (no tool call) — most common terminator.
  • maxTurns reached — safety fuse against runaway loops.
  • An error was raised and no onError handler chose to continue.

Phase 4: return + persist

The runtime returns the final AgentResponse to the caller, optionally persisting the session for later resumption. This is the last window to emit metrics, close scoped resources, and log the completed transcript.

Hook points inside the loop

Every phase exposes a hook you can override on LlmAgent:

  • beforeModelCall(ctx) — inspect prompt about to be sent.
  • afterModelCall(resp, ctx) — inspect model output, mutate before dispatch.
  • beforeToolCall(name, args, ctx) — block, rewrite, rate-limit.
  • afterToolCall(name, result, ctx) — sanitize, add metrics.
  • onError(err, ctx) — decide continue or terminate.