Agent smoke testing: find a broken startup before release

Decision trace

What this lab demonstrates

A smoke test is a fast answer to a narrow question: did this build start, become ready, expose its required tool, and return one structurally valid response? The four cards model those observations in order. They do not judge whether an answer is useful or factual. Injecting a known defect lets you see the difference between a process that looks alive and a build that can perform its basic contract. A green startup check alone is weak evidence for an agent whose tool registry or response envelope can still be broken.

How to use each control

Choose an Injected defect to alter the synthetic fixture, then set Checks included to one through four. One includes only process startup; four includes readiness, tool discovery, and a probe reply. Play test run advances a sixty-second inspection, one stage every fifteen seconds. Pause freezes the displayed stage, the timeline slider seeks directly to a stage boundary, and Restart returns to the beginning with the selected fixture intact. The speed menu changes playback rate without changing a result. The decision trace lists exactly which checks ran and what each checked.

Try a controlled failure

First choose No injected defect and include all four checks. At the end, every card should pass. Next choose Missing registered tool while leaving only one check included. The startup card will pass and the defect will escape, because that selected check never inspects the registry. Increase Checks included to three and replay. Tool discovery now catches the same defect and blocks the release. Repeat with Stale readiness result to see why a fresh readiness probe differs from a cached green badge. Record which selected observation changed the decision, not merely the color of a card.

What this model simplifies

All four results come from fixed teaching fixtures; the page never launches an ADK Java process or calls a real model. The sixty-second clock is an inspection aid, not measured test duration. A real smoke suite also needs environment setup, credentials, cleanup, response-time budgets, and an explicit decision about which failures must stop deployment. This lab shows defect detection under chosen coverage, not the probability that a production agent is safe. A passed fixture is a reason to continue to deeper tests, not a release guarantee.

Ask whether this process can serve one probe

The smoke sequence checks startup, fresh readiness, tool discovery and a well-formed probe response. These are deliberately shallow signals. A process can boot while exposing stale readiness, omit a necessary tool or return a malformed envelope. The four displayed checks keep those distinctions visible instead of collapsing them into one green service indicator. The lesson models check selection using fault tags; it does not spawn a process or make an HTTP readiness request.

Choose coverage deliberately

Injected defect selects one known fault, or a clean fixture. Checks included runs the first one, two, three or all four checks in their displayed order. This is prefix coverage, not a random sample and not a selection of arbitrary individual checks. A skipped check provides no evidence. The model uses an explicit mapping from defect identifiers to checks that detect them; it does not execute Java code, call an LLM, compile a package or infer semantic correctness. An escaped defect is visible because the simulator knows which fault you injected, whereas a real test runner would need independent evidence to discover that escape.

Reproduce a coverage gap

Choose Stale readiness result and include only the first check. Startup passes, but readiness is untested and the known defect escapes. Increase Checks included to two: the fresh-readiness check catches it. With all four selected, change to Missing registered tool and then Malformed probe reply. The failed stage moves from discovery to the probe contract. Startup exception is detected by the first two checks when both are included, illustrating that detection coverage can overlap.

Interpret the failure boundary

A successful probe is not evidence of general answer quality, tenant isolation, throughput or long-running tool reliability. Select No injected defect with one check and inspect the three skipped stages before treating the final PASS FIXTURE label as meaningful. The simulator reports escaped defects from its hidden fixture knowledge, not from an omniscient production monitor. A real smoke suite would need actual assertions, fresh requests and independent evidence that its expected tool names and response shape are current.

Inspect the replay

Play starts the sixty-second inspection and becomes Pause while running. Pause holds the current stage; Restart returns to the beginning without changing the experiment settings. Speed changes viewing pace rather than model outcomes. Timeline seeks directly to a stage and pauses playback. Changing a scenario setting recomputes the deterministic fixture and rewinds it. The final decision appears at the end, so move Timeline to sixty to compare completed runs. Stage cards and the text trace describe the same selected fixture; their elapsed viewing time is not measured agent execution latency.