Agent CI/CD: four evidence gates before promotion

Decision trace

What this lab demonstrates

An agent release pipeline needs evidence from different layers. A successful build establishes that an artifact was produced, but does not show that tool arguments remain compatible, answers still meet an offline quality bar, or a small live cohort behaves acceptably. This lab represents those four gates as exact synthetic fixtures. A deliberately injected defect affects one gate; selecting only the early gates can produce a false green release decision. The point is to reason about coverage and promotion order, not to turn every test result into a single improvement percentage.

How to use each control

Use Injected defect to choose a broken dependency, a tool-contract change, a golden-answer regression, or a canary error spike. Checks included sets the deepest gate that runs. Play test run advances the pipeline for sixty seconds, spending fifteen seconds on each gate; Pause freezes it and Restart starts a new inspection with the same selected fixture. Seek on the timeline to see the exact decision at a boundary. The speed menu changes only how quickly the sequence is viewed. A check marked Not run provides no release evidence. Read the trace beneath the cards to see each gate's fixture and status.

Try a controlled failure

Start with Golden answer regression and include only the first two checks. Compile and package passes, as does the tool-contract suite, but the changed answer behavior escapes. Include the third gate and replay. The fixed offline prompt dataset now detects it and blocks promotion. Next choose Canary error spike. Even an offline evaluation pass cannot see this defect; all four gates are needed to block the synthetic release. Compare the two cases and write down which kind of evidence each gate adds. The most valuable gate is not necessarily the first one to turn green.

What this model simplifies

This browser does not build a Java artifact, execute ADK, call a model, deploy a canary, or measure operational SLOs. The golden data, error threshold, and faults are fixed teaching examples. Real CI/CD must pin dependencies, isolate test tenants and secrets, version evaluation datasets, record reproducible artifacts, and provide rollback controls. An evaluation may itself be noisy or biased; a canary may miss rare failures. Passing the depicted sequence is evidence for the selected fixtures only, not proof that an arbitrary agent release is safe or that a model is accurate.

Separate pipeline stages from coverage

This agent release pipeline has four distinct gates: package construction, tool-contract unit tests, a golden-prompt evaluation and a small canary cohort. Each gate represents evidence about a different failure source. Passing the first three does not establish that the deployed candidate behaves correctly under the last gate. The displayed result is a deterministic release-policy exercise over four named checks, not the output of a CI service. The historical ADK Java URL describes the engineering topic; no Java build or deployment is performed.

Choose coverage deliberately

Injected defect selects one known fault, or a clean fixture. Checks included runs the first one, two, three or all four checks in their displayed order. This is prefix coverage, not a random sample and not a selection of arbitrary individual checks. A skipped check provides no evidence. The model uses an explicit mapping from defect identifiers to checks that detect them; it does not execute Java code, call an LLM, compile a package or infer semantic correctness. An escaped defect is visible because the simulator knows which fault you injected, whereas a real test runner would need independent evidence to discover that escape.

Reproduce a coverage gap

Choose Canary error spike and include three checks. At the end the build, contract and golden-prompt stages pass, the canary check is skipped and the decision is ESCAPED DEFECT. Increase coverage to four while keeping that fault: the canary stage fails and BLOCK RELEASE replaces the escape. Select Broken package dependency with one check; the earliest stage now catches it. This shows why a longer pipeline can reveal a late defect without changing an earlier result.

Interpret the failure boundary

Try No injected defect at coverage one and at coverage four. Both show PASS FIXTURE, but their evidence differs: three additional checks have actually been selected in the latter fixture. A clean label is not a production readiness certificate. There are no real artifact signatures, environment promotions, secrets, rollback execution, statistical evaluation thresholds or external dependency checks. Carry those requirements into a real pipeline separately.

Inspect the replay

Play starts the sixty-second inspection and becomes Pause while running. Pause holds the current stage; Restart returns to the beginning without changing the experiment settings. Speed changes viewing pace rather than model outcomes. Timeline seeks directly to a stage and pauses playback. Changing a scenario setting recomputes the deterministic fixture and rewinds it. The final decision appears at the end, so move Timeline to sixty to compare completed runs. Stage cards and the text trace describe the same selected fixture; their elapsed viewing time is not measured agent execution latency.