Content moderation pipeline: find the missed boundary

Decision trace

What this lab demonstrates

Moderation is not a single green or red badge. A prohibited request can arrive before generation, an unsafe tool action can be proposed during execution, and a candidate answer can expose harmful or private material after the model responds. This local fixture separates four boundaries so a learner can ask exactly where a particular problem is visible. A request-only check cannot detect a later tool action or output leak. The stage order is a teaching choice; in a real service several checks may run in parallel or use different policies and human escalation.

How to use each control

Inspection policy chooses a request-only check or all four boundaries. Injected case chooses a clean run or a defect at one named stage. Play inspection advances one boundary every fifteen seconds. Pause inspection freezes the view, Restart begins again, the Timeline slider seeks to a stage, and Speed changes only viewing pace. Read the trace for a check that was skipped, passed, or caught a defect; a skipped stage is not a pass. The final decision appears after sixty seconds and states whether the selected policy let the defect escape.

Try a controlled comparison

Choose Unsafe tool action and Request check only. The first stage passes because the input itself is not the unsafe action; the later tool boundary is omitted, so the defect escapes. Switch to All four boundaries and replay. The tool stage now blocks it. Repeat with Unsafe final or private-data output: the first three checks can appear healthy, but the final scan catches it. This comparison matters when a team assumes input moderation alone protects everything the agent might do. Identify the side effect that a real system would need to stop before it occurs.

What this model simplifies

This is a deterministic browser fixture with four named failure classes. It does not classify arbitrary text, call a provider model, run ADK Java, redact actual personal information, or prove that a moderation policy is effective. Every detector is assumed perfect for its one assigned example. Real moderation needs category definitions, appeals, false-positive review, privacy-safe logs, adversarial testing, jurisdictional policy, and checks at the actual execution boundaries. The one-minute timeline is for inspection, not measured latency or a promise that content will always be safe.

Put a check at the boundary where the risk appears

The four inspection stages represent input request, tool invocation, candidate output and private-data scanning. A prohibited input and an unsafe later output are different injected cases. Request check only selects the first stage; All four boundaries selects every stage. The policy determines which gates can detect a risk, rather than increasing a generic safety score. No provider classifier, real content sample or natural-language moderation model is called.

Understand what an injected case means

Injected case chooses no defect, prohibited input, unsafe tool action, unsafe generated or returned text, or unsafe final/private-data output. The model maps each case to exactly one stage. A selected matching stage reports caught; selected nonmatching stages pass, and unselected stages are skipped. The fixture does not evaluate arbitrary text, estimate confidence or discover a risk absent from this fixed list. The stage names describe where a real check would need to run.

Reproduce an escaped output

Choose Request check only and Unsafe final or private-data output. At the end the input stage passes, the remaining stages are skipped and ESCAPED DEFECT is displayed. Select All four boundaries while retaining the same case: the private-data stage catches it and the decision becomes BLOCK. Repeat with Unsafe tool action to move detection to the second stage. Prohibited input is caught under either policy because the first stage is always selected.

Separate fixture coverage from production safety

No injected defect produces PASS FIXTURE under both policies, but the single-check run leaves three boundaries unexamined. The simulator knows an escaped defect because you selected it, not because the first gate can see into future outputs. Its matching detectors are perfect by construction; there are no false positives, classifier misses or contextual judgments. A real pipeline also needs enforcement before a tool effect or user-visible output, not merely an after-the-fact label. These independent inspection cards do not execute a real tool or prove that subsequent processing stops on rejection.

Inspect the replay

Play starts the sixty-second inspection and becomes Pause while running. Pause holds the current stage; Restart returns to the beginning without changing the experiment settings. Speed changes viewing pace rather than model outcomes. Timeline seeks directly to a stage and pauses playback. Changing a scenario setting recomputes the deterministic fixture and rewinds it. The final decision appears at the end, so move Timeline to sixty to compare completed runs. Stage cards and the text trace describe the same selected fixture; their elapsed viewing time is not measured agent execution latency.