DEBATE · COUNTEREXAMPLE · JUDGE
Agent debate: let a counterexample change the decision
Let an opponent search the release evidence, then see whether the judge rule lets one verified counterexample change the decision.
Your experiment
A proponent wants to ship an agent-generated daily report. The opponent inspects the six evidence items in order, a few per round. Start with the counterexample in item 5 and the majority judge, then change the judge rule, the search depth and the verification requirement.
Play replays the computed run over sixty inspection seconds; Next event steps, Restart rewinds, and Inspection time scrubs. Any input change recomputes the run and rewinds it without autoplay.
Event trace
Computed data
Final metrics
Deterministic teaching model, not an LLM debate. The opponent inspects items in a fixed order, the data scan is an oracle that knows which item holds customer data, and item 2 always looks suspicious but holds synthetic data. A round is scored for the opponent only when it brings a new objection. No model, file or customer record is accessed.
A debate is only as good as its decision rule
The proponent argues for shipping an agent-generated daily report. The opponent inspects six evidence items in a fixed order; Items the opponent inspects per round sets how many it reads each round and Debate rounds sets how many rounds it gets. A round is scored for the opponent only when it produces a new objection. With the opening settings the counterexample is item 5, the opponent sees two items per round, and it finds and verifies the customer data in round 3. That is one round won out of three, so the Majority of rounds won judge ships the report: a wrong decision even though the debate found the decisive evidence.
Let one verified counterexample decide
Switch the Judge rule to Any verified objection vetoes and the same transcript produces BLOCK, the correct decision. The veto rule treats a verified counterexample as decisive regardless of how many earlier rounds went to the proponent, which is the point of having an opponent at all. Proponent decides ignores the debate entirely and always ships. Counting rounds measures persistence, not evidence.
The search budget is a hard boundary
Keep the veto rule and set Debate rounds to 2. The opponent inspects only items 1 to 4, never reaches item 5, and the judge ships: wrong, and no judge rule can fix it because no objection exists. Raise Items the opponent inspects per round to 3 and two rounds cover all six items; the counterexample is found in round 2 and the release is blocked. Move the counterexample to item 1 and even a one-round debate finds it.
Unverified objections cut both ways
Set Evidence item that holds customer data (ground truth) to None, keep the veto rule and clear Objections must be verified by the data scan. Item 2 looks like a real address but holds synthetic test data; without verification the opponent raises it anyway and the veto blocks a safe release. With verification the suspicion is checked and withdrawn and the release ships correctly. Back on the opening settings, clearing verification makes the majority judge block, but only because the false alarm in round 1 gave the opponent a second round: the right answer for the wrong reason.
Inspecting the run and its limits
Play, Next event, Restart, Speed and Inspection time step through every inspection, the round scores, the judge and the verdict across sixty seconds; the table lists each inspected item and the finding. This is a deterministic model of the decision rule, not of language-model behaviour: the data scan is an oracle, the inspection order is fixed and there is no persuasion, sycophancy or hallucinated evidence. Real debate setups need a verifier that is independent of both debaters.