Debate as a truth mechanism
The premise behind debate is that a single forward pass commits to whatever answer the model’s first sample happens to produce, with no chance to catch its own error. Debate adds a correction channel. You instantiate N agents — usually the same model with different samples or prompts — and ask each to answer independently. Then you show every agent the others’ answers and reasoning and ask it to reconsider.
The mechanism only helps if errors are less correlated than correct answers. When one agent makes a careless slip the others did not, the outlier sees a coherent competing argument and often abandons its mistake; when several agents independently reach the same correct chain, that agreement reinforces it. Debate therefore turns many noisy, partly-independent samples into an answer that reflects where the reasoning is robust rather than where one sample landed by chance.
The argue-then-decide loop
The loop has three moving parts. First, generation: each of the N agents produces an answer with its reasoning. Second, critique and revision: for R rounds, each agent is given the other agents’ current answers and asked to critique them and update its own. Third, aggregation: after the last round a decision rule collapses the answers into one — either a majority vote over the final answers or a separate judge model that reads the transcript and picks a winner.
A minimal, effective configuration is small: three agents, two rounds. Each round’s prompt is essentially ‘here are other solutions to the same problem; use them to reconsider and give an updated answer.’ The loop needs no special training — it is a prompting and orchestration pattern on a frozen model, much of why it is attractive.
Why debate can amplify correct reasoning
The intuition is about verification being easier than generation. Producing a correct multi-step solution from scratch is hard, but spotting that a specific step contradicts another agent’s step is comparatively easy. Debate leans on that gap: an agent acting as a critic on someone else’s work brings fresh scrutiny it would not apply to its own fluent output.
There is also a simple statistical effect. Suppose each agent is independently correct with probability p > 0.5 and errors are uncorrelated. A majority vote over N agents is then more likely to be correct than any single agent — the classic wisdom-of-crowds result. Debate adds a second lever on top of voting: the critique rounds let agents change wrong answers before the vote, nudging that per-agent p upward. When both effects fire, the group lands well above the solo baseline. The lineage traces to Irving et al. 2018, which framed debate under a judge as a way to amplify truthful reasoning.
What the Du et al. results show
Du et al. 2023 studied exactly this argue-then-revise setup across reasoning and factual tasks — grade-school and harder arithmetic, math word problems, and knowledge questions. The consistent finding is that debate beats both a single answer and simple self-reflection, and that the gains are largest where a problem has a checkable answer that a wrong agent can be argued out of.
The reported improvements are real but bounded — typically several accuracy points over a single-pass baseline on the reasoning benchmarks, not a transformation of a weak model into a strong one. Two other observations matter as much as the headline number: agents genuinely revise their answers between rounds rather than restating them, and debate reduces confidently wrong factual claims because a false assertion tends to draw contradiction from at least one peer.
Rounds versus diminishing returns
The number of rounds R is the main quality dial, and it is a curve with sharp diminishing returns. The first round of critique does most of the work: it is where an agent first sees a competing solution and where careless errors get corrected. A second round captures some stragglers. By the third or fourth round the agents have usually converged, and further rounds mostly restate an already settled position.
Convergence cuts both ways. If the group is converging on the correct answer, extra rounds are simply wasted compute. If it is converging on a wrong answer, extra rounds actively harden the mistake as agents reinforce each other’s error. So more rounds are not reliably better; the accuracy-versus-R curve rises fast, flattens, and can even bend down. In practice two to three rounds captures most of the gain, and pushing past that buys little except cost.
The cost of arguing
Debate is expensive, and the cost grows faster than it first appears. Each round every agent must read the other agents’ answers before writing its own, so the prompt an agent processes grows with the number of peers. With all-to-all sharing, per-round work scales roughly as O(N^2) in agents, and total work as O(N^2 × R) across rounds — plus the growing transcript each agent re-reads.
A concrete feel: three agents over three rounds is nine generation calls, and every call after the first carries the accumulated arguments in its context, so the token bill is several times a single answer’s. Against that, ask what a few accuracy points are worth. For a high-stakes one-off the cost is easily justified; for high-volume, latency-sensitive traffic it usually is not.
Judge versus majority vote
The aggregation rule matters more than it seems. A majority vote over final answers is cheap, transparent, and robust when the answer is a discrete value that can be compared exactly — a number, a label, a choice. Its weakness is that it counts votes without weighing the reasoning, so three agents that made the same mistake outvote one agent that got it right.
A judge model reads the full transcript and picks the answer whose argument holds up, which lets a strong minority argument win. That is more powerful for open-ended answers where there is no exact vote to take, but it moves the burden onto the judge: it must be at least as discerning as the debaters, and if it is swayed by fluent, confident prose rather than correct reasoning it becomes the new single point of failure. Vote when answers are comparable; use a judge when they are not.