Multi-Agent Reflection

What reflection is, precisely

Reflection is a feedback loop on top of generation. A model produces a candidate answer; an evaluator inspects it and emits a verbal assessment — ‘the proof skips the base case’ — and that assessment is fed back into the context for a revised attempt. Nothing is trained; the only thing that changes between attempts is the prompt, which now carries the critique.

This is worth stating carefully, because it is easy to over-claim. The model is not improving in any lasting sense; it is being given more informative conditioning. The revision is better only insofar as the critique surfaced information the first pass failed to use. Reflection buys quality only when there is a gap between what a model can generate and what it can recognise — and that gap is what the rest of this article measures.

Reflexion: verbal reinforcement, not gradients

The canonical framing is Reflexion — treating a natural-language critique as a substitute for a gradient update. Ordinary reinforcement learning gets a scalar reward and nudges the weights. Reflexion keeps the weights frozen and instead writes a sentence of self-feedback into a memory the next attempt reads. The reinforcement is a paragraph, not a number.

Language is far higher-bandwidth than a scalar. A reward of -1 says the agent failed; the sentence ‘you passed the API arguments in the wrong order’ says why and implies the fix. The catch is that the feedback is only as good as the evaluator that wrote it — and when the evaluator is the same model that produced the error, its introspective sentence can be as wrong as the answer.

The actor, evaluator, self-reflection split

Multi-agent reflection decomposes the loop into three roles, which can be three prompts or three models. The actor generates the answer. The evaluator scores it against the task — running tests, checking a rubric, or judging quality. The self-reflection role reads both and produces the concrete lesson to carry forward.

Separating the roles matters because they demand opposite things. Generation rewards fluency and coverage; evaluation rewards skepticism and precision. One prompt asked to do both tends to rubber-stamp its own work. Splitting them lets each be tuned, and lets you drop a stronger model into the evaluator seat.

Self-critique versus cross-agent critique

The sharpest design question is who writes the critique. In self-critique the generator reviews its own output in a fresh pass. In cross-agent critique a separate critic — a different model, or the same model under a hostile prompt — reviews the generator’s work.

The difference is about error correlation. A model’s blind spots are a property of its weights and prompt, so when it grades itself its critique inherits them: the error it could not avoid making, it also cannot see. Let m be the probability the generator misses a given flaw. Under self-critique the review misses it with probability close to m too. An independent critic with a decorrelated miss-probability m′ catches far more: the flaw survives only if both miss it, roughly m × m′. That multiplicative gap is the argument for a second, independent critic over self-introspection.

The math of iterative refinement

Model the loop as error reduction. Let e_t be the residual error after round t. Suppose each round fixes a fraction ρ of the fixable error, but a floor ε stays invisible to the critique channel (real blind spots, ambiguous spec). Then:

e_(t+1) = (1 - ρ)·e_t + ρ·ε
closed form:  e_t = ε + (1 - ρ)^t · (e_0 - ε)

This is geometric decay toward the floor ε. The key fact falls out of (1-ρ)^t: error shrinks by the same ratio each round, so absolute gains shrink fast. With ρ = 0.5, round one removes half the fixable error, round two half the rest, round three half of that. Reflection is front-loaded — nearly all the benefit is in the first one or two rounds, and no number of rounds pushes below the floor set by what the evaluator can perceive.

A worked example

Take a first draft with error rate e_0 = 0.40, a critique that fixes ρ = 0.5 of fixable error per round, and a floor ε = 0.10. The recurrence gives:

round 0:  e = 0.40
round 1:  e = 0.10 + 0.5 × (0.40 - 0.10) = 0.25    (gain 0.15)
round 2:  e = 0.10 + 0.5 × (0.25 - 0.10) = 0.175   (gain 0.075)
round 3:  e = 0.10 + 0.5 × (0.175 - 0.10) = 0.1375 (gain 0.0375)

Each round’s improvement is half the last. By round three you are within four points of the 0.10 floor and every further call buys less than two. A rational stopping rule follows: halt when marginal gain drops below the marginal cost of another round — here, around round two or three. The self-refinement literature agrees that most gains land in the first revision.

When refinement plateaus and degrades

The clean picture assumes each round only ever removes error. Reality adds a second term: reflection can inject error. A critic out of real problems starts inventing them, and the generator, primed to defer to feedback, ‘fixes’ things that were correct. Write the injected-error rate per round as δ:

e_(t+1) = (1 - ρ)·e_t + ρ·ε + δ
fixed point:  e* = ε + δ / ρ

The fixed point is now above the floor by δ/ρ. Small δ still converges to something good, but it tends to grow as rounds accumulate — the more polished a draft, the more a critic reaches for nits, and the higher the chance it corrupts something right. That is the origin of the U-shaped curve: quality improves, bottoms out, then worsens. Over-correction and critique-induced sycophancy are the default endgame of a loop run too long.

Both terms favour an external, grounded critic. A model grading its own work has a high ε and a high δ: without a reference it cannot tell a real flaw from a preference. A grounded evaluator — unit tests, a compiler, a retrieval check — drops ε and drives δ toward zero by reporting only real failures. The math rewards a critic that cannot hallucinate a problem.

The cost of extra rounds

Reflection is not free. One round of generate-plus-critique is at least two model calls; add a distinct reflection role and it is three. So k rounds cost roughly 2k to 3k forward passes versus one for a plain answer — before you count that each round re-reads a growing transcript.

That last point is the quiet killer. If every round appends the previous draft and its critique to the context, tokens processed across k rounds scale like 1 + 2 + … + k = O(k^2), not O(k). So gains fall as (1-ρ)^t while cost climbs as t^2 — a product with a sharp early optimum, the reason to cap reflection at one or two rounds unless a grounded evaluator justifies more.

Reflection on a CPU-bound small model

Everything above tilts further for a small model on a CPU. The premise of reflection is a favourable gap between recognition and generation. In large models that gap is often real; in a 1–3 billion-parameter model it frequently is not: it misses its own errors at nearly the rate it makes them, so pure self-reflection barely moves the number and can spend tokens making things worse.

Two corrections follow. Prefer a grounded critic to a verbal one — a unit test or schema check gives a small model the reliable signal its introspection cannot. And respect wall-clock cost: on a CPU each pass is slow, so a 3× multiplier from a three-role loop is felt directly by the user. The right default is one reflection round gated by an external verifier, not an open-ended conversation the model cannot win.

Guidance and pitfalls

Reflection is a scalpel; the failures come from wielding it like a hammer. The recurring pitfalls are three: running too many rounds and sliding down the far side of the U-curve; letting the generator critique itself and inheriting its own blind spots; and carrying the full transcript so cost goes quadratic after benefit has flattened. The countermeasures map one-to-one: cap rounds with a stopping rule, use a separate grounded critic so errors decorrelate, and summarise each critique into a short lesson to keep context bounded. Measured against a real metric, reflection is one of the cheapest quality wins available; run on faith, it is a reliable way to spend more to get less.