What reflection is, precisely
Reflection is a feedback loop on top of generation. A model produces a candidate answer; an evaluator inspects it and emits a verbal assessment — ‘the proof skips the base case’ — and that assessment is fed back into the context for a revised attempt. Nothing is trained; the only thing that changes between attempts is the prompt, which now carries the critique.
This is worth stating carefully, because it is easy to over-claim. The model is not improving in any lasting sense; it is being given more informative conditioning. The revision is better only insofar as the critique surfaced information the first pass failed to use. Reflection buys quality only when there is a gap between what a model can generate and what it can recognise — and that gap is what the rest of this article measures.
Reflexion: verbal reinforcement, not gradients
The canonical framing is Reflexion — treating a natural-language critique as a substitute for a gradient update. Ordinary reinforcement learning gets a scalar reward and nudges the weights. Reflexion keeps the weights frozen and instead writes a sentence of self-feedback into a memory the next attempt reads. The reinforcement is a paragraph, not a number.
Language is far higher-bandwidth than a scalar. A reward of -1 says the agent failed; the sentence ‘you passed the API arguments in the wrong order’ says why and implies the fix. The catch is that the feedback is only as good as the evaluator that wrote it — and when the evaluator is the same model that produced the error, its introspective sentence can be as wrong as the answer.
The actor, evaluator, self-reflection split
Multi-agent reflection decomposes the loop into three roles, which can be three prompts or three models. The actor generates the answer. The evaluator scores it against the task — running tests, checking a rubric, or judging quality. The self-reflection role reads both and produces the concrete lesson to carry forward.
Separating the roles matters because they demand opposite things. Generation rewards fluency and coverage; evaluation rewards skepticism and precision. One prompt asked to do both tends to rubber-stamp its own work. Splitting them lets each be tuned, and lets you drop a stronger model into the evaluator seat.
Self-critique versus cross-agent critique
The sharpest design question is who writes the critique. In self-critique the generator reviews its own output in a fresh pass. In cross-agent critique a separate critic — a different model, or the same model under a hostile prompt — reviews the generator’s work.
The difference is about error correlation. A model’s blind spots are a property of its weights and prompt, so when it grades itself its critique inherits them: the error it could not avoid making, it also cannot see. Let m be the probability the generator misses a given flaw. Under self-critique the review misses it with probability close to m too. An independent critic with a decorrelated miss-probability m′ catches far more: the flaw survives only if both miss it, roughly m × m′. That multiplicative gap is the argument for a second, independent critic over self-introspection.
The math of iterative refinement
Model the loop as error reduction. Let e_t be the residual error after round t. Suppose each round fixes a fraction ρ of the fixable error, but a floor ε stays invisible to the critique channel (real blind spots, ambiguous spec). Then:
e_(t+1) = (1 - ρ)·e_t + ρ·ε
closed form: e_t = ε + (1 - ρ)^t · (e_0 - ε)This is geometric decay toward the floor ε. The key fact falls out of (1-ρ)^t: error shrinks by the same ratio each round, so absolute gains shrink fast. With ρ = 0.5, round one removes half the fixable error, round two half the rest, round three half of that. Reflection is front-loaded — nearly all the benefit is in the first one or two rounds, and no number of rounds pushes below the floor set by what the evaluator can perceive.
A worked example
Take a first draft with error rate e_0 = 0.40, a critique that fixes ρ = 0.5 of fixable error per round, and a floor ε = 0.10. The recurrence gives:
round 0: e = 0.40
round 1: e = 0.10 + 0.5 × (0.40 - 0.10) = 0.25 (gain 0.15)
round 2: e = 0.10 + 0.5 × (0.25 - 0.10) = 0.175 (gain 0.075)
round 3: e = 0.10 + 0.5 × (0.175 - 0.10) = 0.1375 (gain 0.0375)Each round’s improvement is half the last. By round three you are within four points of the 0.10 floor and every further call buys less than two. A rational stopping rule follows: halt when marginal gain drops below the marginal cost of another round — here, around round two or three. The self-refinement literature agrees that most gains land in the first revision.
When refinement plateaus and degrades
The clean picture assumes each round only ever removes error. Reality adds a second term: reflection can inject error. A critic out of real problems starts inventing them, and the generator, primed to defer to feedback, ‘fixes’ things that were correct. Write the injected-error rate per round as δ:
e_(t+1) = (1 - ρ)·e_t + ρ·ε + δ
fixed point: e* = ε + δ / ρThe fixed point is now above the floor by δ/ρ. Small δ still converges to something good, but it tends to grow as rounds accumulate — the more polished a draft, the more a critic reaches for nits, and the higher the chance it corrupts something right. That is the origin of the U-shaped curve: quality improves, bottoms out, then worsens. Over-correction and critique-induced sycophancy are the default endgame of a loop run too long.
Both terms favour an external, grounded critic. A model grading its own work has a high ε and a high δ: without a reference it cannot tell a real flaw from a preference. A grounded evaluator — unit tests, a compiler, a retrieval check — drops ε and drives δ toward zero by reporting only real failures. The math rewards a critic that cannot hallucinate a problem.