The score-function view of diffusion

Diffusion sampling is easiest to reason about through the score: the gradient of the log-density, ∇_x log p(x). It points uphill toward regions of higher data probability, so a sampler that repeatedly nudges a noisy x along the score walks it from noise onto the data manifold. A trained noise-prediction network ε_θ(x_t, t) is exactly a scaled, sign-flipped score estimate:

ε_θ(x_t, t) ≈ -σ_t · ∇_x log p(x_t)

where σ_t is the noise level at step t. For conditional generation we sample from p(x | c) — images given a caption, audio given a speaker — so we need the conditional score ∇_x log p(x | c). Everything in CFG is about building that conditional score from pieces a single network can supply, so keep the score-vs-ε correspondence in mind: whatever we do to scores, we do the sign-flipped version to ε predictions.

Advertisement

Bayes rule and the implicit classifier

The conditional density factors by Bayes’ rule: p(x | c) ∝ p(x) · p(c | x). Take the log-gradient of both sides — the normalizer drops out because it does not depend on x:

∇_x log p(x | c) = ∇_x log p(x) + ∇_x log p(c | x)

The first term is the unconditional score (how to make any realistic sample); the second is the classifier gradient (how to make the sample more consistent with c). Classifier guidance literally trains a separate classifier p(c | x_t) and differentiates it. CFG’s insight is to rearrange the same equation to eliminate that classifier:

∇_x log p(c | x) = ∇_x log p(x | c) − ∇_x log p(x)

The classifier gradient equals the conditional score minus the unconditional score. If one network can produce both scores, we get the classifier for free — no auxiliary model, no noisy-image classifier to train, no separate gradients at sampling time.

Advertisement

Joint training with condition dropout

To supply both scores, CFG trains a single network to be conditional and unconditional at once. During training, with some probability p_uncond (commonly 10–20%), the condition is replaced by a special null token ∅ — an empty caption, a learned ‘no-condition’ embedding. The loss is the ordinary denoising objective:

L = E[ ‖ ε − ε_θ(x_t, t, c) ‖^2 ]   with c := ∅ w.p. p_uncond

The model thus learns ε_θ(x_t, t, c) for real conditions and ε_θ(x_t, t, ∅) for the marginal, sharing every weight; the condition enters through cross-attention or an added embedding, and passing ∅ switches the model into unconditional mode. No architectural change is needed — CFG is a training recipe plus a sampling formula, which is why it retrofits onto almost any conditional diffusion model.

The guidance equation

Now assemble the guided score. We want to sample not from p(x | c) but from a sharpened distribution p(x) · p(c | x)^s that raises the classifier term to a power s ≥ 1. Its score is:

∇ log p_s = ∇ log p(x) + s · ∇ log p(c | x)
        = ∇ log p(x) + s ( ∇ log p(x|c) − ∇ log p(x) )
        = (1 − s) ∇ log p(x) + s · ∇ log p(x|c)

Translating to noise predictions (flip the sign, absorb σ_t) gives the formula every sampler actually runs:

ε̃ = ε_θ(x_t, ∅) + s · ( ε_θ(x_t, c) − ε_θ(x_t, ∅) )

It is a straight-line extrapolation: start at the unconditional prediction, then step s times the vector pointing from unconditional toward conditional. At s = 1 it collapses to the plain conditional prediction; at s = 0 it is purely unconditional; at s > 1 it overshoots past the conditional prediction, exaggerating whatever the condition contributes.

Two conventions for the guidance scale

The literature uses two guidance parameters and they are easy to confuse. The interpolation form above uses a scale s where s = 1 is unguided. The original Ho & Salimans (2022) paper writes it with a weight w:

ε̃ = (1 + w) · ε_θ(x_t, c) − w · ε_θ(x_t, ∅)

Expand it and you find the identical line with s = 1 + w. So Ho’s w = 0 is unguided (pure conditional), whereas the guidance-scale s = 0 is unconditional. Stable Diffusion’s famous default guidance_scale = 7.5 is the s convention, equivalent to w = 6.5. When you read a formula, check which endpoint recovers the plain conditional model — that tells you which convention is in play. The algebra is the same; only the zero-point of the knob moves.

A worked numerical example

Take a single pixel-channel direction and pretend the network outputs 2-vectors. Suppose at some step ε_θ(x_t, c) = (2.0, −1.0) and ε_θ(x_t, ∅) = (1.5, −0.5). The guidance direction is:

d = ε(c) − ε(∅) = (2.0 − 1.5,  −1.0 − (−0.5)) = (0.5, −0.5)

With s = 7.5:

ε̃ = ε(∅) + 7.5 · d
   = (1.5, −0.5) + 7.5 · (0.5, −0.5)
   = (1.5 + 3.75,  −0.5 − 3.75) = (5.25, −4.25)

Notice how much larger the guided prediction is than either input: guidance did not average the two, it amplified their difference. The magnitude jumped from about 2.1 to about 6.8. That inflation is the whole mechanism — and, as we will see, also the source of CFG’s characteristic failure mode when s is pushed too high.

Why guidance sharpens the distribution

Raising p(c | x)^s to a power greater than one makes the classifier term peakier: samples that the implicit classifier considers strongly on-prompt get relatively far more probability mass, and ambiguous ones get suppressed. In distribution terms, guidance reduces variance and sharpens modes — it concentrates sampling onto the images that most decisively satisfy c.

This is exactly the fidelity-versus-diversity trade. Higher s yields images that adhere more tightly to the prompt and look crisper, at the cost of variety: outputs collapse toward a few canonical interpretations and rare valid compositions vanish. Lower s keeps samples diverse and natural but loosely conditioned. There is no universally correct value; you choose where to sit on a curve trading prompt adherence against sample entropy, depending on whether the task values obedience or exploration.

Plugging the guided score into a sampler

CFG changes only which noise estimate you feed the sampler; the DDPM or DDIM update rule is untouched. At each step you compute the two predictions, blend them with the guidance formula, and hand ε̃ to the ordinary update:

for t = T .. 1:
    e_c = eps(x_t, t, c)          # conditional pass
    e_u = eps(x_t, t, ∅)          # unconditional pass
    e   = e_u + s * (e_c - e_u)    # guided estimate
    x_{t-1} = sampler_step(x_t, e, t)

Because the guidance blend happens on the raw noise prediction, CFG is orthogonal to the choice of sampler — ancestral DDPM, deterministic DDIM, DPM-Solver, and friends all accept a guided ε without modification. Some implementations rescale ε̃ back toward the conditional prediction’s norm (‘guidance rescaling’) to fight the magnitude blow-up seen in the worked example, but the core loop is unchanged.

Shapes, batching, and the 2x cost

The price of CFG is stated in the loop above: two forward passes per step, one conditional and one unconditional, versus one for plain sampling. Naively that doubles inference cost. In practice you batch them — stack the conditional and null inputs into one tensor of shape [2B, C, H, W] and run a single forward pass, then split the output back into e_c and e_u:

x_in = concat([x_t, x_t])          # [2B, ...]
c_in = concat([c,   ∅])           # conditional + null
e_c, e_u = split( eps(x_in, t, c_in) )

Batching hides the latency behind parallelism on a GPU, but does not reduce the work: you still evaluate the network twice, doubling FLOPs and, in memory-bound settings, roughly doubling weight and activation traffic. The 2x factor is intrinsic to needing both scores — the single most important cost fact about CFG.