DPM-Solver is the reason a diffusion model can turn noise into an image in ten to twenty steps instead of a thousand. Its trick is not a better network or a shortcut through the model — it is a better way to integrate the differential equation that sampling already is. The diffusion sampling ODE has a special shape: it is semi-linear, a simple linear term plus one expensive nonlinear term (the neural network). DPM-Solver solves the linear part exactly with an exponential integrator and spends its approximation budget only on the neural term. That single structural insight collapses the number of network evaluations by an order of magnitude while keeping sample quality. This piece builds the method from the ODE up: the semi-linear form, the log-SNR change of variables, the exact linear solution, how first- through third-order solvers approximate the rest, and why so few steps work.
The sampling bottleneck: NFE is the currency
Diffusion sampling starts from Gaussian noise and repeatedly denoises. The cost is dominated by one thing: the number of times you evaluate the denoising network — the number of function evaluations, or NFE. Each NFE is a full forward pass of a large UNet or diffusion transformer, so on any hardware, and painfully so on a CPU, halving the NFE roughly halves the wall-clock time.
Ancestral DDPM sampling needs hundreds to a thousand steps; DDIM brought that to roughly fifty. DPM-Solver pushes the frontier to 10–20 NFE at comparable quality, with no retraining — it works with any pretrained ε-prediction or data-prediction model. The gain is purely numerical: it comes from treating sampling as an ODE and integrating it cleverly, not from changing what the model computes. That is why it dropped straight into existing pipelines and became a default.
The probability-flow ODE
Every diffusion model defines a forward process that gradually adds noise, with a noise schedule described by two functions of time: a signal coefficient α_t and a noise coefficient σ_t, so a clean sample x_0 becomes x_t = α_t x_0 + σ_t ε. Reversing this process to generate samples can be done stochastically (an SDE) or deterministically. The deterministic route is the probability-flow ODE, which shares the same marginal distributions as the SDE at every time but follows a smooth trajectory.
Written in terms of the model’s noise prediction ε_θ(x_t, t), that ODE takes the form dx_t/dt = f(t) x_t + (g(t)^2 / 2σ_t) ε_θ(x_t, t), where f and g are fixed by the schedule. Sampling is then just solving this ODE backward in time from t = T down to t = 0. The question is which numerical solver to use.
A semi-linear ODE hiding in plain sight
The naive answer is a generic solver — Euler, or Runge–Kutta. That is exactly what DDIM and early samplers effectively do, and it is wasteful. Look again at the ODE: dx_t/dt = f(t) x_t + h(t) ε_θ(x_t, t). The first term, f(t) x_t, is linear in the state x_t and involves no network. The second term is the only place the expensive, nonlinear neural network appears.
An equation of the form ‘linear part plus nonlinear part’ is called semi-linear. A general-purpose solver ignores this structure and discretizes the whole right-hand side at once, paying approximation error on both terms. But the linear term is elementary — we can write its contribution in closed form. Every unit of error we spend approximating something we could have solved exactly is a step wasted. DPM-Solver is built entirely around not wasting it.
A change of variables to log-SNR
To exploit the structure, DPM-Solver changes the independent variable from time t to the log signal-to-noise ratio, λ_t = log(α_t / σ_t). As sampling proceeds from pure noise toward data, the signal-to-noise ratio rises monotonically, so λ increases smoothly and can serve as a clean clock for the integration.
Under this substitution the messy schedule-dependent coefficients simplify dramatically. The linear term is absorbed into an integrating factor, and the whole ODE can be rewritten so that its solution is an exponentially weighted integral of the network output against λ. This is the pivot of the entire method: by measuring progress in λ rather than t, the part that was hard to integrate becomes an ordinary integral of a smooth function, and the part that was linear becomes an exact prefactor.
Solving the linear part exactly
With the log-SNR clock, the exact solution from a time s to a later time t can be written down without any discretization of the linear term:
x_t = (α_t / α_s) · x_s
− α_t · ∫[λ_s → λ_t] e^(−λ) · ε̂_θ(x̂_λ, λ) dλThe first term, (α_t/α_s) x_s, is the exact contribution of the linear part — no approximation, no error. All the remaining difficulty is packed into the single integral, where ε̂_θ is the network expressed as a function of λ. Notice the e^(−λ) weight: it is a known analytic function, so it too can be handled exactly. The only thing we cannot compute in closed form is the network’s value along the path.
The exponential integrator
This is the exponential-integrator idea. For a semi-linear ODE y’ = a y + N(t), the classical move is to multiply by an integrating factor e^(−at), which makes the linear part integrate exactly and leaves ∫ e^(−at) N(t) dt. You then approximate only N — here, only the neural term — while the exponential weighting stays analytic.
Concretely, DPM-Solver approximates ε̂_θ(x̂_λ, λ) by its Taylor expansion in λ around the current step, then integrates that polynomial against e^(−λ) in closed form. Because the hard weight is integrated exactly and only a smooth, slowly varying function is approximated, the error per step is far smaller than a black-box solver that discretizes everything. Keeping more Taylor terms yields a higher-order solver with correspondingly smaller error — the knob that trades network evaluations for accuracy.
First order recovers DDIM
Take the crudest approximation: assume ε̂_θ is constant over the step, equal to its value at the start. Plugging that into the integral and doing the e^(−λ) integration by hand gives the first-order update:
h = λ_t − λ_s
x_t = (α_t / α_s) x_s − σ_t (e^h − 1) ε_θ(x_s, s)This is exactly the DDIM update. So DDIM is nothing more than the first-order member of the DPM-Solver family — the exponential integrator with a zeroth-order Taylor approximation of the network. That is a reassuring sanity check and a precise statement of what higher orders buy you: DDIM already solves the linear part exactly; it just approximates the neural term with a single constant, which is why it needs many small steps to stay accurate.
Higher-order solvers
The path to fewer steps is to model how ε̂_θ changes across the step rather than pretending it is constant. A second-order solver keeps the linear term of the Taylor expansion, which requires an estimate of the derivative dε̂_θ/dλ; a third-order solver adds the quadratic term.
Those derivative estimates come from extra evaluations of the network at intermediate points, or from finite differences of past evaluations. The payoff is that error shrinks much faster with step size: a k-th order solver has local error O(h^(k+1)), so doubling the order lets you take far coarser steps for the same fidelity. In practice second and third order hit the sweet spot — enough accuracy to sample in a dozen-odd steps, without the instability and diminishing returns that higher orders bring at very low NFE.
Singlestep versus multistep
There are two ways to get the derivative information a high-order step needs. A singlestep solver evaluates the network at fresh intermediate points within the current step (much like Runge–Kutta stages), spending several NFE per step but keeping each step self-contained. A multistep solver instead reuses the network outputs from previous steps and forms a finite-difference estimate of the derivative — so a second-order multistep step costs just one new NFE.
At the low step counts that matter here, multistep usually wins. It extracts high-order accuracy at one evaluation per step, and it tends to be more stable when steps are large. The ratio of consecutive step sizes, r_i = h_{i−1} / h_i, appears in the finite-difference coefficients so the estimate stays correct even on a non-uniform schedule. The popular 2M and 3M variants are exactly these second- and third-order multistep solvers.
DPM-Solver++ for guided sampling
The original DPM-Solver approximates the noise prediction ε_θ. Guided generation — classifier-free guidance with a large scale — amplifies that noise term, and at high guidance the extrapolation can overshoot and produce saturated, artifact-heavy samples at low step counts. DPM-Solver++ fixes this by reformulating the same exponential integrator around the data prediction x_θ instead of the noise.
In the data-prediction form the linear prefactor becomes a ratio of σ values, and the Taylor/finite-difference machinery is applied to x_θ, which is far better-behaved under strong guidance. This keeps the samples in range and pairs naturally with dynamic thresholding. DPM-Solver++(2M) — second-order multistep on the data prediction — is the workhorse behind many modern text-to-image samplers, precisely because it stays robust where the noise-prediction version would break down.
Sampling in 10 to 20 steps
Putting it together explains the headline number. Because the linear part is exact and only a smooth neural term is approximated — to second or third order, reusing past evaluations — each step covers a large stretch of the trajectory with little error. Ten to twenty steps become enough where naive solvers needed hundreds.
Two choices sharpen the result at these low counts. First, the step schedule: spacing steps uniformly in λ (log-SNR), or on a Karras-style sigma curve, distributes error far better than uniform-in-t spacing. Second, order at the boundaries: solvers often drop to lower order on the final step or two, where σ → 0 can cause order reduction. With a sensible schedule and a 2M/3M solver, 15 steps routinely match what DDIM needs 50-plus steps to achieve.
CPU and small-model implications
For CPU inference and small local models, the NFE reduction is the whole game. A forward pass of the denoiser is the dominant cost, and it does not get cheaper on a CPU; the only lever that reliably helps is calling it fewer times. Going from 50 DDIM steps to 15 DPM-Solver++ steps is a direct 3× wall-clock speedup with no change to the model and no retraining.
That property makes DPM-Solver ideal for constrained deployment: it is a drop-in replacement at inference time, adds negligible arithmetic of its own (the exponential-integrator coefficients are a handful of scalar operations per step), and its memory footprint is trivial next to the network. When you cannot afford a GPU, spending your engineering effort on the sampler rather than the model is often the highest-leverage move available.
Common pitfalls
A few traps recur. Pushing the order too high at very low NFE backfires: singlestep third-order solvers can overshoot on large steps, and the multistep variants are usually the safer default under 20 steps. Using the noise-prediction solver with strong guidance invites saturation artifacts — reach for DPM-Solver++ (data prediction) whenever the guidance scale is high.
Ignoring the schedule wastes the method’s accuracy; uniform-in-time steps concentrate error badly, so use log-SNR or Karras spacing. And order reduction near t = 0 is real: the last steps, where σ is tiny, are where samplers commonly lower the order or add correction to avoid blur. None of these require touching the model — they are all choices in how you integrate, which is exactly the surface DPM-Solver gives you to tune.