What min-p actually does

Min-p is a token-filtering step that sits between the model’s softmax and the final draw. Given the probability vector p over the vocabulary, it (1) finds the largest entry max_prob = max_i p_i, (2) forms a threshold τ = p_base × max_prob using a single hyperparameter p_base (typically 0.05–0.1), (3) keeps every token whose probability is at least τ, discards the rest, and (4) renormalizes the survivors so they sum to 1 and samples one.

The crucial word is relative. p_base is not a probability floor you must clear in absolute terms; it is a fraction of the peak. A token with probability 0.04 might be kept on one step and cut on the next, depending entirely on how tall the tallest bar is. That is the whole mechanism: the bar for admission moves with the model’s own confidence rather than being pinned to a constant the way a naive probability cutoff would be.

Advertisement

The threshold, written out

Formally, min-p keeps the candidate set

max_prob = max_i p_i
τ        = p_base × max_prob         # 0 < p_base ≤ 1
keep     = { i : p_i ≥ τ }
p’_i     = p_i / Σ_{j in keep} p_j    # renormalize survivors

Read the middle line carefully. Because τ is proportional to max_prob, the survival condition p_i ≥ p_base × max_prob is equivalent to the ratio test p_i / max_prob ≥ p_base. In words: a token survives if it is at least p_base times as likely as the best token. A larger p_base demands each survivor be closer in likelihood to the peak, so it prunes harder and pushes generation toward greedy; a smaller p_base is permissive and lets more of the distribution through. Setting p_base = 0 keeps everything (pure sampling); pushing it toward 1 keeps only ties with the max.

Advertisement

Why scale the cutoff with confidence

The failure mode min-p is designed to fix is the fat, flat tail. A language model over tens of thousands of tokens spreads a nontrivial amount of mass across a huge number of individually tiny probabilities. Any of those tail tokens is nearly meaningless on its own, but there are so many that their combined mass is real — and if your sampler admits them, you occasionally draw one and derail the sequence.

A fixed absolute floor cannot handle both regimes at once. Set it high and you gut diversity whenever the model is legitimately unsure; set it low and you readmit junk whenever the model is confident. Min-p sidesteps the dilemma by asking a question that is meaningful in both regimes: how does this token compare to the best option right now? When one token dominates, everything an order of magnitude below it is almost certainly noise and gets cut. When no token dominates, ‘an order of magnitude below the best’ is still a respectable candidate and is kept.

Behavior at low entropy: the confident case

Entropy H = -Σ_i p_i log p_i measures how spread out the distribution is. When the model is confident about the next token — the end of a fixed phrase, a closing bracket, the obvious next word — entropy is low and one probability sits near 1. Then max_prob is large, so τ = p_base × max_prob is a high absolute bar.

Concretely, if max_prob = 0.9 and p_base = 0.1, the threshold is 0.09: every token under nine percent probability is discarded. In a peaked distribution almost nothing clears that bar except the peak itself, so min-p collapses toward greedy decoding exactly where you want determinism. This is the safety property — the model rarely gets a chance to make a low-probability mistake at moments when it actually knows the answer. The cutoff tightens itself precisely when tightening is warranted.

Behavior at high entropy: the uncertain case

Now the opposite regime. When the model is genuinely torn — the start of a creative sentence, an open-ended list, a plausible branch point — entropy is high and probability is spread across many tokens, none dominant. Here max_prob is small, so τ = p_base × max_prob is a low absolute bar, and a broad set of reasonable continuations survives.

If the top token is only max_prob = 0.15 and p_base = 0.1, the threshold is just 0.015. Every candidate above 1.5 percent stays in the running, so the sampler retains the diversity the model is signalling it wants. This is the adaptivity that a static floor cannot reproduce: the same p_base = 0.1 produced an aggressive 0.09 cutoff a moment ago and a permissive 0.015 cutoff now, purely because max_prob moved. One knob, two opposite behaviors, chosen automatically by the shape of the distribution.

A worked example

Take a step where one token is confident but a long flat tail lurks behind it. Suppose the softmax gives a top token at 0.50 and then a hundred tail tokens each at roughly 0.005 (the tail holds the remaining 0.50 of mass). With p_base = 0.1:

max_prob = 0.50
τ        = 0.1 × 0.50 = 0.05
keep     = tokens with p_i ≥ 0.05   →  only the 0.50 token
tail     = 0.005 each  < 0.05        →  all 100 discarded

Min-p keeps the single confident token and throws away the entire fat tail, even though that tail collectively held half the mass. That is the point — the mass was diffuse noise, and admitting any single piece of it would risk a derailment for little upside. If instead the top token were only 0.06 (a flat, uncertain step), the threshold would drop to 0.006 and most of that tail would survive, restoring diversity. Same rule, opposite outcome, driven by the peak.

Min-p versus top-p (nucleus)

Top-p keeps the smallest set of highest-probability tokens whose cumulative mass reaches p (say 0.9), then renormalizes. It targets a fixed amount of total mass; min-p targets a fixed ratio to the peak. On the fat-tail example above, top-p at 0.9 must accumulate 0.9 of mass: it takes the 0.50 token and then keeps adding tail tokens — roughly eighty of those 0.005 tokens — until the running sum crosses 0.9. It admits a large swarm of near-noise tokens precisely because they are needed to reach the mass target.

Min-p, judging each token against the peak, cuts that swarm entirely. The distinction is that top-p is mass-relative while min-p is peak-relative. Top-p can behave erratically when the distribution has a tall spike plus a heavy tail, because the tail contributes to the cumulative sum; min-p is unbothered by tail length since it never sums the tail — it only compares heights. This robustness to tail shape is min-p’s central advantage.