Min-p sampling is a truncation rule with one clever idea: instead of a fixed probability floor or a fixed cumulative mass, it sets its cutoff relative to the most likely token. The threshold is p_base × max_prob — a fraction of whatever the peak probability happens to be on this step. That single choice makes the filter adaptive: when the model is confident and its distribution is sharp, the cutoff rises and the tail is pruned hard, keeping generation near-greedy; when the model is genuinely uncertain and its distribution is flat, the cutoff falls and many candidates survive, preserving diversity. This piece derives the rule, shows why coupling the cutoff to the peak is the right instinct, walks a worked numeric example, contrasts it with top-p (nucleus) sampling, and looks at how it behaves under temperature and on a CPU-bound small language model.

What min-p actually does

Min-p is a token-filtering step that sits between the model’s softmax and the final draw. Given the probability vector p over the vocabulary, it (1) finds the largest entry max_prob = max_i p_i, (2) forms a threshold τ = p_base × max_prob using a single hyperparameter p_base (typically 0.05–0.1), (3) keeps every token whose probability is at least τ, discards the rest, and (4) renormalizes the survivors so they sum to 1 and samples one.

The crucial word is relative. p_base is not a probability floor you must clear in absolute terms; it is a fraction of the peak. A token with probability 0.04 might be kept on one step and cut on the next, depending entirely on how tall the tallest bar is. That is the whole mechanism: the bar for admission moves with the model’s own confidence rather than being pinned to a constant the way a naive probability cutoff would be.

Advertisement

The threshold, written out

Formally, min-p keeps the candidate set

max_prob = max_i p_i
τ        = p_base × max_prob         # 0 < p_base ≤ 1
keep     = { i : p_i ≥ τ }
p’_i     = p_i / Σ_{j in keep} p_j    # renormalize survivors

Read the middle line carefully. Because τ is proportional to max_prob, the survival condition p_i ≥ p_base × max_prob is equivalent to the ratio test p_i / max_prob ≥ p_base. In words: a token survives if it is at least p_base times as likely as the best token. A larger p_base demands each survivor be closer in likelihood to the peak, so it prunes harder and pushes generation toward greedy; a smaller p_base is permissive and lets more of the distribution through. Setting p_base = 0 keeps everything (pure sampling); pushing it toward 1 keeps only ties with the max.

Why scale the cutoff with confidence

The failure mode min-p is designed to fix is the fat, flat tail. A language model over tens of thousands of tokens spreads a nontrivial amount of mass across a huge number of individually tiny probabilities. Any of those tail tokens is nearly meaningless on its own, but there are so many that their combined mass is real — and if your sampler admits them, you occasionally draw one and derail the sequence.

A fixed absolute floor cannot handle both regimes at once. Set it high and you gut diversity whenever the model is legitimately unsure; set it low and you readmit junk whenever the model is confident. Min-p sidesteps the dilemma by asking a question that is meaningful in both regimes: how does this token compare to the best option right now? When one token dominates, everything an order of magnitude below it is almost certainly noise and gets cut. When no token dominates, ‘an order of magnitude below the best’ is still a respectable candidate and is kept.

Behavior at low entropy: the confident case

Entropy H = -Σ_i p_i log p_i measures how spread out the distribution is. When the model is confident about the next token — the end of a fixed phrase, a closing bracket, the obvious next word — entropy is low and one probability sits near 1. Then max_prob is large, so τ = p_base × max_prob is a high absolute bar.

Concretely, if max_prob = 0.9 and p_base = 0.1, the threshold is 0.09: every token under nine percent probability is discarded. In a peaked distribution almost nothing clears that bar except the peak itself, so min-p collapses toward greedy decoding exactly where you want determinism. This is the safety property — the model rarely gets a chance to make a low-probability mistake at moments when it actually knows the answer. The cutoff tightens itself precisely when tightening is warranted.

Behavior at high entropy: the uncertain case

Now the opposite regime. When the model is genuinely torn — the start of a creative sentence, an open-ended list, a plausible branch point — entropy is high and probability is spread across many tokens, none dominant. Here max_prob is small, so τ = p_base × max_prob is a low absolute bar, and a broad set of reasonable continuations survives.

If the top token is only max_prob = 0.15 and p_base = 0.1, the threshold is just 0.015. Every candidate above 1.5 percent stays in the running, so the sampler retains the diversity the model is signalling it wants. This is the adaptivity that a static floor cannot reproduce: the same p_base = 0.1 produced an aggressive 0.09 cutoff a moment ago and a permissive 0.015 cutoff now, purely because max_prob moved. One knob, two opposite behaviors, chosen automatically by the shape of the distribution.

A worked example

Take a step where one token is confident but a long flat tail lurks behind it. Suppose the softmax gives a top token at 0.50 and then a hundred tail tokens each at roughly 0.005 (the tail holds the remaining 0.50 of mass). With p_base = 0.1:

max_prob = 0.50
τ        = 0.1 × 0.50 = 0.05
keep     = tokens with p_i ≥ 0.05   →  only the 0.50 token
tail     = 0.005 each  < 0.05        →  all 100 discarded

Min-p keeps the single confident token and throws away the entire fat tail, even though that tail collectively held half the mass. That is the point — the mass was diffuse noise, and admitting any single piece of it would risk a derailment for little upside. If instead the top token were only 0.06 (a flat, uncertain step), the threshold would drop to 0.006 and most of that tail would survive, restoring diversity. Same rule, opposite outcome, driven by the peak.

Advertisement

Min-p versus top-p (nucleus)

Top-p keeps the smallest set of highest-probability tokens whose cumulative mass reaches p (say 0.9), then renormalizes. It targets a fixed amount of total mass; min-p targets a fixed ratio to the peak. On the fat-tail example above, top-p at 0.9 must accumulate 0.9 of mass: it takes the 0.50 token and then keeps adding tail tokens — roughly eighty of those 0.005 tokens — until the running sum crosses 0.9. It admits a large swarm of near-noise tokens precisely because they are needed to reach the mass target.

Min-p, judging each token against the peak, cuts that swarm entirely. The distinction is that top-p is mass-relative while min-p is peak-relative. Top-p can behave erratically when the distribution has a tall spike plus a heavy tail, because the tail contributes to the cumulative sum; min-p is unbothered by tail length since it never sums the tail — it only compares heights. This robustness to tail shape is min-p’s central advantage.

Interaction with temperature

Temperature T rescales the logits before softmax: p_i = softmax(z_i / T). Raising T flattens the distribution and lowers max_prob; lowering T sharpens it and raises max_prob. Because min-p’s threshold rides on max_prob, temperature and min-p compose in a stable way: as high temperature flattens the peak, τ falls in proportion, so min-p keeps pruning the same relative tail instead of suddenly flooding the candidate set.

This is the practical reason min-p is popular for creative generation at high temperature. With top-p, cranking temperature can let a lot of low-quality tokens into the nucleus, since flattening raises the tail’s share of cumulative mass. Min-p tolerates aggressive temperatures more gracefully because the cutoff is anchored to the peak, not to an absolute mass or probability. Note the order of operations: apply temperature first, then compute max_prob and τ on the temperature-scaled probabilities.

Cost on a CPU-bound SLM

Min-p is cheap, which matters when you are decoding a small language model on a CPU and every millisecond per token is visible. The work is a single pass to find max_prob, a scalar multiply for τ, a pass to mask tokens below τ, and a renormalization over the survivors — all O(V) in the vocabulary size V, with no sort.

Contrast top-p, which in its textbook form needs the probabilities in descending order to walk the cumulative sum, an O(V log V) sort of the full vocabulary (implementations often use a partial sort or selection, but it is still heavier than two linear scans). For a vocabulary of 32k–128k entries decoded token by token on a CPU, avoiding the sort is a genuine saving. Min-p gives you adaptive, quality-preserving truncation at close to the cost of an argmax, which is a favorable trade for latency-sensitive local inference.

Choosing p_base, and common pitfalls

Reasonable defaults live around p_base = 0.05–0.1. Larger values (toward 0.2) prune harder and read as more focused or repetitive; smaller values (toward 0.02) admit more of the distribution and read as more varied or risky. Because min-p handles temperature well, a common recipe is a fairly high temperature paired with a moderate p_base to get diverse but coherent text.

The pitfalls are mostly conceptual. First, do not read p_base as an absolute probability floor — a token at 0.04 is kept or cut depending on the peak, not on 0.04 alone. Second, mind the ordering: min-p must run on the temperature-scaled probabilities, not the raw ones. Third, stacking min-p on top of top-p and top-k at once is usually redundant and makes behavior hard to reason about; min-p is meant to replace them, not layer over them. Used alone, its one knob is easy to tune and its behavior is easy to predict.

Min-p sets its truncation cutoff to p_base × max_prob — a fraction of the peak probability — which is the same as keeping every token at least p_base times as likely as the best one. That relative rule makes the filter adaptive: when the model is confident the peak is tall, the cutoff rises and the fat tail is pruned toward greedy; when the model is uncertain the peak is short, the cutoff falls and diversity is preserved — one knob, two opposite behaviors chosen by the distribution’s shape. Because the threshold rides on the peak rather than on cumulative mass, min-p is robust to long tails and composes gracefully with high temperature, where top-p tends to admit noise. It is also cheaper than top-p on a CPU SLM — two linear scans and no sort. Treat p_base as a ratio, apply it after temperature, and use it in place of top-p/top-k rather than on top of them.