The absmax int8 primitive

Everything starts with one idea: map a range of floats onto the 256 integers an 8-bit signed value can hold. bitsandbytes uses absmax quantization. For a block of values b, take the largest magnitude a = max_i |b_i|, form a scale c = 127 / a, and store q_i = round(c · b_i) as an int8 in [-127, 127]. To read a value back you divide out the scale: b_i ≈ q_i / c.

c   = 127 / max_i |b_i|      (one fp16/fp32 scale)
q_i = round(c * b_i)         (int8, in [-127, 127])
b_i ~= q_i / c               (dequantize)

The format is symmetric (zero maps to zero, no separate zero-point) and uniform: the 255 levels are evenly spaced across [-a, +a]. The worst-case rounding error per value is half a step, a / 254 ≈ a/255. That single fact is the whole tension of the method: the error floor is set by the block’s largest element, so anything that inflates a without raising the typical value degrades every value in the block.

Advertisement

Why one big outlier wrecks a whole-tensor scale

Suppose you quantize an entire weight matrix with a single absmax. Neural network tensors are famously heavy-tailed: most values sit within, say, [-1, 1], but a handful of outlier features spike to 30 or 100. Absmax is driven by that maximum. If a = 100, the step size is 100/127 ≈ 0.79 — so every ordinary weight near 0.3 rounds to 0 or ±1. The bulk of the tensor collapses onto a few integer levels and its precision is destroyed, all to faithfully represent a couple of extreme values.

This is not a corner case; it is the normal behavior of transformer activations and of optimizer state during training. A per-tensor scale forces the common mass and the rare tail to share one dynamic range, and the tail always wins. You could clip the outliers, but clipping throws away information that often matters. The better move is to stop letting one outlier speak for the whole tensor — which is exactly what block-wise quantization does.

Advertisement

Block-wise absmax: a scale per block

Block-wise quantization flattens the tensor and splits it into contiguous blocks of fixed size B (bitsandbytes uses B = 2048 for optimizer states, and similar sizes elsewhere). Each block computes its own absmax a_k = max_{i in block k} |b_i| and is quantized independently against it. Instead of one scale for N values you now keep N / B scales, one per block.

tensor  ->  [ block_0 | block_1 | ... | block_{N/B-1} ]
each block k:   a_k = max|b|,   c_k = 127/a_k,   q = round(c_k * b)
store:  N int8 codes  +  (N/B) fp32 absmax scales

Because every block is normalized against its own maximum, the blocks are independent: they can be quantized and dequantized fully in parallel, which maps cleanly onto GPU threads and is cheap on CPU too. The cost is the extra scales, but they are tiny (quantified below). The payoff is that the dynamic range is now local — and locality is what tames the tail.

Why blocks bound outlier damage

Here is the key argument. An outlier inflates the absmax of one block of B values, not the whole tensor. With N = 4,194,304 weights and B = 2048, a lone spike corrupts the resolution of at most 2048 values — roughly 0.05% of the tensor. The other 2047 blocks never see that outlier; each keeps its own tight absmax (maybe a_k ≈ 1) and therefore a fine step of 1/127 ≈ 0.008, preserving full int8 resolution for the bulk of the data.

Contrast the two error floors directly. Whole-tensor: every value pays a_global / 254, dragged up by the global max. Block-wise: a value pays only a_k / 254, set by its neighbors. Since outliers are sparse and clustered, almost every block has a small local max, so almost every value gets a small error. Block-wise quantization does not remove outliers — it quarantines them, capping the blast radius of each spike to a single block. That is why it is robust enough to quantize quantities that a per-tensor scheme cannot touch.

Linear vs dynamic 8-bit formats

Absmax fixes the range; the format decides how the 255 codes are spaced inside it. bitsandbytes offers two. The linear format spaces levels uniformly — simple, and fine when values fill the range evenly. But weights, gradients, and Adam moments are concentrated near zero with a long tail, and uniform spacing wastes most of its codes on large magnitudes that rarely occur while starving the crowded region near zero.

The dynamic format fixes this with a non-linear, dynamic-exponent code. Each 8-bit value splits its bits between an exponent (how big) and a fraction (how precise), and the split floats: small numbers spend more bits on the exponent to get many closely spaced levels near zero, while large numbers trade fraction bits for reach. The result is high relative precision across many orders of magnitude from a single byte — a small floating-point-like code rather than a fixed grid. Paired with block-wise normalization, dynamic quantization is what lets 8 bits stand in for 32-bit optimizer state without derailing training.

8-bit optimizers: the memory Adam hides

Adam’s memory cost is easy to underestimate. For every parameter it keeps two 32-bit buffers: the first moment m (the running mean of the gradient) and the second moment v (the running mean of the squared gradient). At 4 bytes each that is 8 bytes of optimizer state per parameter — twice the size of the fp32 model itself. For large models this state, not the weights, is what overflows memory.

8-bit optimizers store m and v as block-wise, dynamically quantized int8 — 1 byte each instead of 4. Crucially the update math stays in 32-bit. On each step the relevant block is dequantized to fp32, the standard Adam update is applied, and the fresh state is re-quantized back to 8-bit for storage. So the moving averages accumulate at full precision moment to moment; only the resting representation is compressed. bitsandbytes ships drop-in classes (Adam8bit, AdamW8bit, and others) that swap in for the 32-bit versions, plus a stable embedding layer that normalizes the embedding — a known instability source — to keep large-scale training well-behaved.