Quantization

AWQ vs GPTQ Quantization

Two ways to get 4-bit weights — and when each one wins.

bitsandbytes vs AutoAWQ vs AutoGPTQ

Library comparison for INT4 quant.

FP8 Inference Explained

H100-era format with INT8-like speed and FP-like dynamic range.

Future Of Extreme Quantization

GGUF Format Explained

The on-disk format powering llama.cpp and local inference.

GPU Kernels for INT4 Inference

Why hardware support is moving the bar.

Guide To Quantizing With Bitsandbytes

INT8 vs INT4 Quantization

What you actually lose and what you gain.

INT8 Calibration for LLMs

SmoothQuant and the activation outlier story.

Mixed-Precision Inference

FP16 BF16 FP8 INT4 in one model.

QLoRA Fine-Tuning Explained

4-bit base + LoRA adapter = fine-tune big models cheaply.

Quantization-Aware Training

When QAT beats post-training.

Quantization Deep Dive: How 4-bit and 1.5-bit Models Retain 99% of Their Original Accuracy

Quantization Evaluation Methodology

Beyond perplexity — task-specific eval.

Quantization for Attention

KV cache attention scores special cases.

Quantization for Embeddings

Scalar binary product quantization.

SmoothQuant Intuition

Why migrating outliers works.

Speculative Decoding

Use a small model to draft, big model to verify.