Advertisement
Calibration data should match production. Wrong domain = bad quantized model.
What you're seeing
Post-training quantization needs ~512-1000 sample inputs to find activation ranges. Calibration data should match production distribution.
Wrong calibration: ranges set too narrow → clipping → quality drop. Or too wide → wasted precision. Domain-fine-tuned models need domain calibration data.
★ KEY TAKEAWAY
Quantization calibration data must match production. Wrong domain = clipped or wasted ranges = quality drop.
▶ WHAT TO TRY
- Switch between English / Code / Multilingual calibration.
- See how activation range varies by domain.