Why architecture matters here

Precision failures come from underflow + kernel mismatch. Architecture matters because format + accumulator + hardware combine.

Advertisement

The architecture: every piece explained

The top strip is formats. fp32. bf16. fp16. fp8.

The middle row is composition. fp4 / int4. Accumulator. Mixed precision. Numerics safety.

The lower rows are ops. Kernel matrix. Metrics. Ops — training vs inference targeting.

GPU precision — fp32/bf16/fp16/fp8/fp4 + accumulator + mixed precision trainingthe number formats that decide throughputfp32master + controlbf16same range as fp32fp16IEEE + loss scalingfp8 (E4M3/E5M2)Hopper+ inference + trainingfp4 / int4Blackwell + inferenceAccumulatoralways higher precisionMixed precisionstorage vs computeNumerics safetyloss scaling / stochastic roundingKernel matrixper hardwareMetricsTFLOPS + underflow rateOps — training vs inference targetingextremesafecomposeprotectselectwatchwatchoperateoperate
GPU precision zoo + mixed precision recipes.
Advertisement

End-to-end flow

End-to-end: training run in bf16 with fp32 accumulator + fp32 master weights. Loss scaling on. On Hopper, fp8 for GEMM with fp32 accumulator. Inference on Blackwell in fp4 with fp16 accumulator. TFLOPS metrics guide tuning.