Why it matters

DeepSpeed is major training library. Understanding shapes tooling.

Advertisement

The architecture

ZeRO stages 1-3.

Pipeline + tensor parallelism.

Inference: MII + kernels.

DeepSpeed capabilitiesZeROstages 1-3Pipeline + tensor par3D parallelismInference + MIIoptim kernelsMicrosoft Research; used by Bloom + MT-NLG + others; production-ready
DeepSpeed.
Advertisement

How it works end to end

ZeRO variants (Offload + Infinity).

3D parallelism.

MII inference.

Config-driven.