Core concept

Requires InfiniBand or RoCE NIC + supported CPU/chipset. NCCL uses.

Advertisement

How it works

Not all hardware supports. Careful setup + configuration.

Advertisement

Trade-offs + gotchas

Every distributed training node.