Core concept
80GB HBM3 (3TB/s) or 96GB (H100 NVL). NVLink 4 (900 GB/s). PCIe Gen5.
Advertisement
How it works
Peak: 989 TFLOPS BF16 sparse, 1979 FP8 sparse. Actual ~50-70% for LLMs.
Advertisement
Trade-offs + gotchas
Foundation model training + inference through 2025. Cloud availability now common.