Core concept
HBM3: 900 GB/s per stack. H100 with 5 stacks = 3.35 TB/s. HBM3e (H200, B200): higher capacity + bandwidth.
Advertisement
How it works
Expensive. Limited supply. Manufacturing complex (TSV + interposer). Bottleneck for GPU production.
Advertisement
Trade-offs + gotchas
Training + inference at scale. Everywhere consumer cards fall short.