Core concept
NVIDIA's Transformer Engine library. Per-tensor scaling. Requires careful integration.
Advertisement
How it works
Not everything FP8-safe. Norms + softmax stay higher precision. Some models degrade.
Advertisement
Trade-offs + gotchas
Frontier LLM training. Meta uses FP8 for Llama 3. Google Grok 3 too.