Why it matters

Custom high-perf matmul kernels require CUTLASS or equivalent. Understanding enables extreme optimization.

Advertisement

The architecture

Template hierarchy: device → grid → block → warp → mma.

Composability: swap components (epilogue, tile size).

CUTLASS structureTemplatescomposable levelsTensor core opswmma/mma builtinsEpiloguesactivation, add, quantizeUsed to build FlashAttention, mixed-precision matmul, custom fused ops
CUTLASS building blocks.
Advertisement

How it works end to end

Epilogues: apply activation or fuse operations at end of matmul.

Precision: FP8, FP16, BF16, INT8, INT4 supported.

Used by: FlashAttention, TensorRT-LLM, custom kernels.