Core concept
Import from ONNX or use frameworks (torch-tensorrt). Engine specific to GPU architecture.
Advertisement
How it works
Compilation slow (minutes). Some ops unsupported. Debugging harder.
Advertisement
Trade-offs + gotchas
Production inference. Latency-critical. Maximum throughput.