NVIDIA published TensorRT-LLM 1.3.0 on November 20, 2026 at 02:45 UTC. Its documented limitations make configuration-specific validation essential.
More cache paths mean more configuration choices
The release adds cache compression, NVFP4 KV cache, disk cache, per-conversation block reuse and connectors for FlexKV and LMCache. Rubin support is explicitly preview. These are release descriptions; no throughput or memory improvement was measured for this digest.
Serving mode changes the documented limitations
The notes say DSpark draft KV cache is not transferred to the generation worker in disaggregated serving, reducing acceptance rate; aggregated serving is unaffected by that issue. They also flag potentially incorrect output for disaggregated Qwen3-32B-FP8 on B300. A feature's presence therefore does not establish correctness for every model and serving combination.
The tagged license is also configuration-relevant: the project generally uses Apache 2.0, with specified kernel directories subject to NVIDIA terms and additional third-party licenses. It should not be described as uniformly Apache-licensed.
A proposed comparison that changes one cache choice
Original exercise, not executed: pin one supported model checkpoint, tokenizer, GPU, prompt length, output limit and concurrency. Use synthetic prompts to compare a baseline cache configuration with one new option. Record peak memory, time to first token, decode latency and output checks.
Keep the same fixture order and record cache state so repeated prompts do not silently change the comparison. In a separate pair of runs, change only aggregated versus disaggregated serving. Treat an output-check failure as a failed configuration, even if its latency is lower.
This design would isolate a local tradeoff. It would not reproduce the entire release's supported-model matrix or validate its preview hardware paths.
Sources accessed November 21, 2026. Views are my own and do not represent any employer, past or present.
Sources and evidence
Evaluate cache changes within a pinned model, GPU and serving mode, with correctness checks alongside memory and latency.