Core concept
Speedups from prefix sharing + speculative decoding + structured decoding.
Advertisement
How it works
Newer. Feature set catching up with vLLM. RadixAttention is unique benefit.
Advertisement
Trade-offs + gotchas
Structured LLM programs. Multi-turn conversations. High prefix reuse.