Why it matters
Prefill/decode split shapes serving. Understanding shapes optimization.
Advertisement
The architecture
Prefill: parallel over input tokens.
Decode: one token at a time.
Different scaling.
Advertisement
How it works end to end
TTFT (time to first token): prefill dominates.
ITL (inter-token latency): decode dominates.
Chunked prefill mixes.