Why it matters
vLLM benchmarks shape config choices. Understanding shapes tuning.
Advertisement
The architecture
vLLM repo benchmarks/ folder.
Latency + throughput + serving scripts.
Standard sharegpt-like traces.
Advertisement
How it works end to end
benchmark_serving.py.
benchmark_throughput.py.
benchmark_latency.py.
Configuration sweeps.