Why it matters
Distillation data enables small models. Understanding shapes strategy.
Advertisement
The architecture
Teacher LLM generates responses / reasoning.
Student LLM trains via SFT + distillation loss.
Advertisement
How it works end to end
Response generation + optional reasoning.
Soft target logits when available.
Domain coverage.