Core concept
Practical: over-train on data (100+ tokens/param) for cheaper inference. Trade training vs inference cost.
Advertisement
How it works
Data quality bottleneck. High-quality tokens finite.
Advertisement
Trade-offs + gotchas
Model size + data budget planning.