Core concept
Compute cost proportional to active params. Memory to total. Trade compute for memory.
Advertisement
How it works
Load balancing challenging. Some experts hot, others cold.
Advertisement
Trade-offs + gotchas
Cost-effective inference at high total-param scale. DeepSeek-V3 671B total, ~37B active.