Core concept
Simulates larger batch without more memory. Effective for memory-constrained training.
Advertisement
How it works
N steps compute time increases proportionally.
Advertisement
Trade-offs + gotchas
Standard when needing effective batch size > GPU memory allows.