Core concept

Wide dep = stage boundary + shuffle. Narrow = pipelined within stage.

Advertisement

How it works

Too many shuffles slow. Design to minimize.

Advertisement

Trade-offs + gotchas

Every performance tuning discussion.