Core concept

spark.shuffle.push.enabled. LinkedIn contributed.

Advertisement

How it works

Newer, ecosystem support growing.

Advertisement

Trade-offs + gotchas

Large shuffles. Slow disks.