Core concept

Off-heap via spark.memory.offHeap.enabled.

Advertisement

How it works

OOM crashes tasks. Careful sizing.

Advertisement

Trade-offs + gotchas

Every prod job. Especially cache-heavy.