Core concept
Read-only from tasks. Cached on node between tasks in same job.
Advertisement
How it works
Small files ideal. Large files hurt startup + disk.
Advertisement
Trade-offs + gotchas
Lookup tables. UDF jars. Model files.
Read-only from tasks. Cached on node between tasks in same job.
Small files ideal. Large files hurt startup + disk.
Lookup tables. UDF jars. Model files.