Why it matters
For any initial data load or backfill of significant size, bulk load is dramatically faster than client puts. A billion-row backfill via puts might take days; via bulk load it takes hours or less. The difference is enough that bulk load is the default answer for anything bigger than a few million rows.
Bulk load also avoids stress on the write path. Client puts fill memstores, trigger flushes, and stress compaction. Bulk load produces already-optimized HFiles that need no further compaction. The cluster stays healthy during ingest.
The architecture
The workflow has two stages. Stage one is a MapReduce (or Spark) job that reads source data and writes sorted HFiles. The output HFiles must match the target table's region boundaries — each HFile must belong entirely to one region.
Stage two is the completebulkload tool. It moves each HFile from the job's output directory into the appropriate region's HDFS directory, using HDFS atomic move semantics. Once moved, the RegionServer serving that region notices the new HFile and includes it in future reads.
How it works end to end
HFile production uses the HFileOutputFormat2 in MapReduce. This output format handles the sorting and partitioning to match target region boundaries automatically. The reducer count must equal the number of target regions, so that each reducer produces one HFile per region.
The completebulkload tool is a small utility that iterates over produced HFiles and issues an HDFS rename per HFile. Because HDFS renames are metadata-only, this step is very fast even for large HFile sets.
Bulk load bypasses the WAL. If the target table has replication enabled, bulk-loaded data is not automatically replicated. You must run bulk load on the target of replication separately, or copy the HFiles over.