Why it matters

Replication is the standard answer to two enterprise questions: what happens if a datacenter goes down (DR), and how do we serve reads close to users (geographic distribution)? HBase replication answers both with the same mechanism.

Replication also enables offline analytics on a secondary cluster without impacting production. Batch jobs read from the replica; the primary keeps serving traffic. This isolation is worth setting up replication even when DR is not strictly required.

Advertisement

The architecture

Replication uses the source cluster's WAL as its data source. When enabled on a table, the source RegionServer tails its own WAL and streams edits to the sink cluster. The sink applies the edits through its normal write path — HFile writes, memstore updates — as if they were client writes.

Filtering happens on the source. Only edits for replication-enabled column families are shipped. This lets you replicate part of a table without replicating everything.

HBase replication — async cross-cluster copy via WALSource clusterships WAL editsSink clusterapplies edits, catches upasync streamEventual consistency; lag varies with load; sink accepts direct writes too
Replication ships WAL edits asynchronously from source to sink cluster.
Advertisement

How it works end to end

Master-slave is the simplest topology: primary cluster accepts writes; secondary is read-only or accepts writes only for local applications. This is standard DR.

Master-master lets both clusters accept writes to the same tables. HBase uses cluster IDs to prevent replication loops: an edit from cluster A applied at cluster B will not be replicated back to A. This enables active-active geographic deployments.

Cyclic replication chains multiple clusters (A ships to B ships to C ships back to A) with the same loop-prevention. Useful for multi-region topologies with more than two datacenters.

Lag is inherent. Under normal load, lag is subsecond. Under source cluster overload or network issues, lag can grow to minutes or hours. Monitor lag as the primary health metric for replication.