Why it matters

Bloom filters give the largest single speedup for HBase point queries. On tables where each row appears in only a few HFiles (typical for time-series and update-heavy workloads), bloom filters skip 90 percent or more of HFiles per query. Point-lookup latency drops from tens of milliseconds to sub-millisecond.

Bloom filters also reduce load on HDFS. Each skipped HFile is one fewer HDFS read. On busy clusters this dramatically reduces DataNode I/O contention.

Advertisement

The architecture

A bloom filter is a fixed-size bitmap with several hash functions. To insert a key, you hash it with each function and set the corresponding bits. To query, you hash the key and check whether all corresponding bits are set. If any bit is missing, the key is definitely absent. If all bits are set, the key might be present (or might be a false positive).

Each HFile carries its own bloom filter. When a query arrives, the RegionServer checks each HFile's bloom filter first. HFiles that fail the check are skipped entirely; only HFiles that pass are actually read.

Bloom filters — probabilistic HFile skippingGet(row=X)check bloom of each HFileHFile bloom'no' → skip; 'maybe' → readSkips 90%+ of HFiles when only a few actually contain each row — huge read savings
Bloom filter check: 'no' skips the HFile; 'maybe' triggers a full read.
Advertisement

How it works end to end

Two bloom filter configurations exist per column family. ROW builds a bloom filter over row keys only. This is efficient for queries that fetch entire rows or specific columns. ROWCOL builds a bloom filter over row+column pairs. This is more precise for queries that fetch specific columns from specific rows, at the cost of larger bloom filters.

Bloom filter size and false positive rate are tunable. Larger filters have lower false positive rates but consume more memory. Default false positive rate is 1 percent; you can push it to 0.1 percent by roughly doubling the filter size.

Bloom filters are stored in the HFile itself, so they must be loaded when the HFile is opened. The bloom filter cache holds them in memory. If the cache is undersized, bloom filter checks fall back to loading from HDFS, defeating the optimization.