Why architecture matters here
BucketCache matters because it enables HBase to have a large block cache without the garbage-collection pain of an on-heap cache -- solving the GC-pressure problem that limits the on-heap cache at scale. HBase's block cache (caching HFile blocks) is important for read performance (repeated reads hitting the cache, not the disk). But the on-heap cache (LruBlockCache) has a scaling problem: a large on-heap cache (many GB -- lots of Java objects) creates enormous GC pressure (long, unpredictable GC pauses -- hurting HBase's latency and stability) -- so you can't have a large on-heap cache without GC pain (limiting the cache size, and thus the read performance). BucketCache solves this (moving the cached data off-heap or to SSD -- not Java objects on the heap -- drastically reducing the GC pressure) -- enabling a large cache (for good read performance) with stable, predictable GC. So BucketCache is what lets HBase cache at scale (a large cache -- good read performance -- without the GC pain). For HBase deployments needing a large block cache (read-heavy workloads at scale), BucketCache is important, and understanding it (how it avoids the on-heap GC pain) is understanding how HBase caches at scale.
The off-heap-avoids-GC insight is the core, and it's the key to a large cache without GC pain. The GC problem with the on-heap cache: the cached blocks are Java objects on the JVM heap -- so a large cache (many GB of cached blocks) means a large heap with many objects, which the JVM's garbage collector must manage (scanning, collecting -- causing long, unpredictable GC pauses for a large heap). These GC pauses hurt HBase (latency spikes, instability). BucketCache avoids this by storing the cached data off-heap (in off-heap memory -- managed directly by HBase, not the JVM heap -- so it's not Java objects the GC manages) or on SSD (a file). So the large cache (the bulk of the cached data -- off-heap) doesn't burden the JVM heap/GC (it's off-heap -- outside the GC's scope) -- so the GC pressure is drastically reduced (the heap is small -- just HBase's working objects, not the large cache -- so the GC pauses are short and predictable). This off-heap storage (the cache off the JVM heap -- not GC-managed) is the key: it enables a large cache (the data off-heap) without the GC pain (the heap small, GC stable). Understanding the off-heap-avoids-GC core (storing the cache off-heap -- outside the GC's scope -- enabling a large cache with stable GC) is understanding how BucketCache solves the on-heap cache's GC problem.
And the tiered-L1-L2-plus-modes design is what makes BucketCache efficient and flexible. BucketCache is typically used in a tiered setup and with flexible storage modes. Tiered L1/L2: a small on-heap L1 cache (for the index/metadata -- and sometimes the hottest data -- kept on-heap for fast access, but small -- so limited GC impact) plus the large off-heap L2 BucketCache (for the bulk of the cached data -- off-heap, GC-free). This tiering balances speed (the L1 on-heap index/hot data -- fast) with scale (the L2 off-heap bulk -- large, GC-free) -- the best of both (fast access to the index/hot data, a large GC-free cache for the bulk). Modes: BucketCache can use different storage -- offheap (off-heap RAM -- fast, GC-free -- for a large in-memory cache) or file (a file on SSD -- a large, cheaper cache tier -- SSD-backed, even larger than RAM but slower). So you can choose the mode (off-heap RAM for speed, SSD for a larger/cheaper cache) -- flexible per the needs and hardware (a large off-heap RAM cache, or an even-larger SSD cache). This tiered (L1 on-heap index/hot, L2 off-heap bulk) and multi-mode (off-heap RAM or SSD) design makes BucketCache efficient (the tiering balancing speed and scale) and flexible (the modes for different needs/hardware). Understanding the tiered-L1-L2-plus-modes design (the tiering for speed and scale, the modes for RAM/SSD flexibility) is understanding how BucketCache is efficient and flexible.
The architecture: every piece explained
Top row: the problem and caches. The problem: an on-heap cache creates GC pressure (a large cache -- many Java objects -- long, unpredictable GC pauses). Block cache: caching HFile blocks in memory (so repeated reads hit the cache, not the disk -- read performance). LruBlockCache: the default on-heap block cache (stores cached blocks on the JVM heap -- GC-heavy at scale). BucketCache: the off-heap (or SSD) block cache (stores the cached data off the JVM heap -- drastically reducing GC pressure -- enabling a large cache).
Middle row: the design. Buckets by size: BucketCache organizes the off-heap memory into buckets (fixed-size slots -- blocks stored in appropriately-sized buckets -- efficient allocation, avoiding fragmentation) -- the storage organization. L1 + L2 tiers: a small on-heap L1 (the index/metadata and hottest data -- fast, on-heap) plus the large off-heap L2 BucketCache (the bulk of the cached data -- off-heap, GC-free) -- the tiered setup. Modes: offheap (off-heap RAM -- fast, GC-free) or file (a file on SSD -- a large, cheaper cache -- SSD-backed) -- the storage modes. Reduced GC: the result -- a large cache without the GC pain (the cache off-heap -- the heap small -- stable, predictable GC pauses) -- the key benefit.
Bottom rows: warming and sizing. Cache-on-write + prefetch: warming the cache -- cache-on-write (caching blocks as they're written -- so they're cached for subsequent reads) and prefetch (proactively loading blocks into the cache -- e.g., on region open) -- populating the cache proactively (for a warm cache -- good hit ratio). Sizing + hit ratio: sizing the cache (the BucketCache size -- larger caches more data -- higher hit ratio -- but more memory/SSD) and the hit ratio (the cache's effectiveness -- the fraction of reads hitting the cache) -- the sizing/effectiveness tradeoff. The ops strip: sizing (sizing the BucketCache -- the off-heap RAM or SSD size -- for the working set and the hit-ratio/memory tradeoff), hit ratio (monitoring the cache hit ratio -- the effectiveness; a low hit ratio indicating an undersized cache or poor locality), and GC monitoring (monitoring the GC -- confirming the reduced GC pressure -- stable, predictable pauses -- the key benefit of BucketCache -- and the overall HBase latency/stability).
End-to-end flow
Trace BucketCache enabling a large cache without GC pain. An HBase deployment is read-heavy (needing a large block cache for good read performance). With the on-heap LruBlockCache, a large cache (say 30GB) would put 30GB of cached blocks on the JVM heap -- creating enormous GC pressure (the JVM managing the 30GB+ heap -- long, unpredictable GC pauses -- latency spikes, instability). With BucketCache: the bulk of the cache (the 30GB) is stored off-heap (in off-heap memory -- not Java objects on the heap) -- so the JVM heap stays small (just HBase's working objects and the small L1 index -- not the 30GB cache) -- and the GC pressure is drastically reduced (the small heap -- short, predictable GC pauses). A small on-heap L1 holds the index/metadata (fast access) and the large off-heap L2 BucketCache holds the cached blocks (GC-free). So the deployment gets the large 30GB cache (good read performance) with stable, predictable GC (the cache off-heap -- not burdening the JVM) -- the BucketCache enabling the large cache without the GC pain (which the on-heap cache couldn't). The off-heap storage gave the large cache with stable GC.
The tiering and modes vignettes show the design. A tiering case: the team uses the tiered setup -- a small on-heap L1 (holding the block index/metadata -- accessed frequently, kept on-heap for speed -- but small, so limited GC impact) plus the large off-heap L2 BucketCache (the bulk of the cached data blocks -- off-heap, GC-free) -- balancing fast access (the L1 on-heap index) with a large GC-free cache (the L2 off-heap bulk). The tiering gave both speed and scale. A modes case: the team considers the modes -- for a large in-memory cache, offheap mode (off-heap RAM -- fast, GC-free); for an even-larger cache (beyond RAM), file mode (a file on SSD -- a large, cheaper cache tier -- SSD-backed, larger but slower than RAM) -- choosing the mode per their needs (off-heap RAM for speed, SSD for a larger cache) and hardware. The mode choice matched the cache to the needs/hardware.
The GC-monitoring and sizing vignettes complete it. A GC-monitoring case: the team monitors the GC (confirming the BucketCache's benefit -- the GC pauses are short and predictable, versus the long unpredictable pauses the large on-heap cache would cause) -- verifying the reduced GC pressure (the key benefit -- HBase's latency/stability improved by the off-heap cache). The GC monitoring confirmed the benefit. A sizing case: the team sizes the BucketCache (the off-heap RAM or SSD size) -- for the working set (large enough to cache the frequently-read data -- a good hit ratio -- but balanced against the memory/SSD cost) -- and monitors the hit ratio (the cache's effectiveness -- a low hit ratio indicating an undersized cache or poor locality, prompting a larger cache) -- sizing the cache for the hit-ratio/cost balance. The consolidated discipline the team documents: use BucketCache for a large block cache without GC pain (storing the cached data off-heap or on SSD -- not Java objects on the heap -- drastically reducing GC pressure -- enabling a large cache with stable GC -- versus the on-heap LruBlockCache's GC pain at scale), use the tiered L1/L2 setup (small on-heap L1 index/hot data for speed, large off-heap L2 for the bulk -- balancing speed and scale), choose the mode (offheap RAM for speed, file/SSD for a larger cheaper cache), warm the cache (cache-on-write, prefetch -- for a good hit ratio), size the cache (for the working set and the hit-ratio/cost tradeoff), monitor the hit ratio (effectiveness) and the GC (confirming the reduced pressure -- the key benefit) -- because BucketCache enables HBase to have a large block cache without the GC pain of an on-heap cache (storing the cached data off-heap/SSD -- outside the GC's scope), solving the GC-pressure problem that limits the on-heap cache at scale.