Why architecture matters here
Without RSGroups, HBase’s stochastic balancer treats every RegionServer as interchangeable and every region as movable anywhere. That is the right default for a single-tenant cluster and exactly wrong for a shared one: the balancer will happily co-locate a 2TB analytics table’s regions with a 50GB OLTP table’s regions, and from that moment the OLTP table’s tail latency is a function of the analytics workload’s scan schedule. The alternatives are all worse. Running separate clusters per tenant multiplies HMaster, ZooKeeper, and operational overhead, strands capacity in each silo, and turns every cross-tenant join or copy into a cross-cluster replication project. Request quotas (RPC throttling) limit damage but act at the wrong layer — by the time a scan is throttled, it has already polluted the block cache and dirtied the OS page cache.
RSGroups matter architecturally because they move the isolation boundary to the assignment layer, which is the only layer that controls physical placement. Every downstream resource — block cache, memstore heap, compaction threads, handler pools, disk spindles on the RegionServer — is isolated automatically once regions are pinned, because those resources are per-process and the processes are now dedicated. The design also preserves the single-cluster properties that matter: snapshots, replication, and bulk loads work unchanged; security policy is administered once; and capacity can be rebalanced between tenants with a single move_servers_rsgroup command instead of a hardware migration. The one resource RSGroups do not isolate is HDFS — DataNode disks and network remain shared — which is why understanding the boundary of the guarantee is as important as using it.
The feature’s lineage explains its shape. RSGroups came out of Yahoo’s multi-thousand-node HBase estate (HBASE-6721), where the practical alternative on the table was not ‘better quotas’ but ‘seventy separate clusters’ — and the operational arithmetic of seventy HMasters, seventy ZooKeeper ensembles, and seventy security reviews settled the argument. It also explains what RSGroups deliberately are not: they are not a scheduler, not a fair-share system, and not a QoS layer. There is no notion of weight, priority, or burst credit; membership is binary and placement is absolute. That crudeness is a feature — the guarantee is easy to reason about precisely because there is no algorithm between the operator’s intent and the region’s location. Teams that need finer-grained sharing inside a group still layer RPC quotas, request priorities, and read-replica patterns on top; the group boundary is the outer wall, not the whole building.
The architecture: every piece explained
Group metadata. Group definitions live in the hbase:rsgroup system table: each row maps a group name to its member servers (host:port) and member tables or namespaces. Because the HMaster cannot read a user-space table before regions are online — a chicken-and-egg problem at startup — the metadata is mirrored into a ZooKeeper znode (/hbase/rsgroup); the master bootstraps from the mirror, then treats the table as the source of truth.
RSGroupAdminEndpoint. RSGroups ship as a master coprocessor (built-in and enabled via hbase.balancer.rsgroup.enabled in modern HBase 2.x). It exposes the admin RPCs (add_rsgroup, move_servers_rsgroup, move_tables_rsgroup, balance_rsgroup) and hooks table creation so a table’s regions are assigned only to its group’s servers from the first assignment.
The group-aware balancer. RSGroupBasedLoadBalancer wraps the real balancer (usually the StochasticLoadBalancer). It partitions the cluster state by group, runs the inner balancer independently on each partition, and vetoes any plan that would move a region across a group boundary. Balancing the online group never touches the batch group.
Assignment and recovery. The assignment manager consults group membership on every assignment decision, including crash recovery: when a RegionServer dies, its regions are reassigned only to surviving members of the same group. If a group has no live servers, its regions stay unassigned (RIT) rather than spilling onto other tenants’ hardware — a deliberate choice of unavailability over isolation breach.
The default group. Every server and table not explicitly grouped belongs to default. It is the elastic pool: new servers join it on registration, and it absorbs everything unpinned.
Two refinements round out the mechanism. First, fallback grouping: newer releases offer hbase.rsgroup.fallback.enable, which lets regions of an exhausted group be assigned temporarily to default-group servers instead of sitting in RIT — a deliberate inversion of the isolation-over-availability default for teams whose SLAs read the other way; when the group recovers, the balancer pulls the refugees home. Second, procedure-based moves: in HBase 2.x the server and table move operations run as master procedures with persisted state in the procedure WAL, so a master crash mid-move resumes rather than orphans the operation — an important property when a single move_servers_rsgroup can imply hundreds of region transitions. Group configuration can also carry per-group balancer tuning, letting the batch group run aggressive move thresholds while the online group barely tolerates movement at all.
End-to-end flow
Follow a tenant onboarding end to end. An operator creates a group and populates it: add_rsgroup ‘online’, then move_servers_rsgroup with three server addresses. Each server move is itself a small workflow: the master marks the server’s current regions for movement, drains them to the servers remaining in the source group (usually default), flips the membership row in hbase:rsgroup, updates the ZooKeeper mirror, and only then considers the server a member of online. The server itself is oblivious — group membership is entirely a master-side concept.
Next, move_tables_rsgroup ‘online’ with the payment tables (or better, move_namespaces_rsgroup so future tables in the tenant’s namespace inherit the pin). The master computes the set of regions now on wrong-group servers and issues standard region moves: close on the old server (flush memstore, drop locality), update hbase:meta, open on a group member. Clients notice nothing but a brief region-in-transition blip per region; their meta cache repopulates on the first NotServingRegionException retry.
From then on, steady state is group-scoped. A write to a payment table lands on RS1–RS3 only; its memstore flushes, compactions, and block-cache churn touch only those three heaps. The nightly balance_rsgroup ‘batch’ evens out the analytics regions across RS4–RS5 without generating a single move in the online group. When RS2 crashes, the SCP (ServerCrashProcedure) replays its WALs and reassigns its regions to RS1 and RS3; the batch group’s spare capacity is invisible to the recovery, by design. Capacity changes are the same commands in reverse: promote a fresh server from default into online before a traffic peak, return it afterwards, and let each group’s balancer settle the regions.
Rolling maintenance shows the same group-scoped discipline. A per-group rolling restart walks only that group’s servers: drain RS1 (regions redistribute to RS2 and RS3 — never outside), restart, rejoin, next server. The blast radius of a bad configuration push is one tenant class, and a canary group — a small group holding low-stakes tables that takes every change first — becomes a natural deployment stage. Meta lookups are unaffected throughout: clients resolve regions via hbase:meta exactly as before, because RSGroups changed where regions live, not how they are found. The only client-visible artifact of the whole machinery is that a tenant’s regions happen to always resolve to the same subset of servers — which is, of course, the entire point.