Why it matters
Every scaling limit in YARN passes through the ResourceManager. If it slows down, container allocation slows down for every application. If it fails without HA, the entire cluster stops scheduling work until it comes back. Getting it right — right sized, right configured, right monitored — is not optional at production scale.
Scheduler choice also has huge organizational implications. Capacity scheduler with hierarchical queues encodes org-chart resource ownership. Fair scheduler treats users more egalitarian. The choice reflects real policy about how compute gets shared.
The architecture
Internally the RM has two main modules. The Scheduler is a pluggable component (typically Capacity or Fair) that maps resource requests to available cluster capacity. It does not care about application semantics — it just matches container requests to nodes. The ApplicationsManager handles application submission, launches the ApplicationMaster for each app in its first container, monitors the AM, and restarts it if it fails.
The Capacity Scheduler organizes cluster capacity into a tree of queues. Each queue has a configured minimum capacity (guaranteed) and a maximum capacity (elastic cap). Applications submitted to a queue can borrow unused capacity from sibling queues but never exceed the parent's max.
How it works end to end
When a client submits an application, the ApplicationsManager assigns it an application ID, picks a node with capacity for the AM container, and instructs the target NodeManager to launch the AM. The AM then negotiates its own resource needs with the scheduler over the ApplicationMasterProtocol.
The scheduler receives resource requests from the AM specifying container memory, vcores, node preferences (locality), and priority. It matches these against unused capacity in the AM's queue plus any borrowable capacity from siblings. When a match is found, the scheduler allocates a container and tells the AM which NM will host it.
HA is achieved by running two RMs backed by a ZooKeeper quorum. One is active; the other is standby, tailing scheduler state. On active failure, standby takes over. Application state is preserved in the state store, so in-flight applications survive failover with only a brief pause.