Why it matters
Most YARN debugging happens at the NodeManager level. When a container gets killed for exceeding memory limits, the RM says nothing useful; you have to read NodeManager logs to see the actual kill signal. When a job cannot start on a specific host, the NM's health checker is usually why. When localization is slow, the NM's localizer is the bottleneck.
Getting the NM configuration right also has huge cluster-wide effects. Under-reserving OS overhead leads to OOM kills of critical processes. Over-reserving wastes capacity that could run work. A properly tuned NM makes the difference between a stable cluster and one that flails.
The architecture
The container manager subsystem receives launch requests from the ResourceManager, spawns processes inside cgroups with the requested memory and CPU limits, and streams container status back. It uses either the default Linux container executor or the Docker container executor depending on cluster configuration.
The localizer subsystem downloads any files an application depends on (JARs, config files, uploaded scripts) from HDFS to a local staging area before the container starts. Localized files are cached per-application-user and per-application depending on their visibility, so subsequent containers for the same user do not re-download.
How it works end to end
The container lifecycle starts when the NM receives a StartContainerRequest RPC from the ResourceManager. It parses the request, verifies the security tokens, ensures resources are within the container's allocation, calls the localizer to pull dependencies, and then the container executor spawns the actual process inside a cgroup.
Once running, the NM periodically samples container resource usage via cgroup metrics. If memory exceeds the container's allocation, the NM sends SIGTERM followed by SIGKILL. If CPU exceeds allocation, cgroup CPU throttling kicks in but the container is not killed.
When the container exits (normally or via kill), the NM cleans up cgroups, removes non-persistent files, releases the resources back to its available pool, and reports completion back to the RM on the next heartbeat.