Why it matters
Interactive query workloads are where Fair Scheduler shines. Analysts running Hive queries expect their queries to start immediately, and Fair Scheduler makes that happen by cutting shares of running batch jobs down when interactive work arrives. Capacity Scheduler, in the same scenario, might make the analyst wait until the batch job releases a container.
Fair Scheduler is also the right choice when the organization thinks of users rather than queues as the unit of ownership. A cluster where 50 analysts each submit occasional queries works better with fair per-user shares than with a rigid queue tree.
The architecture
Fair Scheduler tracks the fair share of every running application: total cluster capacity divided by number of active apps, weighted by app priority. Applications running below fair share get container allocations first; applications running above fair share are preempted if needed.
Queues layer on top. A queue's fair share is total cluster capacity divided by number of active queues, weighted by queue weight. Within a queue, applications split the queue's share the same way. So a busy queue with two apps gets its full queue share, and each app gets half.
How it works end to end
When a new application submits, the scheduler adds it to the active app set and immediately recomputes fair shares for everyone. Existing applications may drop below their new fair share; the scheduler notes this and starts preempting containers from over-share applications to bring shares in line.
Preemption is aggressive by default in Fair Scheduler. If an app is starved (below its minimum share) for longer than a timeout, containers are killed from other apps until the starved app gets its share. This is what makes interactive workloads work; without it, batch jobs would hold containers forever.
Weights let administrators prioritize certain queues or users. A queue with weight 2 gets twice the share of a queue with weight 1 when both are active. This is how Fair Scheduler models priority without rigid guarantees.