YARN

YARN manages cluster resources for applications. The ResourceManager allocates resources across applications, NodeManagers manage resources and execution on individual nodes, and an ApplicationMaster coordinates a particular application.

A YARN container is a resource allocation, such as memory and CPU capacity. It does not necessarily mean a Docker container. Applications negotiate allocations and arrange their tasks within the configured framework.

Queue capacities, limits, and preemption influence contention. Creating a separate queue alone does not guarantee a response-time target. Observe waiting time, failed allocations, task behavior, and workload demand.

YARN does not store HDFS blocks or replace a catalog. Distinguish losing current work from losing new allocation or automatic recovery capability. High-availability behavior depends on the deployed coordination and state configuration.

See What Hadoop Left Behind for worked examples.

Reference: Apache documentation.