An E-MapReduce (EMR) cluster has three node categories: master, core, and task. Each category runs different processes and serves a distinct role. Use this page to understand how EMR uses each node type — this is the foundation for planning cluster capacity and choosing the right Elastic Compute Service (ECS) instance specifications before you create a cluster.
Before creating a cluster, determine the ECS instance specifications for each node category. Instances of the same category form a group. After the cluster is created, you can scale out by adding ECS instances to the core node group or task node group.
Node roles at a glance
| Node category | Key processes | Stores HDFS data | Scalable dynamically |
|---|---|---|---|
| Master | NameNode (HDFS), ResourceManager (YARN) | No | No |
| Core | DataNode (HDFS), NodeManager (YARN) | Yes | Yes |
| Task | NodeManager (YARN) | No | Yes |
Master node
The master node hosts the management components of cluster services, including NameNode of Hadoop Distributed File System (HDFS) and ResourceManager of Hadoop YARN. You can access web UIs to view the running status of services in a cluster.
To submit a job, connect to the master node and run it on the command line. For connection instructions, see Log on to a cluster.
Core node
Core nodes are managed by the master node and serve a dual role: they run DataNode to store all cluster data in HDFS, and run NodeManager of Hadoop YARN to execute computing tasks.
Scale out core nodes at any time to handle growing data storage or computing workloads — scaling does not interrupt the running cluster. Core nodes support multiple storage media:
Task node
Task nodes run only computing tasks — they do not store HDFS data. Add them when core nodes can no longer handle peak computing demand, such as running Hadoop MapReduce jobs or Spark executors. Remove them when the extra capacity is no longer needed.
Adding or removing task nodes has no impact on the running cluster.