Instance types

Each instance node type runs different services and performs different tasks.

The following sections describe each instance type.

Master Node

Master nodes deploy services such as the HDFS NameNode, YARN ResourceManager, and ZooKeeper Server.

Utility nodes

Utility nodes primarily deploy CM, the management service for CDP. These nodes also handle Hive Metastore, Hue, and cluster metadata. For high availability clusters, utility nodes also deploy the HDFS JournalNode and ZooKeeper Server. For high-security clusters, they install the Solr server, Ranger, and the Knox proxy.

Core nodes

Core nodes handle data storage and computation. They run the HDFS DataNode and YARN NodeManager services.

Data service nodes

Data service nodes deploy the Data Warehouse, Data Engineering, and Machine Learning services:

• Data Warehouse: This service lets you create isolated virtual data warehouses. These warehouses automatically scale based on the workload. They suspend when idle to improve resource efficiency.

• Data Engineering: This service lets you create Spark virtual clusters that scale automatically. It provides an interactive interface to submit Spark jobs and includes the Airflow service to schedule them.

• Machine Learning: This service provides a complete AI Workspace. Data scientists can use the workspace for data preparation, machine learning experiments, model testing, and model deployment. Data engineers can also use the service to write extract, transform, and load (ETL) scripts for data processing.