创建E-MapReduce集群时,不同类型集群的实例节点上会部署不同的服务角色。例如,Hadoop HDFS中的NameNode部署在主实例节点中。本文为您介绍如何查看E-MapReduce集群中服务组件在各节点的部署信息。
前提条件
已创建集群,详情请参见创建集群。
操作步骤
Hadoop集群
以EMR-3.29.0版本为例,Hadoop集群服务组件的具体部署信息如下。
- 必选服务
服务名 主实例节点 核心实例节点 HDFS - KMS
- SecondaryNameNode
- HttpFS
- HDFS Client
- NameNode
- DataNode
- HDFS Client
YARN - ResourceManager
- App Timeline Server
- JobHistory
- WebAppProxyServer
- Yarn Client
- Yarn Client
- NodeManager
Hive - Hive MetaStore
- HiveServer2
- Hive Client
Hive Client Spark - Spark Client
- SparkHistory
- ThriftServer
Spark Client Knox Knox 无 Tez - Tomcat
- Tez Client
Tez Client Ganglia - Gmond
- Httpd
- Gmetad
- Ganglia Client
- Gmond
- Ganglia Client
Sqoop Sqoop Client Sqoop Client Bigboot - Bigboot Client
- Bigboot Monitor
- Bigboot Client
- Bigboot Monitor
OpenLDAP OpenLDAP 无 Hue Hue 无 SmartData - Jindo Namespace Service
- Jindo Storage Service
- Jindo Client
- Jindo Storage Service
- Jindo Client
- 可选服务
服务名 主实例节点 核心实例节点 LIVY Livy 无 Superset Superset 无 Flink - FlinkHistoryServer
- Flink Client
Flink Client RANGER - RangerPlugin
- RangerAdmin
- RangerUserSync
- Solr
RangerPlugin Storm - Storm Client
- UI
- Nimbus
- Logviewer
- Storm Client
- Supervisor
- Logviewer
Phoenix Phoenix Client Phoenix Client Kudu - Kudu Master
- Kudu Client
- Kudu Tserver
- Kudu Master
- Kudu Client
HBase - HMaster
- HBase Client
- ThriftServer
- HBase Client
- HRegionServer
ZooKeeper - ZooKeeper follower
- ZooKeeper Client
- ZooKeeper follower
- ZooKeeper leader
- ZooKeeper Client
Oozie Oozie 无 Presto - Presto Client
- PrestoMaster
- Presto Client
- PrestoWorker
Impala - Impala Runtime and Shell
- Impala Catalog Server
- Impala StateStore Server
- Impala Runtime and Shell
- Impala Daemon Server
Pig Pig Client Pig Client Zeppelin Zeppelin 无 FLUME - Flume Agent
- Flume Client
- Flume Agent
- Flume Client
Druid集群
以EMR-3.29.0版本为例,Druid集群服务组件的具体部署信息如下。
- 必选服务
服务名 主实例节点 核心实例节点 Druid - Druid Client
- Coordinator
- Overlord
- Broker
- Router
- MiddleManager
- Historical
- Druid Client
HDFS - KMS
- SecondaryNameNode
- HttpFS
- HDFS Client
- NameNode
- DataNode
- HDFS Client
Ganglia - Gmond
- Httpd
- Gmetad
- Ganglia Client
- Gmond
- Ganglia Client
ZooKeeper - ZooKeeper follower
- ZooKeeper Client
- ZooKeeper leader
- ZooKeeper follower
- ZooKeeper Client
OpenLDAP OpenLDAP 无 Bigboot - Bigboot Client
- Bigboot Monitor
- Bigboot Client
- Bigboot Monitor
SmartData - Jindo Namespace Service
- Jindo Storage Service
- Jindo Client
- Jindo Storage Service
- Jindo Client
- 可选服务
服务名 主实例节点 核心实例节点 YARN - ResourceManager
- App Timeline Server
- JobHistory
- WebAppProxyServer
- Yarn Client
- Yarn Client
- NodeManager
Superset Superset 无
Dataflow-Kafka集群
以EMR-3.29.0版本为例,Dataflow-Kafka集群服务组件的具体部署信息如下。
- 必选服务
服务名 主实例节点 核心实例节点 Kafka-Manager Kafka Manager 无 Kafka - Kafka Client
- KafkaMetadataMonitor
- Kafka Rest Proxy
- Kafka Broker broker
- Kafka Schema Registry
- Kafka Broker broker
- Kafka Client
Ganglia - Gmond
- Httpd
- Gmetad
- Ganglia Client
- Gmond
- Ganglia Client
ZooKeeper - ZooKeeper follower
- ZooKeeper Client
- ZooKeeper leader
- ZooKeeper follower
- ZooKeeper Client
OpenLDAP OpenLDAP 无 - 可选服务
服务名 主实例节点 核心实例节点 RANGER - RangerPlugin
- RangerUserSync
- RangerAdmin
- Solr
RangerPlugin Knox Knox 无
Flink集群
以EMR-3.30.0版本为例,Flink集群服务组件的具体部署信息如下。
-
必选服务
服务名 主实例节点 核心实例节点 HDFS - KMS
- SecondaryNameNode
- HttpFS
- HDFS Client
- NameNode
- DataNode
- HDFS Client
YARN - ResourceManager
- App Timeline Server
- JobHistory
- WebAppProxyServer
- Yarn Client
- Yarn Client
- NodeManager
Ganglia - Gmond
- Httpd
- Gmetad
- Ganglia Client
- Gmond
- Ganglia Client
ZooKeeper - ZooKeeper
- ZooKeeper Client
- ZooKeeper
- ZooKeeper Client
Knox Knox 无 Flink-Vvp Flink-Vvp 无 OpenLDAP OpenLDAP 无 - 可选服务
服务名 主实例节点 核心实例节点 PAI-Alink Alink 无
ZooKeeper集群
以EMR-3.29.0版本为例,ZooKeeper集群服务组件的具体部署信息如下。
服务名 | 主实例节点 | 核心实例节点 |
---|---|---|
Ganglia | 无 |
|
ZooKeeper | 无 |
|
Data Science集群
以EMR-3.29.1版本为例,Data Science集群服务组件的具体部署信息如下。
- 必选服务
服务名 主实例节点 核心实例节点 HDFS - HDFS Client
- KMS
- HttpFS
- NameNode
- SecondaryNameNode
- HDFS Client
- DataNode
YARN - WebAppProxyServer
- JobHistory
- App Timeline Server
- ResourceManager
- Yarn Client
- NodeManager
- Yarn Client
ZooKeeper - ZooKeeper Client
- ZooKeeper follower
- ZooKeeper Client
- ZooKeeper leader
- ZooKeeper follower
Knox Knox 无 Tensorflow on YARN - TensorFlow-On-YARN-Gateway
- TensorFlow-On-YARN-History-Server
- TensorFlow-On-YARN
- TensorFlow-On-YARN-Client
- TensorFlow-On-YARN-Gateway
SmartData - Jindo Namespace Service
- Jindo Storage Service
- Jindo Client
- Jindo Storage Service
- Jindo Client
Bigoot - Bigboot Monitor
- Bigboot Client
- Bigboot Monitor
- Bigboot Client
PAI-EASYREC Easyrec Easyrec PAI-EAS PAIEAS PAIEAS PAI-Faiss Faiss Faiss PAI-Redis Redis Redis PAI-Alink Alink 无 Flink-Vvp Flink-Vvp 无 OpenLDAP OpenLDAP 无 Jindo SDK Jindo SDK Jindo SDK - 可选服务
服务名 主实例节点 核心实例节点 Zeppelin Zeppelin 无 PAI-REC Rec 无 AUTOML AUTOML AUTOML TensorFlow TensorFlow TensorFlow
ClickHouse集群
以EMR-3.35.0版本为例,ClickHouse集群服务组件的具体部署信息如下。
服务名 | 主实例节点 | 核心实例节点 |
---|---|---|
Ganglia |
|
|
ZooKeeper |
|
|
ClickHouse |
|
|
EMR Studio集群
以EMR-3.33.111版本为例,EMR Studio集群服务组件的具体部署信息如下。
服务名 | 主实例节点 | 核心实例节点 |
---|---|---|
Ganglia |
|
|
Zeppelin | Zeppelin Master | Zeppelin Worker |
Knox | Knox | 无 |
JupyterHub | JupyterHub | |
OpenLDAP | OpenLDAP | |
RabbitMQ | RabbitMQ | |
Mysql | MySQL | |
Airflow |
|
|
Studio Workspace | Studio Workspace | 无 |