本文介绍服务组件在E-MapReduce不同集群中各节点的部署情况。
背景信息
查看集群服务组件具体部署信息
Hadoop集群
EMR-3.29.0版本,Hadoop集群服务组件具体部署信息:
- 必选服务
服务名 主实例节点 核心实例节点 HDFS - KMS
- SecondaryNameNode
- HttpFS
- HDFS Client
- NameNode
- DataNode
- HDFS Client
YARN - ResourceManager
- App Timeline Server
- JobHistory
- WebAppProxyServer
- Yarn Client
- Yarn Client
- NodeManager
Hive - Hive MetaStore
- HiveServer2
- Hive Client
Hive Client Spark - Spark Client
- SparkHistory
- ThriftServer
Spark Client Knox Knox 无 Tez - Tomcat
- Tez Client
Tez Client Ganglia - Gmond
- Httpd
- Gmetad
- Ganglia Client
- Gmond
- Ganglia Client
Sqoop Sqoop Client Sqoop Client Bigboot - Bigboot Client
- Bigboot Monitor
- Bigboot Client
- Bigboot Monitor
OpenLDAP OpenLDAP 无 Hue Hue 无 SmartData - Jindo Namespace Service
- Jindo Storage Service
- Jindo Client
- Jindo Storage Service
- Jindo Client
- 可选服务
服务名 主实例节点 核心实例节点 LIVY Livy 无 Superset Superset 无 Flink - FlinkHistoryServer
- Flink Client
Flink Client RANGER - RangerPlugin
- RangerAdmin
- RangerUserSync
- Solr
RangerPlugin Storm - Storm Client
- UI
- Nimbus
- Logviewer
- Storm Client
- Supervisor
- Logviewer
Phoenix Phoenix Client Phoenix Client Kudu - Kudu Master
- Kudu Client
- Kudu Tserver
- Kudu Master
- Kudu Client
HBase - HMaster
- HBase Client
- ThriftServer
- HBase Client
- HRegionServer
ZooKeeper - ZooKeeper follower
- ZooKeeper Client
- ZooKeeper follower
- ZooKeeper leader
- ZooKeeper Client
Oozie Oozie 无 Presto - Presto Client
- PrestoMaster
- Presto Client
- PrestoWorker
Impala - Impala Runtime and Shell
- Impala Catalog Server
- Impala StateStore Server
- Impala Runtime and Shell
- Impala Daemon Server
Pig Pig Client Pig Client Zeppelin Zeppelin 无 FLUME - Flume Agent
- Flume Client
- Flume Agent
- Flume Client
Druid集群
EMR-3.29.0版本,Druid集群服务组件具体部署信息:
- 必选服务
服务名 主实例节点 核心实例节点 Druid - Druid Client
- Coordinator
- Overlord
- Broker
- Router
- MiddleManager
- Historical
- Druid Client
HDFS - KMS
- SecondaryNameNode
- HttpFS
- HDFS Client
- NameNode
- DataNode
- HDFS Client
Ganglia - Gmond
- Httpd
- Gmetad
- Ganglia Client
- Gmond
- Ganglia Client
ZooKeeper - ZooKeeper follower
- ZooKeeper Client
- ZooKeeper leader
- ZooKeeper follower
- ZooKeeper Client
OpenLDAP OpenLDAP 无 Bigboot - Bigboot Client
- Bigboot Monitor
- Bigboot Client
- Bigboot Monitor
SmartData - Jindo Namespace Service
- Jindo Storage Service
- Jindo Client
- Jindo Storage Service
- Jindo Client
- 可选服务
服务名 主实例节点 核心实例节点 YARN - ResourceManager
- App Timeline Server
- JobHistory
- WebAppProxyServer
- Yarn Client
- Yarn Client
- NodeManager
Superset Superset 无
Kafka集群
EMR-3.29.0版本,Kafka集群服务组件具体部署信息:
- 必选服务
服务名 主实例节点 核心实例节点 Kafka-Manager Kafka Manager 无 Kafka - Kafka Client
- KafkaMetadataMonitor
- Kafka Rest Proxy
- Kafka Broker broker
- Kafka Schema Registry
- Kafka Broker broker
- Kafka Client
Ganglia - Gmond
- Httpd
- Gmetad
- Ganglia Client
- Gmond
- Ganglia Client
ZooKeeper - ZooKeeper follower
- ZooKeeper Client
- ZooKeeper leader
- ZooKeeper follower
- ZooKeeper Client
OpenLDAP OpenLDAP 无 - 可选服务
服务名 主实例节点 核心实例节点 RANGER - RangerPlugin
- RangerUserSync
- RangerAdmin
- Solr
RangerPlugin Knox Knox 无
ZooKeeper集群
EMR-3.29.0版本,ZooKeeper集群服务组件具体部署信息如下。
服务名 | 主实例节点 | 核心实例节点 |
---|---|---|
Ganglia | 无 |
|
ZooKeeper | 无 |
|
Dataflow集群
EMR-3.28.2版本,Dataflow集群服务组件具体部署信息:
-
必选服务
服务名 主实例节点 核心实例节点 HDFS - KMS
- SecondaryNameNode
- HttpFS
- HDFS Client
- NameNode
- DataNode
- HDFS Client
YARN - ResourceManager
- App Timeline Server
- JobHistory
- WebAppProxyServer
- Yarn Client
- Yarn Client
- NodeManager
Ganglia - Gmond
- Httpd
- Gmetad
- Ganglia Client
- Gmond
- Ganglia Client
ZooKeeper - ZooKeeper follower
- ZooKeeper Client
- ZooKeeper leader
- ZooKeeper follower
- ZooKeeper Client
Flink-Vvp Flink-Vvp 无 SmartData - JindoFS Namespace Service
- JindoFS Storage Service
- SmartData Client
- JindoFS Storage Service
- SmartData Client
Bigboot - Bigboot Client
- Bigboot Monitor
- Bigboot Client
- Bigboot Monitor
OpenLDAP OpenLDAP 无 - 可选服务
PAI-Alink:Alink
Data Science集群
EMR-3.29.1版本,Dataflow集群服务组件具体部署信息:
- 必选服务
服务名 主实例节点 核心实例节点 HDFS - HDFS Client
- KMS
- HttpFS
- NameNode
- SecondaryNameNode
- HDFS Client
- DataNode
YARN - WebAppProxyServer
- JobHistory
- App Timeline Server
- ResourceManager
- Yarn Client
- NodeManager
- Yarn Client
ZooKeeper - ZooKeeper Client
- ZooKeeper follower
- ZooKeeper Client
- ZooKeeper leader
- ZooKeeper follower
Knox Knox 无 Tensorflow on YARN - TensorFlow-On-YARN-Gateway
- TensorFlow-On-YARN-History-Server
- TensorFlow-On-YARN
- TensorFlow-On-YARN-Client
- TensorFlow-On-YARN-Gateway
SmartData - Jindo Namespace Service
- Jindo Storage Service
- Jindo Client
- Jindo Storage Service
- Jindo Client
Bigoot - Bigboot Monitor
- Bigboot Client
- Bigboot Monitor
- Bigboot Client
PAI-EASYREC Easyrec Easyrec PAI-EAS PAIEAS PAIEAS PAI-Faiss Faiss Faiss PAI-Redis Redis Redis PAI-Alink Alink 无 Flink-Vvp Flink-Vvp 无 OpenLDAP OpenLDAP 无 Jindo SDK Jindo SDK Jindo SDK - 可选服务
服务名 主实例节点 核心实例节点 Zeppelin Zeppelin 无 PAI-REC Rec 无 AUTOML AUTOML AUTOML TensorFlow TensorFlow TensorFlow
在文档使用中是否遇到以下问题
更多建议
匿名提交