本节介绍服务进程在 E-MapReduce 不同集群中各节点的部署情况。

E-MapReduce 集群创建时，会在实例类型中部署不同的服务角色，例如 Hadoop HDFS 中的 NameNode 部署在 Master 节点中。对于大规模集群，可以通过引导操作实现自定义的部署和分配，例如独立部署 ZooKeeper 集群，采用独立的阿里云 RDS 作为 Hive MetaStore 存储。默认的详细服务部署信息如下：

Hadoop 集群

Hadoop 集群服务组件具体部署信息
  • 必选服务
    组件名 主实例节点 核心实例节点
    HDFS
    • KMS
    • SecondaryNameNode
    • HttpFS
    • HDFS Client
    • NameNode
    • DataNode
    • HDFS Client
    YARN
    • ResourceManager
    • App Timeline Server
    • JobHistory
    • WebAppProxyServer
    • Yarn Client
    • Yarn Client
    • NodeManager
    Hive
    • Hive MetaStore
    • HiveServer2
    • Hive Client
    		 Hive Client
    Spark
    • Spark Client
    • SparkHistory
    • ThriftServer
    		 Spark Client
    Knox Knox N/A
    Zeppelin Zeppelin N/A
    Tez
    • Tomcat
    • Tez Client
    		 Tez Client
    Ganglia
    • Gmond
    • Httpd
    • Gmetad
    • Ganglia Client
    • Gmond
    • Ganglia Client
    Pig Pig Client Pig Client
    Sqoop Sqoop Client Sqoop Client
    Bigboot
    • Bigboot Client
    • Bigboot Monitor
    • Bigboot Client
    • Bigboot Monitor
    OpenLDAP OpenLDAP N/A
    Hue Hue N/A
  • 可选服务
    组件名 主实例节点 核心实例节点
    Flume
    • Flume Agent
    • Flume Client
    • Flume Agent
    • Flume Client
    Livy Livy N/A
    Superset Superset N/A
    Flink
    • FlinkHistoryServer
    • Flink Client
    		 Flink Client
    Ranger
    • RangerPlugin
    • RangerAdmin
    • RangerUserSync
    		 RangerPlugin
    Storm
    • Storm Client
    • UI
    • Nimbus
    • Logviewer
    • Storm Client
    • Supervisor
    • Logviewer
    Phoenix Phoneix Client Phoneix Client
    Analytics Zoo
    • Analytics-Zoo-Scala
    • Analytics-Zoo-Python
    • Analytics-Zoo-Scala
    • Analytics-Zoo-Python
    SmartData
    • JindoFS Namespace Service
    • JindoFS Storage Service
    • SmartData Client
    • JindoFS Storage Service
    • SmartData Client
    Kudu
    • Kudu Master
    • Kudu Client
    • Kudu Tserver
    • Kudu Master
    • Kudu Client
    HBase
    • HMaster
    • HBase Client
    • ThriftServer
    • HBase Client
    • HRegionServer
    ZooKeeper
    • ZooKeeper follower
    • ZooKeeper Client
    • ZooKeeper leader
    • ZooKeeper follower
    • ZooKeeper Client
    Oozie Oozie N/A
    Presto
    • Presto Client
    • PrestoMaster
    • Presto Client
    • PrestoWorker
    Impala
    • Impala Runtime and Shell
    • Impala Catalog Server
    • Impala StateStore Server
    • Impala Runtime and Shell
    • Impala Daemon Server
    • Impala StateStore Server

Druid 集群

Druid 集群服务组件具体部署信息
  • 必选服务
    组件名 主实例节点 核心实例节点
    Druid
    • Druid Client
    • Coordinator
    • Overlord
    • Broker
    • MiddleManager
    • Historical
    • Druid Client
    HDFS
    • KMS
    • SecondaryNameNode
    • HttpFS
    • HDFS Client
    • NameNode
    • DataNode
    • HDFS Client
    Ganglia
    • Gmond
    • Httpd
    • Gmetad
    • Ganglia Client
    • Gmond
    • Ganglia Client
    ZooKeeper
    • ZooKeeper follower
    • ZooKeeper Client
    • ZooKeeper leader
    • ZooKeeper follower
    • ZooKeeper Client
    OpenLDAP OpenLDAP N/A
  • 可选服务
    组件名 主实例节点 核心实例节点
    YARN
    • ResourceManager
    • App Timeline Server
    • JobHistory
    • WebAppProxyServer
    • Yarn Client
    • Yarn Client
    • NodeManager
    Superset Superset N/A

Kafka 集群

Kafka 集群服务组件具体部署信息
  • 必选服务
    组件名 主实例节点 核心实例节点
    Kafka-Manager Kafka Manager N/A
    Kafka
    • Kafka Client
    • KafkaMetadataMonitor
    • Kafka Rest Proxy
    • Kafka Broker controller
    • Kafka Schema Registry
    • Kafka Broker broker
    • Kafka Client
    Ganglia
    • Gmond
    • Httpd
    • Gmetad
    • Ganglia Client
    • Gmond
    • Ganglia Client
    ZooKeeper
    • ZooKeeper follower
    • ZooKeeper Client
    • ZooKeeper leader
    • ZooKeeper follower
    • ZooKeeper Client
  • 可选服务
    组件名 主实例节点 核心实例节点
    Ranger
    • RangerPlugin
    • RangerUserSync
    • RangerAdmin
    		 RangerPlugin
    Knox Knox N/A
    OpenLDAP OpenLDAP N/A

ZooKeeper 集群

ZooKeeper 集群服务组件具体部署信息
组件名 主实例节点 核心实例节点
Ganglia N/A
  • Gmond
  • Httpd
  • Gmetad
  • Ganglia Client
ZooKeeper N/A
  • ZooKeeper leader
  • ZooKeeper Client
  • ZooKeeper follower

Data Science 集群

Data Science 集群服务组件具体部署信息
  • 必选服务
    组件名 主实例节点 核心实例节点
    HDFS
    • KMS
    • SecondaryNameNode
    • HttpFS
    • HDFS Client
    • NameNode
    • DataNode
    • HDFS Client
    Jupyter Jupyter Jupyter
    Ganglia
    • Gmond
    • Httpd
    • Gmetad
    • Ganglia Client
    • Gmond
    • Ganglia Client
    ZooKeeper
    • ZooKeeper follower
    • ZooKeeper Client
    • ZooKeeper leader
    • ZooKeeper follower
    • ZooKeeper Client
    Analytics Zoo
    • Analytics-Zoo-Python
    • Analytics-Zoo-Scala
    • Analytics-Zoo-Python
    • Analytics-Zoo-Scala
    ApacheDS ApacheDS N/A
    Tensorflow on YARN
    • TensorFlow-On-YARN
    • TensorFlow-On-YARN-Gateway
    • TensorFlow-On-YARN-History-Server
    • TensorFlow-On-YARN-Client
    • TensorFlow-On-YARN-Gateway
    Zeppelin Zeppelin N/A
    Spark
    • Spark Client
    • ThriftServer
    • SparkHistory
    		 Spark Client
    YARN
    • ResourceManager
    • App Timeline Server
    • JobHistory
    • WebAppProxyServer
    • Yarn Client
    • Yarn Client
    • NodeManager
  • 可选服务
    组件名 主实例节点 核心实例节点
    Hue Hue RangerPlugin
    Hive
    • HiveServer2
    • Hive MetaStore
    • Hive Client
    		 Hive Client
    Tensorflow Tensorflow Tensorflow