组件角色部署_集群规划_集群规划与配置

背景信息

创建E-MapReduce集群时，不同类型的实例节点（实例类型）上会部署不同的服务角色，例如，Hadoop HDFS中的NameNode部署在Master节点中。如果是大规模集群，您可通过引导操作实现自定义部署和分配，例如，独立部署ZooKeeper集群和采用独立的阿里云RDS作为Hive MetaStore存储。

查看集群服务组件具体部署信息

登录阿里云E-MapReduce控制台。
在顶部菜单栏处，选择地域（Region）。
单击上方的集群管理。
在集群管理页面，单击相应集群所在行的详情。
在左侧导航栏，单击集群服务，选择查看的集群服务。
单击部署拓扑，可查看服务组件的具体部署信息。

Hadoop集群

Hadoop集群服务组件具体部署信息

必选服务


服务名	主实例节点	核心实例节点
HDFS	KMS SecondaryNameNode HttpFS HDFS Client NameNode	DataNode HDFS Client
YARN	ResourceManager App Timeline Server JobHistory WebAppProxyServer Yarn Client	Yarn Client NodeManager
Hive	Hive MetaStore HiveServer2 Hive Client	Hive Client
Spark	Spark Client SparkHistory ThriftServer	Spark Client
Knox	Knox	N/A
Zeppelin	Zeppelin	N/A
Tez	Tomcat Tez Client	Tez Client
Ganglia	Gmond Httpd Gmetad Ganglia Client	Gmond Ganglia Client
Pig	Pig Client	Pig Client
Sqoop	Sqoop Client	Sqoop Client
Bigboot	Bigboot Client Bigboot Monitor	Bigboot Client Bigboot Monitor
OpenLDAP	OpenLDAP	N/A
Hue	Hue	N/A
SmartData	Jindo Namespace Service Jindo Storage Service Jindo Client	Jindo Storage Service Jindo Client

可选服务


服务名	主实例节点	核心实例节点
Flume	Flume Agent Flume Client	Flume Agent Flume Client
Livy	Livy	N/A
Superset	Superset	N/A
Flink	FlinkHistoryServer Flink Client	Flink Client
Ranger	RangerPlugin RangerAdmin RangerUserSync	RangerPlugin
Storm	Storm Client UI Nimbus Logviewer	Storm Client Supervisor Logviewer
Phoenix	Phoenix Client	Phoenix Client
Analytics Zoo	Analytics-Zoo-Scala Analytics-Zoo-Python	Analytics-Zoo-Scala Analytics-Zoo-Python
Kudu	Kudu Master Kudu Client	Kudu Tserver Kudu Master Kudu Client
HBase	HMaster HBase Client ThriftServer	HBase Client HRegionServer
ZooKeeper	ZooKeeper ZooKeeper Client	ZooKeeper leader ZooKeeper follower ZooKeeper Client
Oozie	Oozie	N/A
Presto	Presto Client PrestoMaster	Presto Client PrestoWorker
Impala	Impala Runtime and Shell Impala Catalog Server Impala StateStore Server	Impala Runtime and Shell Impala Daemon Server

Druid集群

Druid集群服务组件具体部署信息

必选服务


服务名	主实例节点	核心实例节点
Druid	Druid Client Coordinator Overlord Broker Router	MiddleManager Historical Druid Client
HDFS	KMS SecondaryNameNode HttpFS HDFS Client NameNode	DataNode HDFS Client
Ganglia	Gmond Httpd Gmetad Ganglia Client	Gmond Ganglia Client
ZooKeeper	ZooKeeper follower ZooKeeper Client	ZooKeeper leader ZooKeeper follower ZooKeeper Client
OpenLDAP	OpenLDAP	N/A
Bigboot	Bigboot Client Bigboot Monitor	Bigboot Client Bigboot Monitor
SmartData	Jindo Namespace Service Jindo Storage Service Jindo Client	Jindo Storage Service Jindo Client

可选服务


服务名	主实例节点	核心实例节点
YARN	ResourceManager App Timeline Server JobHistory WebAppProxyServer Yarn Client	Yarn Client NodeManager
Superset	Superset	N/A

Kafka集群

Kafka集群服务组件具体部署信息

必选服务


服务名	主实例节点	核心实例节点
Kafka-Manager	Kafka Manager	N/A
Kafka	Kafka Client KafkaMetadataMonitor Kafka Rest Proxy Kafka Broker broker Kafka Schema Registry	Kafka Broker broker Kafka Client
Ganglia	Gmond Httpd Gmetad Ganglia Client	Gmond Ganglia Client
ZooKeeper	ZooKeeper follower ZooKeeper Client	ZooKeeper leader ZooKeeper follower ZooKeeper Client
OpenLDAP	OpenLDAP	N/A

可选服务


服务名	主实例节点	核心实例节点
Ranger	RangerPlugin RangerUserSync RangerAdmin	RangerPlugin
Knox	Knox	N/A

ZooKeeper集群

ZooKeeper集群服务组件具体部署信息


服务名	主实例节点	核心实例节点
Ganglia	N/A	Gmond Httpd Gmetad Ganglia Client
ZooKeeper	N/A	ZooKeeper leader ZooKeeper Client ZooKeeper follower

Flink集群

Flink集群服务组件具体部署信息

必选服务


服务名	主实例节点	核心实例节点
HDFS	KMS SecondaryNameNode HttpFS HDFS Client NameNode	DataNode HDFS Client
YARN	ResourceManager App Timeline Server JobHistory WebAppProxyServer Yarn Client	Yarn Client NodeManager
Ganglia	Gmond Httpd Gmetad Ganglia Client	Gmond Ganglia Client
ZooKeeper	ZooKeeper follower ZooKeeper Client	ZooKeeper leader ZooKeeper follower ZooKeeper Client
Zeppelin	Zeppelin	N/A
Flink	FlinkHistoryServer Flink Client	Flink Client
Has	HASServer	N/A
SmartData	JindoFS Namespace Service JindoFS Storage Service SmartData Client	JindoFS Storage Service SmartData Client
Bigboot	Bigboot Client Bigboot Monitor	Bigboot Client Bigboot Monitor
OpenLDAP	OpenLDAP	N/A
GRAFANA	GrafanaServer	N/A
Prometheus	PrometheusServer	N/A
AlertManager	AlertManager	N/A