本文为您介绍E-MapReduce中常用文件的路径。您可以登录Master节点查看常用文件的安装路径。
数据湖集群
大数据组件安装目录
组件安装在/opt/apps/xxx目录下,例如:
- HDFS:/opt/apps/HDFS/hdfs-current 
- Hive:/opt/apps/HIVE/hive-current 
- Hudi:/opt/apps/HUDI/hudi-current 
- YARN:/opt/apps/YARN/yarn-current 
- Presto:/opt/apps/PRESTO/presto-current 
- Ranger:/opt/apps/RANGER/ranger-current 
 您也可以通过登录Master节点,执行env |grep xxx命令查看软件的安装目录,其中xxx为服务名。
例如,执行命令env |grep hive,查看Hive的安装目录。
JINDOTABLE_EXTRA_CLASSPATH=/opt/apps/METASTORE/metastore-current/hive2
HIVE_HOME=/opt/apps/HIVE/hive-current
HIVE_LOG_DIR=/var/log/taihao-apps/hive
HIVE_CONF_DIR=/etc/taihao-apps/hive-conf
PATH=/opt/apps/JINDOSDK/jindosdk-current/bin:/opt/apps/HADOOP-COMMON/hadoop-common-current/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/apps/HIVE/hive-current/bin:/opt/apps/JINDODATA/jindodata-current/bin:/opt/apps/JINDODATA/jindodata-current/sbin:/opt/apps/SPARK-EXTENSION/spark-extension-current/bin:/opt/apps/SPARK3/spark-current/bin:/root/bin
OLDPWD=/var/log/emr/hive日志目录
日志在/var/log/emr/xxx目录下,例如:
- Spark:/var/log/emr/spark/ 
- Hive:/var/log/emr/hive/ 
- YARN:/var/log/emr/yarn/ 
- Jindosdk:/var/log/emr/jindosdk/ 
配置文件目录
配置文件在/etc/emr/xxx目录下,例如:
- HDFS:/etc/emr/hdfs-conf/ 
- Spark:/etc/emr/spark-conf/ 
- Hive:/etc/emr/hive-conf/ 
- Hudi:/etc/emr/hudi-conf/ 
- Knox:/etc/emr/knox-conf/ 
- YARN:/etc/emr/hadoop-conf/ 
- Zookeeper:/etc/emr/zookeeper-conf/ 
旧版数据湖集群
大数据组件安装目录
软件安装在/usr/lib/xxx目录下,例如:
- Hadoop:/usr/lib/hadoop-current 
- Spark :/usr/lib/spark-current 
- Hive:/usr/lib/hive-current 
- Flink:/usr/lib/flink-current 
- Flume:/usr/lib/flume-current 
您也可以通过登录Master节点,执行env |grep xxx命令查看软件的安装目录。
例如,执行以下命令,查看Spark的安装目录。
env |grep spark返回如下信息,其中/usr/lib/spark-current为Spark的安装目录。
SPARK_HOME=/usr/lib/spark-current
SPARK_CONF_DIR=/etc/ecm/spark-conf
SPARK_LOG_DIR=/mnt/disk1/log/spark
PATH=/usr/lib/sqoop-current/bin:/usr/lib/jindosdk-current/bin:/usr/lib/hudi-current/bin:/usr/lib/hive-current/hcatalog/bin:/usr/lib/hive-current/bin:/usr/lib/datafactory-current/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/lib/flow-agent-current/bin:/usr/lib/hadoop-current/bin:/usr/lib/hadoop-current/sbin:/usr/lib/jindodata-current//bin:/usr/lib/jindodata-current//sbin:/usr/lib/spark-current/bin:/usr/lib/hadoop-current/bin:/usr/lib/hadoop-current/sbin:/root/bin
HADOOP_CLASSPATH=/opt/apps/extra-jars/*:/usr/lib/spark-current/yarn/spark-3.2.1-yarn-shuffle.jar
SPARK_PID_DIR=/usr/lib/spark-current/pids日志目录
组件日志在/mnt/disk1/log/xxx目录下,例如:
- Yarn ResourceManager日志:Master节点/mnt/disk1/log/hadoop-yarn 
- Yarn NodeNanager日志:Slave节点/mnt/disk1/log/hadoop-yarn 
- HDFS NameNode日志:Master节点/mnt/disk1/log/hadoop-hdfs 
- HDFS DataNode日志:Slave节点/mnt/disk1/log/hadoop-hdfs 
- Hive日志:Master节点/mnt/disk1/log/hive 
- ESS日志:Master和Worker节点/mnt/disk1/log/ess/ 
配置文件目录
配置文件在/etc/ecm/xxx目录下,例如:
- Hadoop:/etc/ecm/hadoop-conf/ 
- Spark:/etc/ecm/spark-conf/ 
- Hive:/etc/ecm/hive-conf/ 
- Flink:/etc/ecm/flink-conf/ 
- Flume:/etc/ecm/flume-conf/ 
如果您需要修改配置文件中的参数,请登录E-MapReduce控制台操作,通过SSH方式只能浏览配置文件中的参数。
数据目录
JindoFS缓存数据:/mnt/disk1/jindodata/