Dataphin项目绑定的计算引擎为CDH6,代码运行报错“File does not exist: hdfs://domain_name:PORT/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.2-mr-framework.tar.gz”

问题描述

Dataphin项目绑定的计算引擎为CDH6，代码运行报错“ Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask. File does not exist: hdfs://domain_name:PORT/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.2-mr-framework.tar.gz"，完整的报错信息如下：

Process:Task failed : java.sql.SQLException: Error while processing statement: FAILED:
 Execution Error,return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask. File does not exist: 
hdfs://domain_name:8020/user/yarn/mapreduce/mr-framework/3.0.0-cdh6.3.2-mr-framework.tar.gz
 at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:284)
 at com.alibaba.dt.oneservice.sdk.task.BaseJdbcOsTask.runTask(BaseJdbcOsTask.java:118)
 at com.alibaba.dt.oneservice.sdk.task.BaseOsTask.run(BaseOsTask.java:116)
 at com.alibaba.dt.oneservice.sdk.job.OsJob.run(OsJob.java:214)
 at com.alibaba.dt.oneservice.sdk.executor.BaseOsExecutor.lambda$run$0(BaseOsExecutor.java:48)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:627)
 at java.lang.Thread.run(Thread.java:882)

问题原因

CDH中没有安装YARN MapReduce框架相关的Jar包导致。

解决方案

联系后台运维人员登录CDH集权管理界面安装YARN MapReduce框架相关Jar即可。具体步骤如下：

适用于

• Dataphin