Dataphin代码任务运行报错：ODPS-0010000:System internal error - fuxi job failed, caused by: kWorkerOutOfMemory

问题描述

Dataphin中代码任务运行报错：ODPS-0010000:System internal error - fuxi job failed, caused by: kWorkerOutOfMemory

完整的报错信息如下：

2022-09-27 13:24:57.974 com.aliyun.odps.OdpsException: ODPS-0010000:System internal error - fuxi job failed, caused by: kWorkerOutOfMemory(errCode:256) at Odps/sdiccy_cdm_2022092705115921ge6jiwy9_SQL_0_1_0_job_0/M1_3_8_10_16@k22e12297.cloud.nm125#3. Detail error msg: KILL_NAKILL_NA: plannedResource({Memory=1280, SInstance=1, CPU=100}) usedResource({Memory=12032, CPU=102})

 at com.aliyun.odps.Instance.waitForSuccess(Instance.java:809)
 at com.aliyun.odps.Instance.waitForSuccess(Instance.java:786)
 at com.alibaba.dt.oneservice.sdk.OdpsSqlExecutor.run(OdpsSqlExecutor.java:86)
 at com.alibaba.dt.oneservice.sdk.OdpsTask.runTask(OdpsTask.java:94)
 at com.alibaba.dt.oneservice.sdk.task.BaseOsTask.run(BaseOsTask.java:116)
 at com.alibaba.dt.oneservice.sdk.job.OsJob.run(OsJob.java:219)
 at com.alibaba.dt.oneservice.sdk.executor.BaseOsExecutor.lambda$run$0(BaseOsExecutor.java:48)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:627)
 at java.lang.Thread.run(Thread.java:882)
2022-09-27 13:24:58.049 Job failed

问题原因

计算源MaxCompute后台fuxi节点内存溢出导致。

解决方案

代码任务中调大map的内存和并发问题解决：

set odps.stage.mapper.mem=12288;
set odps.stage.mapper.split.size=128;

更多信息

 

MaxCompute中SET操作

适用于

• Dataphin