Dataphin代码任务运行报错:ODPS-0010000:System internal error - fuxi job failed, caused by: kWorkerOutOfMemory
更新时间:
问题描述
Dataphin中代码任务运行报错:ODPS-0010000:System internal error - fuxi job failed, caused by: kWorkerOutOfMemory
完整的报错信息如下:
2022-09-27 13:24:57.974 com.aliyun.odps.OdpsException: ODPS-0010000:System internal error - fuxi job failed, caused by: kWorkerOutOfMemory(errCode:256) at Odps/sdiccy_cdm_2022092705115921ge6jiwy9_SQL_0_1_0_job_0/M1_3_8_10_16@k22e12297.cloud.nm125#3. Detail error msg: KILL_NAKILL_NA: plannedResource({Memory=1280, SInstance=1, CPU=100}) usedResource({Memory=12032, CPU=102})
at com.aliyun.odps.Instance.waitForSuccess(Instance.java:809)
at com.aliyun.odps.Instance.waitForSuccess(Instance.java:786)
at com.alibaba.dt.oneservice.sdk.OdpsSqlExecutor.run(OdpsSqlExecutor.java:86)
at com.alibaba.dt.oneservice.sdk.OdpsTask.runTask(OdpsTask.java:94)
at com.alibaba.dt.oneservice.sdk.task.BaseOsTask.run(BaseOsTask.java:116)
at com.alibaba.dt.oneservice.sdk.job.OsJob.run(OsJob.java:219)
at com.alibaba.dt.oneservice.sdk.executor.BaseOsExecutor.lambda$run$0(BaseOsExecutor.java:48)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:627)
at java.lang.Thread.run(Thread.java:882)
2022-09-27 13:24:58.049 Job failed
问题原因
计算源MaxCompute后台fuxi节点内存溢出导致。
解决方案
代码任务中调大map的内存和并发问题解决:
set odps.stage.mapper.mem=12288;
set odps.stage.mapper.split.size=128;
更多信息
适用于
- Dataphin
文档内容是否对您有帮助?