Dataphin代码任务报错："Fatal error occurred when node tried to create too many dynamic partitions"
更新时间：2021-11-16 07:56:52
问题描述
Dataphin代码任务运行报错："Fatal error occurred when node tried to create too many dynamic partitions"。
2021-11-11 11:13:12,999 INFO [IPC Server handler 24 on 64040] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Diagnostics report from attempt_1636027603751_126799_r_000113_2: Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveFatalException: [Error 20004]: Fatal error occurred when node tried to create too many dynamic partitions. The maximum number of dynamic partitions is controlled by hive.exec.max.dynamic.partitions and hive.exec.max.dynamic.partitions.pernode. Maximum was set to: 100
问题原因
在每个执行MR的节点上，默认最大可以创建100个动态分区。
解决方案
添加参数进行修改默认配置。
SET hive.exec.max.dynamic.partitions.pernode=10000;
适用于
- Dataphin
该文章对您有帮助吗？
- 本页导读
- 问题描述
- 问题原因
- 解决方案
- 适用于
