Dataphin中管道任务将MySQL数据同步到Maxcompute,运行报错[同步数据出现业务脏数据情况,数据类型转换错误.]-String['']不能转为BigDecimal.-java.lang.NumberFormatException-阿里云帮助中心

问题描述

管道任务将MySQL数据同步到Maxcompute，运行报错“[同步数据出现业务脏数据情况，数据类型转换错误 .] - String [''] 不能转为BigDecimal . - java.lang.NumberFormatException”。完整的异常如下：

2021-11-05 00:12:18.966 [0-0-0-writer] ERROR StdoutPluginCollector - 
com.alibaba.datax.common.exception.DataXException: Code:[Common-01], Describe:[同步数据出现业务脏数据情况，数据类型转换错误 .] - String [""] 不能转为BigDecimal . - java.lang.NumberFormatException
 at java.math.BigDecimal.<init>(BigDecimal.java:596)
 at java.math.BigDecimal.<init>(BigDecimal.java:383)
 at java.math.BigDecimal.<init>(BigDecimal.java:806)
 at com.alibaba.datax.common.element.StringColumn.asBigDecimal(StringColumn.java:90)
 at com.alibaba.datax.common.element.StringColumn.asBigInteger(StringColumn.java:54)
 at com.alibaba.datax.common.element.StringColumn.asLong(StringColumn.java:71)
 at com.alibaba.datax.plugin.writer.odpswriter.OdpsWriterProxy.dataxRecordToOdpsRecord(OdpsWriterProxy.java:179)
 at com.alibaba.datax.plugin.writer.odpswriter.OdpsWriterProxy.writeOneRecord(OdpsWriterProxy.java:97)
 at com.alibaba.datax.plugin.writer.odpswriter.OdpsWriter$Task.startWrite(OdpsWriter.java:354)
 at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56)
 at java.lang.Thread.run(Thread.java:882)
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:41) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
 at com.alibaba.datax.common.element.StringColumn.asBigDecimal(StringColumn.java:92) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
 at com.alibaba.datax.common.element.StringColumn.asBigInteger(StringColumn.java:54) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
 at com.alibaba.datax.common.element.StringColumn.asLong(StringColumn.java:71) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
 at com.alibaba.datax.plugin.writer.odpswriter.OdpsWriterProxy.dataxRecordToOdpsRecord(OdpsWriterProxy.java:179) [odpswriter-0.0.1-SNAPSHOT.jar:na]
 at com.alibaba.datax.plugin.writer.odpswriter.OdpsWriterProxy.writeOneRecord(OdpsWriterProxy.java:97) [odpswriter-0.0.1-SNAPSHOT.jar:na]
 at com.alibaba.datax.plugin.writer.odpswriter.OdpsWriter$Task.startWrite(OdpsWriter.java:354) [odpswriter-0.0.1-SNAPSHOT.jar:na]
 at com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56) [datax-core-0.0.1-SNAPSHOT.jar:na]
 at java.lang.Thread.run(Thread.java:882) [na:1.8.0_152]
Caused by: java.lang.NumberFormatException: null
 at java.math.BigDecimal.<init>(BigDecimal.java:596) ~[na:1.8.0_152]
 at java.math.BigDecimal.<init>(BigDecimal.java:383) ~[na:1.8.0_152]
 at java.math.BigDecimal.<init>(BigDecimal.java:806) ~[na:1.8.0_152]
 at com.alibaba.datax.common.element.StringColumn.asBigDecimal(StringColumn.java:90) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
 ... 7 common frames omitted
2021-11-05 00:12:18.975 [0-0-0-writer] ERROR StdoutPluginCollector - 脏数据: 
{"exception":"Code:[Common-01], Describe:[同步数据出现业务脏数据情况，数据类型转换错误 .] - String [\"\"] 不能转为BigDecimal .
 - java.lang.NumberFormatException\n\tat java.math.BigDecimal.<init>(BigDecimal.java:596)\n\tat java.math.BigDecimal.<init>(BigDecimal.java:383)\n\tat 
java.math.BigDecimal.<init>(BigDecimal.java:806)\n\tat com.alibaba.datax.common.element.StringColumn.asBigDecimal(StringColumn.java:90)\n\tat
 com.alibaba.datax.common.element.StringColumn.asBigInteger(StringColumn.java:54)\n\tat com.alibaba.datax.common.element.StringColumn.asLong(StringColumn.java:71)\n\tat 
com.alibaba.datax.plugin.writer.odpswriter.OdpsWriterProxy.dataxRecordToOdpsRecord(OdpsWriterProxy.java:179)\n\tat 
com.alibaba.datax.plugin.writer.odpswriter.OdpsWriterProxy.writeOneRecord(OdpsWriterProxy.java:97)\n\tat 
com.alibaba.datax.plugin.writer.odpswriter.OdpsWriter$Task.startWrite(OdpsWriter.java:354)\n\tat 
com.alibaba.datax.core.taskgroup.runner.WriterRunner.run(WriterRunner.java:56)\n\tat 
java.lang.Thread.run(Thread.java:882)\n","message":"写入 ODPS 目的表时遇到了脏数据: 第[18]个字段的数据出现错误，
请检查该数据并作出修改

问题原因

类型为String的输入字段，有值为空字符串的记录导致。

解决方案

如果输入字段类型是String，那么转换到decimal的话，需要输入字段不能是空字符串，要么有内容要么是null，但不能是空字符串。将原始数据中为空字符串的记录处理后，问题解决。

适用于

Dataphin