本文介绍了OTSReader运行时可能出现的问题,请根据实际问题排查处理错误。
OTSReader运行时出现“Input size of values not equal size of primary key. input size: xx, primary key size: xx”错误
-
问题现象
OTSReader运行时出现“Input size of values not equal size of primary key. input size: xx, primary key size: xx”错误。
com.alibaba.datax.common.exception.DataXException: com.alibaba.datax.plugin.reader.otsreader.OtsReaderError@76f4b65 - IllegalArgumentException[ErrorMessage:Input size of values not equal size of primary key. input size:2, primary key size:1 .] - java.lang.IllegalArgumentException: Input size of values not equal size of primary key. input size:2, primary key size:1 . at com.alibaba.datax.plugin.reader.otsreader.utils.ParamChecker.checkInputPrimaryKeyAndGet(ParamChecker.java:119) at com.alibaba.datax.plugin.reader.otsreader.utils.ParamChecker.checkRangeAndGet(ParamChecker.java:151) at com.alibaba.datax.plugin.reader.otsreader.OtsReaderMasterProxy.init(OtsReaderMasterProxy.java:86) at com.alibaba.datax.plugin.reader.otsreader.OtsReader$Job.init(OtsReader.java:34) at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085) at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497) at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220) at com.alibaba.datax.core.Engine.start(Engine.java:100) at com.alibaba.datax.core.Engine.entry(Engine.java:321) at com.alibaba.datax.core.Engine.main(Engine.java:354) at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:41) at com.alibaba.datax.plugin.reader.otsreader.OtsReader$Job.init(OtsReader.java:47) at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085) at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497) at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220) at com.alibaba.datax.core.Engine.start(Engine.java:100) at com.alibaba.datax.core.Engine.entry(Engine.java:321) -
可能原因
OTSReader脚本中配置的主键个数与Tablestore数据表中的主键个数不一致。
-
解决方案
首先查看读取的Tablestore数据表Schema,确认主键个数,然后确保OTSReader脚本中配置的主键个数与Tablestore数据表中的主键个数一致。
假设Tablestore数据表有两个主键列,以全表同步为例,则OTSReader中应该配置的range范围如下:
"range": { "begin": [ { "type": "INF_MIN" }, { "type": "INF_MIN" } ], "end": [ { "type": "INF_MAX" }, { "type": "INF_MAX" } ] }
OTSReader运行时出现“Input type of 'range-split' not match partition key. Item of 'range-split' type:xx, Partition type:xx”错误
-
问题现象
OTSReader运行时出现“Input type of 'range-split' not match partition key. Item of 'range-split' type:xx, Partition type:xx”错误。
com.alibaba.datax.common.exception.DataXException: com.alibaba.datax.plugin.reader.otsreader.OtsReaderError@22f31dec - IllegalArgumentException[ErrorMessage:Input type of 'range-split' not match partition key. Item of 'range-split' type:null, Partition type:STRING] - java.lang.IllegalArgumentException: Input type of 'range-split' not match partition key. Item of 'range-split' type:null, Partition type:STRING at com.alibaba.datax.plugin.reader.otsreader.utils.ParamChecker.checkInputSplitPoints(ParamChecker.java:231) at com.alibaba.datax.plugin.reader.otsreader.OtsReaderMasterProxy.init(OtsReaderMasterProxy.java:92) at com.alibaba.datax.plugin.reader.otsreader.OtsReader$Job.init(OtsReader.java:34) at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085) at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497) at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220) at com.alibaba.datax.core.Engine.start(Engine.java:100) at com.alibaba.datax.core.Engine.entry(Engine.java:321) at com.alibaba.datax.core.Engine.main(Engine.java:354) at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:41) at com.alibaba.datax.plugin.reader.otsreader.OtsReader$Job.init(OtsReader.java:47) at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085) at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497) at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220) at com.alibaba.datax.core.Engine.start(Engine.java:100) at com.alibaba.datax.core.Engine.entry(Engine.java:321) at com.alibaba.datax.core.Engine.main(Engine.java:354) -
可能原因
OTSReader脚本中Split配置错误。不支持将Split配置为如下格式JSON:
"split": [ { "type": "INF_MIN" }, { "type": "INF_MAX" } ] -
解决方案
-
删除OTSReader脚本中的Split配置。
-
重新配置Split。具体操作,请参见使用Tablestore(OTS)Reader同步全量数据时如何配置Split。
-
OTSReader运行时出现“Invalid 'column', Can not parse Object to 'OTSColumn', item of list is not a map”错误
-
问题现象
OTSReader运行时出现“Invalid 'column', Can not parse Object to 'OTSColumn', item of list is not a map”错误。
com.alibaba.datax.common.exception.DataXException: com.alibaba.datax.plugin.reader.otsreader.OtsReaderError@64bce832 – IllegalArgumentException[ErrorMessage:Invalid 'column', Can not parse Object to 'OTSColumn', item of list is not a map.] – java.lang.IllegalArgumentException: Invalid 'column', Can not parse Object to 'OTSColumn', item of list is not a map. at com.alibaba.datax.plugin.reader.otsreader.utils.ReaderModelParser.parseOTSColumnList(ReaderModelParser.java:104) at com.alibaba.datax.plugin.reader.otsreader.OtsReaderMasterProxy.init(OtsReaderMasterProxy.java:80) at com.alibaba.datax.plugin.reader.otsreader.OtsReader$Job.init(OtsReader.java:34) at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085) at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497) at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220) at com.alibaba.datax.core.Engine.start(Engine.java:100) at com.alibaba.datax.core.Engine.entry(Engine.java:321) at com.alibaba.datax.core.Engine.main(Engine.java:354) at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:41) at com.alibaba.datax.plugin.reader.otsreader.OtsReader$Job.init(OtsReader.java:47) at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085) at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497) at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220) at com.alibaba.datax.core.Engine.start(Engine.java:100) at com.alibaba.datax.core.Engine.entry(Engine.java:321) at com.alibaba.datax.core.Engine.main(Engine.java:354) -
可能原因
OTSReader脚本中的column配置错误。错误配置示例如下:
"column": [ "id", "name", "age" ] -
解决方案
请确保OTSReader脚本中的column配置正确。正确配置示例如下:
"column": [ { "name": "id" }, { "name": "name" }, { "name": "age" } ]