OTSReader常见问题

更新时间:
复制为 MD 格式

本文介绍了OTSReader运行时可能出现的问题,请根据实际问题排查处理错误。

OTSReader运行时出现“Input size of values not equal size of primary key. input size: xx, primary key size: xx”错误

  • 问题现象

    OTSReader运行时出现“Input size of values not equal size of primary key. input size: xx, primary key size: xx”错误。

    com.alibaba.datax.common.exception.DataXException: com.alibaba.datax.plugin.reader.otsreader.OtsReaderError@76f4b65 - IllegalArgumentException[ErrorMessage:Input size of values not equal size of primary key. input size:2, primary key size:1 .] - java.lang.IllegalArgumentException: Input size of values not equal size of primary key. input size:2, primary key size:1 .
    	at com.alibaba.datax.plugin.reader.otsreader.utils.ParamChecker.checkInputPrimaryKeyAndGet(ParamChecker.java:119)
    	at com.alibaba.datax.plugin.reader.otsreader.utils.ParamChecker.checkRangeAndGet(ParamChecker.java:151)
    	at com.alibaba.datax.plugin.reader.otsreader.OtsReaderMasterProxy.init(OtsReaderMasterProxy.java:86)
    	at com.alibaba.datax.plugin.reader.otsreader.OtsReader$Job.init(OtsReader.java:34)
    	at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085)
    	at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497)
    	at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220)
    	at com.alibaba.datax.core.Engine.start(Engine.java:100)
    	at com.alibaba.datax.core.Engine.entry(Engine.java:321)
    	at com.alibaba.datax.core.Engine.main(Engine.java:354)
    	at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:41)
    	at com.alibaba.datax.plugin.reader.otsreader.OtsReader$Job.init(OtsReader.java:47)
    	at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085)
    	at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497)
    	at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220)
    	at com.alibaba.datax.core.Engine.start(Engine.java:100)
    	at com.alibaba.datax.core.Engine.entry(Engine.java:321)
  • 可能原因

    OTSReader脚本中配置的主键个数与Tablestore数据表中的主键个数不一致。

  • 解决方案

    首先查看读取的Tablestore数据表Schema,确认主键个数,然后确保OTSReader脚本中配置的主键个数与Tablestore数据表中的主键个数一致。

    假设Tablestore数据表有两个主键列,以全表同步为例,则OTSReader中应该配置的range范围如下:

    "range": {
      "begin": [
        {
          "type": "INF_MIN"
        },
        {
          "type": "INF_MIN"
        }
      ],
      "end": [
        {
          "type": "INF_MAX"
        },
        {
          "type": "INF_MAX"
        }
      ]
    }

OTSReader运行时出现“Input type of 'range-split' not match partition key. Item of 'range-split' type:xx, Partition type:xx”错误

  • 问题现象

    OTSReader运行时出现“Input type of 'range-split' not match partition key. Item of 'range-split' type:xx, Partition type:xx”错误。

    com.alibaba.datax.common.exception.DataXException: com.alibaba.datax.plugin.reader.otsreader.OtsReaderError@22f31dec - IllegalArgumentException[ErrorMessage:Input type of 'range-split' not match partition key. Item of 'range-split' type:null, Partition type:STRING] - java.lang.IllegalArgumentException: Input type of 'range-split' not match partition key. Item of 'range-split' type:null, Partition type:STRING
    	at com.alibaba.datax.plugin.reader.otsreader.utils.ParamChecker.checkInputSplitPoints(ParamChecker.java:231)
    	at com.alibaba.datax.plugin.reader.otsreader.OtsReaderMasterProxy.init(OtsReaderMasterProxy.java:92)
    	at com.alibaba.datax.plugin.reader.otsreader.OtsReader$Job.init(OtsReader.java:34)
    	at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085)
    	at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497)
    	at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220)
    	at com.alibaba.datax.core.Engine.start(Engine.java:100)
    	at com.alibaba.datax.core.Engine.entry(Engine.java:321)
    	at com.alibaba.datax.core.Engine.main(Engine.java:354)
    	at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:41)
    	at com.alibaba.datax.plugin.reader.otsreader.OtsReader$Job.init(OtsReader.java:47)
    	at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085)
    	at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497)
    	at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220)
    	at com.alibaba.datax.core.Engine.start(Engine.java:100)
    	at com.alibaba.datax.core.Engine.entry(Engine.java:321)
    	at com.alibaba.datax.core.Engine.main(Engine.java:354)
  • 可能原因

    OTSReader脚本中Split配置错误。不支持将Split配置为如下格式JSON:

    "split": [
      {
        "type": "INF_MIN"
      },
      {
      "type": "INF_MAX"
      }
    ]
  • 解决方案

OTSReader运行时出现“Invalid 'column', Can not parse Object to 'OTSColumn', item of list is not a map”错误

  • 问题现象

    OTSReader运行时出现“Invalid 'column', Can not parse Object to 'OTSColumn', item of list is not a map”错误。

    com.alibaba.datax.common.exception.DataXException: com.alibaba.datax.plugin.reader.otsreader.OtsReaderError@64bce832 – IllegalArgumentException[ErrorMessage:Invalid 'column', Can not parse Object to 'OTSColumn', item of list is not a map.] – java.lang.IllegalArgumentException: Invalid 'column', Can not parse Object to 'OTSColumn', item of list is not a map.
            at com.alibaba.datax.plugin.reader.otsreader.utils.ReaderModelParser.parseOTSColumnList(ReaderModelParser.java:104)
            at com.alibaba.datax.plugin.reader.otsreader.OtsReaderMasterProxy.init(OtsReaderMasterProxy.java:80)
            at com.alibaba.datax.plugin.reader.otsreader.OtsReader$Job.init(OtsReader.java:34)
            at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085)
            at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497)
            at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220)
            at com.alibaba.datax.core.Engine.start(Engine.java:100)
            at com.alibaba.datax.core.Engine.entry(Engine.java:321)
            at com.alibaba.datax.core.Engine.main(Engine.java:354)
            at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:41)
            at com.alibaba.datax.plugin.reader.otsreader.OtsReader$Job.init(OtsReader.java:47)
            at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085)
            at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497)
            at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220)
            at com.alibaba.datax.core.Engine.start(Engine.java:100)
            at com.alibaba.datax.core.Engine.entry(Engine.java:321)
            at com.alibaba.datax.core.Engine.main(Engine.java:354)
  • 可能原因

    OTSReader脚本中的column配置错误。错误配置示例如下:

    "column": [
      "id",
      "name",
      "age"
    ]
  • 解决方案

    请确保OTSReader脚本中的column配置正确。正确配置示例如下:

    "column": [
      {
        "name": "id"
      },
      {
        "name": "name"
      },
      {
        "name": "age"
      }
    ]