本文为您介绍在配置DTS任务时,系统返回的报错信息(DTS-RETRY-ERR-0601至DTS-RETRY-ERR-0700)及其对应的解决方法。
本文以正则表达式((.*)?)的形式表示报错信息中的变量。
若报错无法解决,您可以通过钉钉(钉钉通讯客户端下载地址)进入到DTS客户交流钉钉群(群号:116655009709),进行咨询。
DTS-RETRY-ERR-0601:could not write to file
可能原因:源端PostgreSQL数据库服务器的磁盘空间不足。当DTS进行逻辑复制时,源库需要创建快照文件,但由于设备上没有空间(No space left on device),此操作失败。
解决方法:请清理源端PostgreSQL服务器的磁盘空间,确保为逻辑复制操作提供足够的存储。空间释放后,重新启动DTS任务。
报错示例:
DTS-52111: Increment Context Is Not Running..: cause: CriticalAnyAllException: postgresql-reader: DTS-52510: Fetch postgresql logical log failed FatalAnyAllException: common: DTS-100047: retry 0 times, 5029990526 seconds, which exceed the supposed 43200 seconds IOException: org.postgresql.util.PSQLException: ERROR: could not write to file "pg_logical/snapshots/401-xxx.snap.xxx.tmp": 设备上没有空间 PSQLException: ERROR: could not write to file "pg_logical/snapshots/401-xxx.snap.xxx.tmp": 设备上没有空间DTS-RETRY-ERR-0602:CROSSSLOT Keys
可能原因:DTS尝试在目标Redis集群中执行一个涉及多个Key的命令,但这些Key被哈希到不同的Slot上。这通常发生在源库和目标库的拓扑结构不一致时(例如,源库为标准版,目标库为集群版)。
解决方法:请检查并确保源库与目标库的拓扑结构一致。
报错示例:
redis.clients.jedis.exceptions.JedisDataException: CROSSSLOT Keys in request don't hash to the same slotDTS-RETRY-ERR-0603:Database (.*)? cannot be opened due to inaccessible files or insufficient memory or disk space. See the SQL Server errorlog for details.
可能原因:目标SQL Server数据库因底层资源问题而无法打开。具体原因可能是数据文件不可访问、服务器内存不足或磁盘空间已满。
解决方法:请登录目标数据库服务器,查看SQL Server的错误日志(errorlog)以获取详细的失败原因。根据日志提示解决文件权限、内存或磁盘空间问题后,重新启动DTS任务。
报错示例:
DTS-100047: retry 112 times, 1208 seconds, which exceed the supposed 1200 seconds cause: SQLServerException: Database 'xxx' cannot be opened due to inaccessible files or insufficient memory or disk space. See the SQL Server errorlog for details.DTS-RETRY-ERR-0604:Database (.*)? is in transition. Try the statement later.
可能原因:目标SQL Server数据库当前正处于一个过渡状态,例如启动、恢复、故障转移等,暂时无法接受新的连接和操作。
解决方法:请等待目标数据库完成其当前操作并恢复到运行中状态。确认数据库可用后,重新启动DTS任务。
报错示例:
com.microsoft.sqlserver.jdbc.SQLServerException: Database 'xxx' is in transition. Try the statement later.DTS-RETRY-ERR-0605:Database connection failed when reading from copy EOFException: null
可能原因:源端PostgreSQL的逻辑复制连接因wal_sender_timeout超时而中断。当DTS长时间未与源库交互(例如,过滤了大量无需同步的事务)时,源库可能因未收到心跳而主动断开连接。
解决方法:连接到源数据库,将wal_sender_timeout参数值设置为0以禁用超时机制。
执行SQL命令:
ALTER SYSTEM SET wal_sender_timeout = '0';重新加载配置:
SELECT pg_reload_conf();修改配置后,重新启动DTS任务。
报错示例:
xDTS-52111: Increment Context Is Not Running..: cause: CriticalAnyAllException: postgresql-reader: DTS-52510: Fetch postgresql logical log failed FatalAnyAllException: common: DTS-100047: retry 0 times, 1016336561 seconds, which exceed the supposed 43200 seconds IOException: org.postgresql.util.PSQLException: Database connection failed when reading from copy PSQLException: Database connection failed when reading from copy EOFException: nullDTS-RETRY-ERR-0606:date format not recognized
可能原因:源端Oracle数据库中存在DTS无法识别或解析的日期格式数据,导致全量数据拉取失败。
解决方法:请根据报错信息中的ROWID或数据内容,定位到源库中的异常数据行,并将其修正为标准的日期格式。数据修复后,重新启动DTS任务。
报错示例:
DTS-30020: execute sql:/* DTS-full-dl0y6l2a214xxxx */SELECT xxx FROM "xxx"."xxx" --> (null, null) record range id = 48754756125982xxxx, fetchSize = 1024 cause: SQLDataException: ORA-01821: date format not recognizedDTS-RETRY-ERR-0607:Disperse target record field (.*)? failed
可能原因:DTS在处理数据时,无法将源库的某个字段值转换为目标库对应字段的类型。这通常是因为源库与目标库的字段类型不兼容,且DTS不支持该种转换。
解决方法:请检查报错字段在源库和目标库中的数据类型。您可以尝试修改目标表结构使之兼容,或在DTS任务中排除该字段的同步。调整后,重新启动DTS任务。
报错示例:
DTS-71001: Disperse target record field collectInfo BYTES nullable as collectInfo failed for record databaseName: xxx tableName:xxx {offset [35699] operationType [UPDATE],checkpoint [0@35699@160288815@12568@175275215xxx]} cause: ConcurrentModificationException: nullDTS-RETRY-ERR-0608:disperser is null
可能原因:DTS在进行数据类型转换时,找不到处理特定字段的转换器(disperser)。这通常是由于源端和目标端表的字段类型不兼容。
解决方法:请检查并对比源端与目标端不兼容字段的表结构。您可以尝试修改目标表字段类型,使其与源端兼容,或者在DTS任务中排除该字段的同步。调整后,重新启动DTS任务。
报错示例:
DTS-61001: disperser is null,can not find disperser for field billing_dateDTS-RETRY-ERR-0609:document size is larger than maximum
可能原因:待同步的数据中,存在单个文档的大小超过了MongoDB所允许的最大16 MB限制,导致该文档无法写入目标库。
解决方法:请根据业务情况,在源库中处理这个超大的文档(例如,进行拆分或删除)。
报错示例:
DTS-100047: retry 0 times, 0 seconds, which exceed the supposed 922337203685477xxxx seconds cause: SQLException: DTS full single insert oversize, throw exception and retry, caused by:Payload document size is larger than maximum of xxxx.DTS-RETRY-ERR-0610:does not contain shard key
可能原因:目标MongoDB集合是一个分片集合,但待同步的某条数据中缺少必要的分片键(Shard Key)字段,导致无法写入。
解决方法:请检查源端数据,确保所有待同步的记录都包含了目标表所需的分片键字段。
报错示例:
DTS-100047: retry 0 times, 0 seconds, which exceed the supposed 922337203685477xxxx seconds cause: SQLException: com.mongodb.MongoWriteException: Write operation error on server 10.xxx.xxx.xxx:xxx. Write error: WriteError{code=61, message='document { xxx } does not contain shard key for pattern { brandId: 1.0, createTime: 1.0 }', details={}}. MongoWriteException: Write operation error on server 10.xxx.xxx.xxx:xxx. Write error: WriteError{code=61, message='document { xxx } does not contain shard key for pattern { brandId: 1.0, createTime: 1.0 }', details={}}.DTS-RETRY-ERR-0611:Don't allow to dump binary log on standby instance
可能原因:DTS被配置连接到了源MySQL数据库的一个备库(Standby/Slave Instance),而该备库的配置禁止外部连接(如DTS)拉取其Binlog。
解决方法:请修改DTS任务的源库连接信息,将其指向源数据库的主库(Master/Primary Instance)。
报错示例:
DTS-100047: retry 0 times, 1001 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: mysql-reader: DTS-52200: process error failed cause: IOException: java.sql.SQLException: Don't allow to dump binary log on standby instance SQLException: Don't allow to dump binary log on standby instance CriticalAnyAllException: mysql-reader: DTS-52200: process error failed IOException: java.sql.SQLException: Don't allow to dump binary log on standby instance SQLException: Don't allow to dump binary log on standby instanceDTS-RETRY-ERR-0612:DTS-50019
报错说明:DTS写入模块读取不到DTS缓存模块中指定时间点的数据。
可能原因与解决方法:
可能原因1:任务暂停过久,重启后DTS缓存模块中已没有相应位点(如2021-5-20 21:17:51)的数据。
说明报错提示显示的时间格式为Unix时间戳,您可以使用搜索引擎获取Unix时间戳转换工具,将Unix时间戳转换为UTC时间。
解决方法1:
方法一:请查看源库中用于增量数据解析的日志(如Binlog或Redolog等),其时间范围是否包括该时间点。
方法二:需重新创建任务。后续执行任务时,请勿长时间暂停任务,暂停超过6个小时,任务将不能成功启动。
可能原因2:任务失败过久,修复后DTS缓存模块中已没有相应位点的数据。
解决方法2:需重新创建任务。后续执行任务时,如遇任务报错,请尽快修复问题并恢复任务,避免任务因失败过久而无法恢复。
可能原因3:全量迁移或全量同步期间,DTS本地会循环存储源库近24小时或者50 GB的数据日志,用于增量数据解析。如果全量迁移或全量同步持续时间较长,DTS本地可能已经清除全量任务启动后缓存的数据日志,导致用于增量解析的数据日志不完整。
解决方法3:需重新创建任务。创建任务前,请评估迁移或同步数据量,如数据量较大,建议拆分成多个迁移或同步任务。
报错示例:
DTS-31009: In process of processing data (recordRange: 15907394498750xxx) failed cause: FatalAnyAllException: common:
DTS-100047: retry 0 times, 3 seconds, which exceed the supposed 43200 seconds CriticalAnyAllException: capture-as400:
DTS-50019: Lost journal {timestampMs: 1752047995xxx} log {journalName: NBJRN, journalLibraryName: COASSYSTF1}DTS-RETRY-ERR-0613:DTS-52061
报错说明:DTS写入模块读取不到DTS缓存模块中指定时间点的数据。
可能原因与解决方法:
可能原因1:任务暂停过久,重启后DTS缓存模块中已没有相应位点(如2021-5-20 21:17:51)的数据。
说明报错提示显示的时间格式为Unix时间戳,您可以使用搜索引擎获取Unix时间戳转换工具,将Unix时间戳转换为UTC时间。
解决方法1:
方法一:请查看源库中用于增量数据解析的日志(如Binlog或Redolog等),其时间范围是否包括该时间点。
方法二:需重新创建任务。后续执行任务时,请勿长时间暂停任务,暂停超过6个小时,任务将不能成功启动。
可能原因2:任务失败过久,修复后DTS缓存模块中已没有相应位点的数据。
解决方法2:需重新创建任务。后续执行任务时,如遇任务报错,请尽快修复问题并恢复任务,避免任务因失败过久而无法恢复。
可能原因3:全量迁移或全量同步期间,DTS本地会循环存储源库近24小时或者50 GB的数据日志,用于增量数据解析。如果全量迁移或全量同步持续时间较长,DTS本地可能已经清除全量任务启动后缓存的数据日志,导致用于增量解析的数据日志不完整。
解决方法3:需重新创建任务。创建任务前,请评估迁移或同步数据量,如数据量较大,建议拆分成多个迁移或同步任务。
报错示例:
DTS-31009: In process of processing data (recordRange: 15907394498750xxxx) failed cause: FatalAnyAllException: common:
DTS-100047: retry 0 times, 161765 seconds, which exceed the supposed 43200 seconds RuntimeException: AS400DB2Source:journalFetcher-1: failed cause retrieve capture-as400:
DTS-52061: {ReceiverName: NBRCV9xxxx, ReceiverLibrary: COASSYSTF1}'s journal receiver is deleted, That journal log isn't continuousDTS-RETRY-ERR-0615:DTS-52722
可能原因:DTS任务配置的数据库账号没有读取所需表的权限。
解决方法:请为DTS使用的账号授予对同步对象的读取权限。授权完成后,重新启动DTS任务。
报错示例:
DTS-31009: In process of processing data (recordRange: 456002531641996xxxx) failed cause: FatalAnyAllException: common:
DTS-100047: retry 0 times, 81170 seconds, which exceed the supposed 43200 seconds RuntimeException: AS400DB2Source:JournalEntryParser: parser entry failed, cause capture-as400: DTS-52722: Read table xxx.xxx info error, case by [SQL0551] Not authorized to object xxx in xxx type *FILE. Cause . . . . . : An operation was attempted on object xxx in xxx type *FILE. This operation cannot be performed without the required authority. Recovery . . . : Obtain the required authority from either the security officer, the object owner, or a user that is authorized to the QIBM_DB_SECADM function. If you are not authorized to a logical file, obtain the authority to the based-on files of the logical file. Try the operation again.DTS-RETRY-ERR-0616:ERR DUMP payload version or checksum are wrong
可能原因:源Redis和目标Redis的版本不兼容,导致DTS无法使用RESTORE命令直接迁移二进制数据。
解决方法:此问题无法通过重启任务解决。建议采取以下任一措施:
将目标Redis实例的版本调整为与源库兼容或相同的版本。
重新配置DTS任务,选择全量同步+增量同步的同步模式。
报错示例:
DTS-100047: retry 614 times, 43208 seconds, which exceed the supposed 43200 seconds cause: JedisDataException: ERR DUMP payload version or checksum are wrongDTS-RETRY-ERR-0617:ERR invalid DB index
可能原因:DTS尝试在目标Redis中选择一个非法的DB。
解决方法:请在DTS任务配置中,使用库表映射功能将源DB映射到目标Redis允许的DB上。
报错示例:
DTS-100047: retry 60 times, 601 seconds, which exceed the supposed 43200 seconds cause: RedisCannotRetryException: redis: DTS-10009: Exception in stage: [begin], errMsg: ERR invalid DB index JedisDataException: ERR invalid DB indexDTS-RETRY-ERR-0618:ERR-CODE: [TDDL-10000][ERR_X_PROTOCOL_BAD_PACKET]
可能原因:DTS发送的数据包大小超过了目标库的packet的限制。
解决方法:请登录目标库,适当调大packet参数的值。
报错示例:
java.sql.BatchUpdateException: [19d7e170d4402000][26.xxx.xxx.xxx:xxx][pro_scrm]ERR-CODE: [TDDL-10000][ERR_X_PROTOCOL_BAD_PACKET] X-NIO-Client /26.xxx.xxx.xxx:xxx to /10.xxx.xxx.xxx:xxx sent packet length (81341097) exceeds the allowed maximum (67108864).DTS-RETRY-ERR-0619:ERR-CODE: [TDDL-5311][ERR_GLOBAL_SECONDARY_INDEX_MODIFY_GSI_TABLE_DIRECTLY]
可能原因:DTS尝试直接修改一个全局二级索引(GSI)表,但目标库不允许此操作。
解决方法:此问题无法通过重启任务解决。全局二级索引应通过其主表进行更新。请检查您的同步对象配置,确保您同步的是主表而非GSI表。
报错示例:
java.sql.BatchUpdateException: [1a0e2bffcfb3xxxx][172.xxx.xxx.xxx:xxx][sto_sitems]ERR-CODE: [TDDL-5311][ERR_GLOBAL_SECONDARY_INDEX_MODIFY_GSI_TABLE_DIRECTLY] Does not support modify global secondary index table 'xxxx'DTS-RETRY-ERR-0622:Error on Send.This server does not host this topic-partition
可能原因:DTS内部的数据分区元数据信息过时,通常是由于内部组件(如DStore)重启或发生迁移导致。
解决方法:此为DTS内部异常。请尝试重新启动DTS任务,这会强制刷新元数据信息。
报错示例:
DTS-100047: retry 0 times, 0 seconds, which exceed the supposed 43200 seconds cause: RuntimeException: Error on Send.This server does not host this topic-partition. UnknownTopicOrPartitionException: This server does not host this topic-partition.DTS-RETRY-ERR-0623:estimated need client-output-buffer
可能原因:源端Redis为DTS分配的客户端输出缓冲区不足,导致连接被主动关闭以保护主库。
解决方法:请根据报错信息中estimated need client-output-buffer的建议值,在源端Redis中调大配置值。
报错示例:
DTS-10015: Slave connection closed by source redis, Reach 90.00% hard-client-output-buffer-limit, current size: 1453056xxx, hard limit: 1610612xxx.RDB parse progress: 28.97%, already cost time: 209, estimated finish cost time: 721, remaining time: 512, current client-output-buffer: 1453056xxx, estimated need client-output-buffer: 478x.xxMB.DTS-RETRY-ERR-0624:Expiring (.*)? record(s) (.*)?
可能原因:DTS向目标Kafka集群发送消息时长时间未收到确认,导致消息超时。这可能是由于部分Kafka Broker异常,或者DTS的IP白名单未在所有Broker上配置。
解决方法:
检查目标Kafka集群中所有Broker的健康状态。
确认DTS的IP白名单已添加到Kafka集群的所有Broker节点上。
检查DTS与Kafka集群之间的网络连接是否存在高延迟或丢包。
报错示例:
DTS-100047: retry 107 times, 43338 seconds, which exceed the supposed 43200 seconds cause: ExecutionException: com.xxx.kafka_thin_client1d0d0.common.errors.TimeoutException: Expiring xxx record(s) for dls_hr_ram_t_xxx_tp-22: 391390 ms has passed since last append TimeoutException: Expiring xxx record(s) for dls_hr_ram_t_xxx_tp-22: 391390 ms has passed since last appendDTS-RETRY-ERR-0625:Failed to calculate delete bitmap
可能原因:DTS通过StreamLoad向目标SelectDB写入数据时,SelectDB内部在计算删除位图时发生超时或错误。这表明目标SelectDB实例可能存在健康问题或内部锁争用。
解决方法:请检查目标SelectDB实例的运行状态、日志和负载情况。确认数据库恢复正常后,重新启动DTS任务。
报错示例:
java.io.IOException: Failed to stream load data to SelectDB.Key status is fail. Load result: {"Status":"Fail","Comment":"","BeginTxnTimeMs":28,"Message":"[DELETE_BITMAP_LOCK_ERROR]TStatus: errCode = 301, detailMessage = Failed to calculate delete bitmap. Timeout. Unfinished mark: 392787086xxx=23524707046653xxx","NumberUnselectedRows":0,"CommitAndPublishTimeMs":0,"Label":"DTS-SelectDB-Sink2ea489f7-1684-xxxx-xxxx-9c6adc2e36d0","LoadBytes":2285539,"StreamLoadPutTimeMs":17,"NumberTotalRows":10240,"WriteDataTimeMs":70024,"ReceiveDataTimeMs":118,"TxnId":2352470704665xxxx,"LoadTimeMs":220693,"TwoPhaseCommit":"false","ReadDataTimeMs":1,"NumberLoadedRows":10240,"NumberFilteredRows":0}DTS-RETRY-ERR-0626:Failed to create upload session with tunnel endpoint
可能原因:DTS无法为目标MaxCompute表创建数据上传通道,通常是由于网络问题或目标服务不可用。
解决方法:请检查DTS与MaxCompute服务之间的网络连接。确认网络通畅且MaxCompute服务正常后,重新启动DTS任务。
报错示例:
DTS-70004: execute upload failed, may try it again cause: TransactionReplicateException: transaction-replicate: DTS-70003: 5 execute transaction has excess max transaction retry time [100] cause:odps: DTS-11014: Failed to get the upload session for Prd_IOT_DataWorks.ods_t_ord_affiliated_deduction_log caused by ErrorCode=Local Error, ErrorMessage=Failed to create upload session with tunnel endpoint xxx ErrorCode=Local Error, ErrorMessage=Failed to create upload session with tunnel endpoint xxx CriticalAnyAllException: odps: DTS-11014: Failed to get the upload session for Prd_IOT_DataWorks.ods_t_ord_affiliated_deduction_log caused by ErrorCode=Local Error, ErrorMessage=Failed to create upload session with tunnel endpoint xxx ErrorCode=Local Error, ErrorMessage=Failed to create upload session with tunnel endpoint xxx...DTS-RETRY-ERR-0627:failed to parse field
可能原因:源库中某个字段的数据与目标Elasticsearch(ES)中定义的映射(mapping)类型不匹配,导致ES无法解析并拒绝写入。例如,将一个字符串写入一个被定义为long类型的字段。
解决方法:请检查源数据和目标ES的mapping,确保数据类型兼容。您可以选择:
修正源端数据。
修改目标ES的索引mapping,然后重新导入数据。
报错示例:
DTS-100047: retry 15 times, 43 seconds, which exceed the supposed 922337203685477xxxx seconds cause: IOException: es: DTS-10035: [1:412] failed to parse field [xxx.xxx] of type [long] in document with id 'xxx'. Preview of field's value: 'xxx', For input string: "xxx" CriticalAnyAllException: es: DTS-10035: [1:412] failed to parse field [xxx.xxx] of type [long] in document with id 'xxx'. Preview of field's value: 'xxx', For input string: "xxx"DTS-RETRY-ERR-0628:fetch target schema from target table (.*)? failed
可能原因:DTS无法获取目标数据库的表结构信息。这可能是因为目标表不存在,或者DTS使用的数据库账号没有足够的权限来查看表结构。
解决方法:
确认目标库中是否存在该表。
检查并确保DTS同步账号拥有对该表的
DESCRIBE或等效权限。确认无误后,重新启动DTS任务。
报错示例:
DTS-100047: retry 0 times, 0 seconds, which exceed the supposed 43200 seconds cause: AnyCoreException: framework: DTS-31008: fetch target schema from target table xxx.xxx failedDTS-RETRY-ERR-0629:file (.*)? cannot be read at this time ORA-01110: data file (.*)?
可能原因:DTS在读取源端Oracle数据库时,无法读取某个数据文件(.dbf)。这通常是因为DTS账号缺少对该文件所在表空间的访问权限,或者数据库正在进行恢复、文件处于脱机状态。
解决方法:请检查源端Oracle数据库的状态,确保数据库处于正常打开状态。同时,请确认DTS使用的账号具有访问所需表空间的权限。确认无误后,重新启动DTS任务。
报错示例:
DTS-30020: execute sql:/* DTS-full-xa7m7590u2xxxx */SELECT xxx FROM xxx.xxx --> (null, null) record range id = 25182463197225xxxx, fetchSize = 1024 cause: SQLException: ORA-00376: file xxx cannot be read at this time ORA-01110: data file 13: 'xxx'DTS-RETRY-ERR-0630:get table read lock timeout
可能原因:DTS通过StreamLoad向目标SelectDB写入数据时,因无法在超时时间内获取到表的读锁而失败。这通常表明目标表上有长时间运行的DDL操作或其他阻塞性任务。
解决方法:请检查目标SelectDB实例,查看是否有长时间运行的ALTER TABLE等操作。待阻塞任务完成后,重新启动DTS任务。
报错示例:
java.io.IOException: Failed to stream load data to SelectDB.Key status is fail. Load result: {"Status":"Fail","Comment":"","BeginTxnTimeMs":11,"Message":"[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = get table read lock timeout, database=xxx,table=xxx","NumberUnselectedRows":0,"CommitAndPublishTimeMs":0,"Label":"DTS-SelectDB-Sink1b0aec5d-xxxx-xxxx-b9f1-9a1e62dde118","LoadBytes":0,"StreamLoadPutTimeMs":60000,"NumberTotalRows":0,"WriteDataTimeMs":0,"ReceiveDataTimeMs":0,"TxnId":2581391489973xxxx,"LoadTimeMs":0,"TwoPhaseCommit":"false","ReadDataTimeMs":0,"NumberLoadedRows":0,"NumberFilteredRows":0}DTS-RETRY-ERR-0631:given ts is smaller than oldest message's ts, some binlogs may lose
可能原因:DTS尝试从指定的起始时间点(timestamp)在源端Kafka(用于TiDB Binlog)中拉取数据,但发现Kafka中最早的消息也晚于该时间点。这意味着所需的增量日志已经被清理。
解决方法:请检查源端Kafka中TiDB Binlog数据的保留策略,确保其保留时间足够长。
报错示例:
DTS-31009: In process of processing data (recordRange: 776029846907125xxxx) failed cause: FatalAnyAllException: common: DTS-100047: retry 0 times, 0 seconds, which exceed the supposed 43200 seconds RuntimeException: seek timestamp for topic [xxx] with timestamp [1747724390xxx] failed RuntimeException: given ts is smaller than oldest message's ts, some binlogs may lose, given ts: 1747724390xxx, oldest ts: 1747726918xxxDTS-RETRY-ERR-0632:got sequence entry (.*)? for toast chunk (.*)?
可能原因:源端PostgreSQL数据库在处理大对象(TOAST)数据时,内部逻辑出现错误,导致DTS收到了乱序或非预期的日志条目。这是一个数据库自身的异常。
解决方法:这是一个数据库内部错误,请检查源端PostgreSQL的健康状态和错误日志。
报错示例:
DTS-52111: Increment Context Is Not Running..: cause: CriticalAnyAllException: postgresql-reader: DTS-52510: Fetch postgresql logical log failed FatalAnyAllException: common: DTS-100047: retry 58 times, 600947 seconds, which exceed the supposed 600 seconds IOException: org.postgresql.util.PSQLException: ERROR: got sequence entry xxx for toast chunk xxx instead of seq 0 PSQLException: ERROR: got sequence entry xxx for toast chunk xxx instead of seq 0DTS-RETRY-ERR-0633:Got timeout reading communication packets
可能原因:DTS与源端MySQL数据库之间的网络连接出现读取超时。这通常是由于网络延迟高、不稳定,或数据库服务器负载过重导致响应缓慢。
解决方法:请排查DTS与源库之间的网络链路质量。如果网络正常,请检查数据库服务器的负载情况。重新启动DTS任务可以解决瞬时的网络问题。
报错示例:
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Got timeout reading communication packetsDTS-RETRY-ERR-0635:Host (.*)? is not allowed to connect to this (.*)? server
可能原因:目标端的IP白名单未包含DTS服务器的IP地址,导致连接被数据库拒绝。
解决方法:请将DTS服务器的IP地址添加到目标数据库实例的IP白名单中。
报错示例:
java.sql.SQLException: null, message from server: "Host '100.xxx.xxx.xxx' is not allowed to connect to this xxx server"DTS-RETRY-ERR-0636:ignore with global index not supported
可能原因:目标PolarDB-X 2.0数据库的表上存在全局索引(Global Index),而DTS使用的INSERT IGNORE语句在这种情况下不被支持。
解决方法:这是一个数据库限制。如果业务允许,您可以考虑删除目标表的全局索引,或者将分区表改为非分区表。修改数据库结构后,重新启动DTS任务。
报错示例:
DTS-30011: currentRunningSQL: /* DTS-full-ch7c7pjz16mxxxx */insert ignore into `xxx`.`xxx`() VALUES (),currentRow:,
reason: ignore with global index not supported cause: BatchUpdateException: ignore with global index not supported SQLException: ignore with global index not supportedDTS-RETRY-ERR-0638:immediate shutdown in progress - no operations are permitted
可能原因:DTS尝试连接的Oracle数据库正在执行紧急关闭(immediate shutdown),在此过程中不接受任何新操作。
解决方法:请等待Oracle数据库完成关闭并重新启动。待数据库恢复正常服务后,重新启动DTS任务。
报错示例:
java.sql.SQLRecoverableException: ORA-01089: immediate shutdown in progress - no operations are permittedDTS-RETRY-ERR-0639:insufficient privileges
可能原因:DTS用于连接源端Oracle数据库的账号缺少必要的权限,例如访问系统数据字典视图(如sys.USER$)的权限。
解决方法:请参照Oracle数据库的限制和准备工作,为源端Oracle的DTS专用账号授予所有必需的权限。
报错示例:
DTS-52110: OracleRecordExtractor Init Error: oracle-reader: DTS-52050: Mirror Meta error, cause: CriticalAnyAllException: oracle-reader: DTS-52050: Mirror Meta error, CriticalAnyAllException: oracle-reader: DTS-52050: Mirror meta failed CriticalAnyAllException: oracle-reader: DTS--0002: execute sql SELECT USER#,NAME,TYPE# FROM sys.USER$ failed, case by common: DTS-100047: retry 4307 times, 43200410 seconds, which exceed the supposed 43200 seconds FatalAnyAllException: common: DTS-100047: retry 4307 times, 43200410 seconds, which exceed the supposed 43200 seconds SQLSyntaxErrorException: ORA-01031: insufficient privilegesDTS-RETRY-ERR-0640:Insufficient result space to convert uniqueidentifier value to char.
可能原因:DTS在读取SQL Server数据时,用于存放uniqueidentifier类型转换后结果的内部缓冲区空间不足。这可能是DDL变更后DTS未及时同步元数据导致的内部问题。
解决方法:请去确认业务是否正在做过DDL变更,并尝试重新启动DTS任务。
报错示例:
DTS-31009: read source data error cause: SQLServerException: Insufficient result space to convert uniqueidentifier value to char.DTS-RETRY-ERR-0641:invalid input syntax for type money
可能原因:源库和目标库对于money类型的单位不同,导致DTS写入的货币格式字符串(如¥118,971.00)不被目标库接受。
解决方法:请在源库和目标库上分别检查并统一两者的货币格式设置。修复后,重新启动DTS任务。
报错示例:
DTS-100047: retry 180 times, 1802 seconds, which exceed the supposed 1800 seconds cause: CriticalAnyAllException: framework: DTS-30011: currentRunningSQL: /* DTS-full-jw3c85ox14cxxxx */insert into "xxx"."xxx"() values () on conflict do nothing,currentRow:,
reason: Batch entry 0 /* DTS-full-jw3c85ox14cxxxx */insert into "xxx"."xxx"() values () on conflict do nothing was aborted: ERROR: invalid input syntax for type money: "¥118,971.00" Call getNextException to see other errors in the batch. BatchUpdateException: Batch entry 0 /* DTS-full-jw3c85ox14cxxx */insert into "xxx"."xxx"() values () on conflict do nothing was aborted: ERROR: invalid input syntax for type money: "¥118,971.00" Call getNextException to see other errors in the batch. PSQLException: ERROR: invalid input syntax for type money: "¥118,971.00"DTS-RETRY-ERR-0642:invalid username/password; logon denied
可能原因:DTS连接Oracle数据库时使用的用户名或密码不正确。
解决方法:请检查并更新DTS任务配置中的Oracle数据库账号和密码,确保其正确有效。确认后,重新启动DTS任务。
报错示例:
java.sql.SQLException: ORA-01017: invalid username/password; logon deniedDTS-RETRY-ERR-0643:(.*)? is an identity column defined as GENERATED ALWAYS
可能原因:目标PostgreSQL表的某个列被定义为GENERATED ALWAYS的标识列,这种列不允许显式插入值。但DTS正试图将源库的值写入该列。
解决方法:此问题无法通过重启任务解决。您需要修改目标表的结构,例如将该列改为GENERATED BY DEFAULT,或者在DTS任务的列映射中排除此列的同步。
报错示例:
DTS-30011: currentRunningSQL: /* DTS-full-nezi7395167xxxx */insert into "xxx"."xxx"() values () on conflict do nothing,currentRow:xxx, reason: Batch entry 0 /* DTS-full-nezi7395167xxx */insert into "xxx"."xxx"() values (xxx) on conflict do nothing was aborted: ERROR: cannot insert a non-DEFAULT value into column "id" Detail: Column "id" is an identity column defined as GENERATED ALWAYS. Hint: Use OVERRIDING SYSTEM VALUE to override. Call getNextException to see ...DTS-RETRY-ERR-0644:(.*)? is INTEGER, but it's defined as STRING in table meta
可能原因:待同步的数据中,某个主键列的类型(例如是整数INTEGER)与目标表格存储(Tablestore)中定义的元数据类型(例如是字符串STRING)不匹配。
解决方法:请检查并统一源数据类型与目标表格存储的表结构定义。修复后,重新启动DTS任务。
报错示例:
DTS-100047: retry 0 times, 1 seconds, which exceed the supposed 43200 seconds cause: ClientException: The type of primary key column 'xxx' is INTEGER, but it's defined as STRING in table meta.DTS-RETRY-ERR-0645:java.net.UnknownHostException
可能原因:DTS服务器无法通过DNS解析源数据库配置的域名。
解决方法:请检查源库的连接地址是否填写正确。并在DTS服务器所在的网络环境中确认该域名可以被正常解析。
报错示例:
java.sql.SQLException: java.net.UnknownHostException: database-instance-1.c96cqe6wxxxx.ap-southeast-1.rds.xxx.comDTS-RETRY-ERR-0646:java.sql.SQLException: Binary log is not open
可能原因:源端数据库未按预期开启Binlog。
解决方法:请检查并确保源端数据库已正确开启Binlog功能。确认后,重新启动DTS任务。
报错示例:
DTS-100047: retry 0 times, 1101 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: mysql-reader: DTS-52200: process error failed cause: IOException: java.sql.SQLException: Binary log is not open SQLException: Binary log is not open CriticalAnyAllException: mysql-reader: DTS-52200: process error failed IOException: java.sql.SQLException: Binary log is not open SQLException: Binary log is not openDTS-RETRY-ERR-0647:java.sql.SQLException: Not Support Unusing binary logging.
可能原因:源端MySQL数据库未开启Binlog,而DTS的增量同步依赖于此功能。
解决方法:请登录源MySQL数据库,检查并开启Binlog,然后重新启动DTS任务。
报错示例:
java.sql.SQLException: Not Support Unusing binary logging.DTS-RETRY-ERR-0648:java.sql.SQLException: The requested gtid transaction no longer exists and may have been cleaned up
可能原因:DTS进行增量同步时,需要的GTID位点在源MySQL的Binlog文件中已不存在。这通常是因为Binlog文件已被数据库自动清理(purge)。
解决方法:请延长源库的Binlog保留时间。修复后,您需要重新配置DTS任务以从头开始同步。
报错示例:
DTS-100047: retry 0 times, 1101 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: mysql-reader: DTS-52200: process error failed cause: IOException: java.sql.SQLException: The requested gtid transaction no longer exists and may have been cleaned up SQLException: The requested gtid transaction no longer exists and may have been cleaned up CriticalAnyAllException: mysql-reader: DTS-52200: process error failed IOException: java.sql.SQLException: The requested gtid transaction no longer exists and may have been cleaned up SQLException: The requested gtid transaction no longer exists and may have been cleaned upDTS-RETRY-ERR-0649:java.sql.SQLNonTransientConnectionException: Got timeout reading communication packets
可能原因:DTS在读取源端MySQL数据库的响应时发生网络超时。这可能是由于网络延迟或数据库负载过高。
解决方法:请检查DTS与源库之间的网络状况,并排查源库的性能负载。重新启动DTS任务可以解决瞬时问题。
报错示例:
java.sql.SQLNonTransientConnectionException: Got timeout reading communication packetsDTS-RETRY-ERR-0650:java.sql.SQLNonTransientConnectionException: Server shutdown in progress
可能原因:源端MySQL数据库服务器正在关闭过程中,不再接受任何操作。
解决方法:请检查源数据库的状态。待其完成关闭并重启恢复服务后,再重新启动DTS任务。
报错示例:
java.sql.SQLNonTransientConnectionException: Server shutdown in progressDTS-RETRY-ERR-0651:LeaderNotAvailableException
可能原因:目标Kafka集群正在进行分区(Partition)迁移,导致该分区暂时无法写入。这通常是由于集群节点重启或网络波动触发的。
解决方法:此问题通常是暂时的。请重新启动DTS任务,DTS会重新尝试连接并获取最新的分区Leader信息。
报错示例:
java.util.concurrent.ExecutionException: com.xxx.kafka_thin_client1d0d0.common.errors.LeaderNotAvailableException: There is no leader for this topic-partition as we are in the middle of a leadership election.DTS-RETRY-ERR-0652:listener does not currently know of SID given in connect descriptor
可能原因:源端Oracle的监听器(Listener)无法识别DTS连接请求中指定的SID(System Identifier)。这可能是因为SID配置错误、数据库实例未运行或未向监听器注册。
解决方法:请检查源数据库的连接信息,确保SID填写正确。同时,确认Oracle数据库及其监听器服务均在正常运行。确认无误后,重新启动DTS任务。
报错示例:
java.sql.SQLException: Listener refused the connection with the following error: ORA-12505, TNS:listener does not currently know of SID given in connect descriptorDTS-RETRY-ERR-0653:MySQLSyntaxErrorException: Result consisted of more than one row
可能原因:DTS在目标MySQL数据库中执行一个期望只影响单行的操作时,数据库返回了多行结果。这通常表明目标表存在重复数据,很可能是因为缺少主键或唯一约束。
解决方法:请检查目标表,根据业务主键定位并清理重复数据。同时,请确保目标表上有正确的主键或唯一索引约束。
报错示例:
DTS-100047: retry 3106 times, 43203 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: framework: DTS-30011: currentRunningSQL: /* DTS-full-y5jg7pd1138xxxx */insert ignore into `xxx`.`xxx`() VALUES (),currentRow:xxx, reason: Result consisted of more than one row BatchUpdateException: Result consisted of more than one row MySQLSyntaxErrorException: Result consisted of more than one rowDTS-RETRY-ERR-0654:new row for relation (.*)? violates check constraint
可能原因:源端的一条数据不满足目标PostgreSQL表上定义的检查约束(CHECK constraint)。
解决方法:请在源库中定位并修正不符合业务规则的数据。或者,如果业务逻辑允许,可以考虑修改或移除目标表的检查约束。数据或表结构调整后,重新启动DTS任务。
报错示例:
DTS-100047: retry 4247 times, 43207 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: framework: DTS-30011: currentRunningSQL: /* DTS-full-j87z7m2b16dxxxx */insert into "xxx"."xxx"() values () on conflict do nothing,currentRow:xxx, reason: Batch entry 32 /* DTS-full-j87z7m2b16dxxxx */insert into "xxx"."xxx"() values () on conflict do nothing was aborted: ERROR: new row for relation "xxx" violates check constraint "xxx_constraint" Detail: Failing row contains (). Call getNextException to see other errors in the batch.
BatchUpdateException: Batch entry 32 /* DTS-full-j87z7m2b16dxxxx */insert into "xxx"."xxx"() values (xxx) on conflict do nothing was aborted: ERROR: new row for relation "xxx" violates check constraint "xxx_constraint" Detail: Failing row contains (). Call getNextException to see other errors in the batch. PSQLException: ERROR: new row for relation "xxx" violates check constraint "xxx_constraint" Detail: Failing row contains ().DTS-RETRY-ERR-0655:No more data to read from socket
可能原因:DTS与Oracle数据库之间的网络连接被意外关闭。这可能是由于数据库端进程异常、网络设备(如防火墙)因空闲超时而切断会话或网络瞬时中断。
解决方法:请检查DTS与Oracle服务器之间的网络稳定性,并确认防火墙没有中断长连接。同时,检查Oracle的告警日志排查服务端异常。重新启动DTS任务通常可以解决瞬时问题。
报错示例:
DTS-30020: execute sql:/* DTS-full-etzf7t4x24xxxxx */SELECT /*+ index("xxx" SYS_C0044122) */ xxx FROM xxx WHERE xxx --> (null, [Lcom.alibaba.amp.any.record.value.Value;@20510xxx) record range id = 712551168722599xxxx, fetchSize = 1024 cause: SQLRecoverableException: No more data to read from socketDTS-RETRY-ERR-0656:no privileges on tablespace (.*)?
可能原因:连接目标Oracle数据库的账号缺少在指定表空间(Tablespace)上创建对象的权限。
解决方法:请为DTS使用的目标库账号授予对相关表空间的权限。
报错示例:
DTS-100047: retry 4314 times, 43201 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: framework: DTS-30011: currentRunningSQL: /* DTS-full-r87i6i04j33xxxx */insert into "xxx"."xxx"() VALUES (),currentRow:xxx, reason: ORA-01950: no privileges on tablespace 'xxx' BatchUpdateException: ORA-01950: no privileges on tablespace 'xxx'DTS-RETRY-ERR-0657:no privileges on tablespace (.*)?
可能原因:连接目标数据库的账号没有在指定表空间(Tablespace)上创建对象的权限。
解决方法:请为DTS使用的目标库账号授予对相关表空间的权限。
报错示例
DTS-30011: currentRunningSQL: /* DTS-full-q0yy62nq16fxxx */insert into "xxx"."xxx"() VALUES (),currentRow:null, reason: ORA-01950: no privileges on tablespace 'xxx' cause: BatchUpdateException: ORA-01950: no privileges on tablespace 'xxx'DTS-RETRY-ERR-0658:No route to host (Host unreachable)
可能原因:DTS无法找到到达源数据库主机的网络路由,导致连接失败。这通常是由于网络配置错误、防火墙策略或子网间路由不通。
解决方法:请检查DTS服务器与源数据库服务器之间的网络配置和连通性。可以尝试在DTS服务器上使用ping或telnet命令测试目标主机的可达性。
报错示例:
java.sql.SQLException: java.net.NoRouteToHostException: No route to host (Host unreachable)DTS-RETRY-ERR-0659:Not allowed to perform current operation on async materialized view
可能原因:DTS尝试通过StreamLoad方式向目标SelectDB(Doris)的一个异步物化视图(async materialized view)写入数据,但该操作不被允许。数据应写入基表。
解决方法:请修改DTS任务配置,将同步对象设置为异步物化视图对应的基表,而不是物化视图本身。
报错示例:
DTS-70003: 29 execute transaction has excess max transaction retry time [64] cause:Failed to stream load data to table: xxx.xxx status is fail. Load result: {"Status":"Fail","Comment":"","BeginTxnTimeMs":0,"Message":"[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = Not allowed to perform current operation on async materialized view","NumberUnselectedRows":0,"CommitAndPublishTimeMs":0,"Label":"DTS-SelectDB-Sink4b5a1dcd-xxx-xxx-88ad-d9ce08bcf343","LoadBytes":0,"StreamLoadPutTimeMs":0,"NumberTotalRows":0,"WriteDataTimeMs":0,"ReceiveDataTimeMs":0,"TxnId":-1,"LoadTimeMs":0,"TwoPhaseCommit":"false","ReadDataTimeMs":0,"NumberLoadedRows":0,"NumberFilteredRows":0} cause: IOException: Failed to stream load data to table: xxx.xxx status is fail. Load result: {"Status":"Fail","Comment":"","BeginTxnTimeMs":0,"Message":"[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = Not allowed to perform current operatio...DTS-RETRY-ERR-0660:not enough space. (NOT_ENOUGH_SPACE)
可能原因:目标ClickHouse实例或其所在节点的磁盘空间不足,无法为写入操作预留所需内存或写入数据。
解决方法:请检查目标ClickHouse实例的存储容量,并清理或扩容磁盘空间。
报错示例:
DTS-100047: retry 1044 times, 43202 seconds, which exceed the supposed 43200 seconds cause: TransactionReplicateException: transaction-replicate: DTS-70003: 5 execute transaction has excess max transaction retry time [64] cause:transaction-replicate: DTS-70004: EXEC statement failed, may try it again RecoverableAnyAllException: transaction-replicate: DTS-70004: EXEC statement failed, may try it again SQLException: ClickHouseEngine(5): meet error, insertSQL:insert into `xxx`.`xxx` () values (), error detail message:Code: 243. DB::Exception: Cannot reserve 1.00 MiB, not enough space. (NOT_ENOUGH_SPACE) (version 23.8.16.1) , server ClickHouseNode [uri=http://10.xxx.xxx.xxx:xxx/x_magnet_dev]@-184683xxxx SQLException: Code: 243. DB::Exception: Cannot reserve 1.00 MiB, not enough space. (NOT_ENOUGH_SPACE) (version 23.8.16.1) , server ClickHouseNode [uri=http://10.xxx.xxx.xxx:xxx/x_magnet_dev]@-184683xxx IOException: Code: 243. DB::Exception: Cannot reserve 1.00 MiB, not enough space. (NOT_ENOUGH_SPACE) (version 23.8.16.1)DTS-RETRY-ERR-0661:not support exactly-once with thread id mode
可能原因:DTS任务配置了Exactly-Once写入功能,但源数据库未开启GTID。该模式强依赖GTID来保证事务的唯一性。
解决方法:请为源数据库开启GTID功能。
报错示例:
DTS-11005: not support exactly-once with thread id modeDTS-RETRY-ERR-0662:Not support fetch backup log MissActiveLogException
可能原因:DTS在从源库读取增量日志时,发现需要的日志(包括活动日志和备份日志)已被清理。
解决方法:请延长源库的事务日志备份保留周期。
报错示例:
DTS-100047: retry 0 times, 107903 seconds, which exceed the supposed 43200 seconds cause: RecoverableAnyAllException: dts-k-src: DTS-52110: SQLServerRecordExtractor Init Error: sqlserver-reader: DTS-52061: Failed to seek sqlserver position CriticalAnyAllException: sqlserver-reader: DTS-52061: Failed to seek sqlserver position CriticalAnyAllException: sqlserver-reader: DTS-52411: Not support fetch backup log MissActiveLogException: ERROR_MISS_TRANSACTION_LOG(timestamp:1749775487)DTS-RETRY-ERR-0663:Object (.*)? in library (.*)? not found
可能原因:DTS在源库中找不到指定的对象(例如Journal日志文件)。这通常意味着DTS需要的日志已被清理或移动。
解决方法:请检查源库的日志管理策略,确保日志保留时间足够长。
报错示例:
DTS-31009: In process of processing data (recordRange: 456002531641996xxxx) failed cause: FatalAnyAllException: common: DTS-100047: retry 0 times, 276393 seconds, which exceed the supposed 43200 seconds RuntimeException: AS400DB2Source:journalFetcher-0: failed cause false; rc=0x501 CPF9801: Object xxx in library xxx not found.DTS-RETRY-ERR-0664:ODPS partition exception - maximum (.*)? partitions allowed, or more than (.*)? partitions created
可能原因:目标MaxCompute表的分区数量达到了上限,或者一次操作试图创建的分区数超过了单次限制。
解决方法:请检查您的分区策略,避免产生过多分区。您可能需要清理不需要的历史分区,或优化分区字段的设计。
报错示例:
DTS-70004: execute upload failed, may try it again cause: TransactionReplicateException: transaction-replicate: DTS-70003: 17 execute transaction has excess max transaction retry time [100] cause:ODPS-0110061: Failed to run ddltask - Modify DDL meta encounter exception : ODPS-0123031:ODPS partition exception - maximum xxx partitions allowed, or more than xxx partitions created OdpsException: ODPS-0110061: Failed to run ddltask - Modify DDL meta encounter exception : ODPS-0123031:ODPS partition exception - maximum xxx partitions allowed, or more than xxx partitions createdDTS-RETRY-ERR-0665:Operation is not allowed under READ-ONLY mode!
可能原因:目标AnalyticDB MySQL 3.0实例当前处于只读模式,DTS无法写入数据。
解决方法:请检查目标AnalyticDB MySQL 3.0实例的状态。
报错示例:
java.sql.SQLException: [xxx, 202507301xxxx41921680681130315111xxxx] Operation is not allowed under READ-ONLY mode!DTS-RETRY-ERR-0666:ORACLE initialization or shutdown in progress
可能原因:DTS尝试连接的Oracle数据库正在进行初始化或关闭操作,暂时无法提供服务。
解决方法:请等待Oracle数据库完成启动或关闭流程并恢复正常服务。之后,重新启动DTS任务。
报错示例:
java.sql.SQLRecoverableException: ORA-01033: ORACLE initialization or shutdown in progressDTS-RETRY-ERR-0669:org.bson.json.JsonParseException
可能原因:DTS任务中为MongoDB源配置的过滤条件不是一个合法的JSON格式字符串。
解决方法:请检查并修正DTS任务过滤条件中填写的JSON语句,确保其语法正确。
报错示例:
org.bson.json.JsonParseException: Invalid JSON input. Position: 2. Character: '“'.DTS-RETRY-ERR-0670:PROCEDURE (.*)? does not exist
可能原因:DTS同步的数据触发了目标MySQL数据库中的某个操作(例如触发器),该操作试图调用一个不存在的存储过程。
解决方法:请检查目标数据库,确保所需的存储过程已创建,或者移除调用该存储过程的触发器等逻辑。修复后,重新启动DTS任务。
报错示例:
DTS-100047: retry 4290 times, 43205 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: framework: DTS-30011: currentRunningSQL: /* DTS-full-lsqb6q1p18txxx */insert ignore into `xxx`.`xxx`() VALUES (),currentRow:xxx, reason: PROCEDURE xxx.xxx does not exist BatchUpdateException: PROCEDURE xxx.xxx does not exist SQLSyntaxErrorException: PROCEDURE xxx.xxx does not existDTS-RETRY-ERR-0671:Project: (.*)? Slot Quota Exceeded
可能原因:目标MaxCompute消耗的计算资源(Slot)超出了其配额(Quota)。
解决方法:请检查MaxCompute项目的资源使用情况。您可以等待其他任务释放资源,或提升该项目的Slot配额。确认无误后,重新启动DTS任务。
报错示例:
DTS-70004: execute upload failed, may try it again cause: TransactionReplicateException: transaction-replicate: DTS-70003: 12 execute transaction has excess max transaction retry time [100] cause:odps: DTS-11014: Failed to get the upload session for xxx.xxx caused by RequestId=20250601131605b381c40b05d6xxxx, ErrorCode=SlotExceeded, ErrorMessage=Region: cn-zhangjiakou Project: xxx Slot Quota Exceeded. Total:300, Acquired:349RequestId=20250601131605b381c40b05d6xxxx, ErrorCode=SlotExceeded, ErrorMessage=Region: cn-zhangjiakou Project: xxxx Slot Quota Exceeded. Total:300, Acquired:349 CriticalAnyAllException: odps: DTS-11014: Failed to get the upload session for xxx.xxx caused by RequestId=20250601131605b381c40b05d6xxxx, ErrorCode=SlotExceeded, ErrorMessage=Region: cn-zhangjiakou Project: xxx Slot Quota Exceeded. Total:3...DTS-RETRY-ERR-0672:Query was selected to stop by OvercommitTracker
可能原因:目标ClickHouse实例因内存超用,其内部的OvercommitTracker机制终止了DTS的查询。
解决方法:请检查目标ClickHouse实例的内存使用情况和负载。您可能需要优化查询、增加节点内存或升级实例规格。
报错示例:
java.sql.BatchUpdateException: Code: 241. DB::Exception: Memory limit (total) exceeded: would use xxx GiB (attempt to allocate chunk of xxx bytes), maximum: xxx GiB. OvercommitTracker decision: Query was selected to stop by OvercommitTracker.: (avg_value_size_hint = xxx, avg_chars_size = xxx, limit = xxx): (while reading column order_sn): (while reading from part /clickhouse/xxx/xxx/store/xxx/c3ff3852-5f5d-xxx-xxx-95e34f91fdf8/all_xxx/ in table xxx (c3ff3852-5f5d-xxx-xxx-95e34f91fdf8) located on disk default of type local, from mark 192 with max_rows_to_read = xxx): While executing MergeTreeThread: while pushing to view xxx (2b1cfb37-6694-xxx-xxx-785513b23836). (MEMORY_LIMIT_EXCEEDED) (version 23.8.16.1) , server ClickHouseNode [uri=http://10.xxx.xxx.xxx:xxx/default]@742123xxxDTS-RETRY-ERR-0673:READONLY You can't write against a read only replica.
可能原因:DTS被配置向一个只读的Redis副本(Replica)节点写入数据。
解决方法:请修改DTS任务的目标端配置,将其指向Redis的主节点(Master)。
报错示例:
redis.clients.jedis.exceptions.JedisDataException: READONLY You can't write against a read only replica.DTS-RETRY-ERR-0674:RecordTooLargeException
可能原因:DTS尝试发送到目标Kafka的一条消息的大小,超过了Kafka的最大投递消息大小限制。
解决方法:检查源端数据,确认是否存在包含超大字段(如巨大的TEXT或BLOB)的记录。如果数据正常,请调大目标Kafka的最大投递消息大小限制。您也可以在DTS任务中配置对象列映射,剔除导致消息过大的字段。
报错示例:
DTS-100047: retry 4102 times, 43206 seconds, which exceed the supposed 43200 seconds cause: ExecutionException: com.xxx.kafka_thin_xxx.common.errors.RecordTooLargeException: The request included a message larger than the max message size the server will accept. RecordTooLargeException: The request included a message larger than the max message size the server will accept.DTS-RETRY-ERR-0675:rejected execution of primary operation
可能原因:目标Elasticsearch因资源压力过大(如CPU、内存或磁盘使用率过高),触发了熔断机制,拒绝了DTS的写入请求。
解决方法:请检查目标Elasticsearch的健康状态和资源负载。您可能需要扩容集群、优化索引或降低写入速率。
报错示例:
DTS-100047: retry 15 times, 219 seconds, which exceed the supposed 922337203685477xxxx seconds cause: IOException: es: DTS-10035: Index error: rejected execution of primary operation [coordinating_and_primary_bytes=xxx, replica_bytes=0, all_bytes=xxx, primary_operation_bytes=xxx, max_coordinating_and_primary_bytes=xxx] CriticalAnyAllException: es: DTS-10035: Index error: rejected execution of primary operation [coordinating_and_primary_bytes=xxx, replica_bytes=0, all_bytes=141360231, primary_operation_bytes=xxx, max_coordinating_and_primary_bytes=xxx]DTS-RETRY-ERR-0676:Requests rate limit exceeded
可能原因:DTS向目标Elasticsearch实例的写入速率超过了服务设置的请求频率限制。
解决方法:请在DTS任务配置中设置合理的同步限速(RPS),或者升级目标Elasticsearch实例以获取更高的吞吐能力。
报错示例:
DTS-100047: retry 15 times, 79 seconds, which exceed the supposed 922337203685477xxx seconds cause: IOException: es: DTS-10035: Index error: Error from remote service: {"error": {"code": "Throttling.RateQuota", "message": "Requests rate limit exceeded, please try again later."}} CriticalAnyAllException: es: DTS-10035: Index error: Error from remote service: {"error": {"code": "Throttling.RateQuota", "message": "Requests rate limit exceeded, please try again later."}}DTS-RETRY-ERR-0677:rollback segment number (.*)? with name "(.*)?" too small
可能原因:Oracle的ORA-01555: snapshot too old错误。DTS执行的全量查询时间过长,在此期间源库发生了大量数据变更,导致Oracle无法利用UNDO信息构造查询开始时的一致性数据视图。
解决方法:请增大源端Oracle数据库的UNDO表空间大小。
报错示例:
DTS-31009: In process of processing data (recordRange: 868490004694617xxxx) failed cause: FatalAnyAllException: common: DTS-100047: retry 0 times, 48910 seconds, which exceed the supposed 43200 seconds CriticalAnyAllException: framework: DTS-31009: read source data error SQLException: ORA-01555: snapshot too old: rollback segment number xxx with name "xxx" too smallDTS-RETRY-ERR-0678:routing is required
可能原因:目标Elasticsearch索引强制要求在写入文档时提供_routing值,但DTS任务没有配置相应的路由规则。
解决方法:请在DTS任务中,根据您的Elasticsearch索引设计,配置正确的路由字段。
报错示例:
DTS-100047: retry 15 times, 15 seconds, which exceed the supposed 922337203685477xxxx seconds cause: IOException: es: DTS-10035: routing is required for [xxx]/[xxx] CriticalAnyAllException: es: DTS-10035: routing is required for [xxx]/[xxx]DTS-RETRY-ERR-0679:Row size too large
可能原因:DTS尝试写入的一行数据总大小超过了目标MySQL表所允许的最大行大小限制。
解决方法:请检查并优化源表和目标表的结构。可以将部分VARCHAR长字段修改为TEXT或BLOB类型,因为这些类型的数据不完全计入行大小限制。修改表结构后,重新启动DTS任务。
报错示例:
DTS-100047: retry 112 times, 1203 seconds, which exceed the supposed 1200 seconds cause: CriticalAnyAllException: framework: DTS-30011: currentRunningSQL: /* DTS-full-dedl7sabm8gxxxx */insert ignore into `xxx`.`xxx`() VALUES (),currentRow:xxx, reason: Row size too large (> xxx). Changing some columns to TEXT or BLOB or using ROW_FORMAT=DYNAMIC or ROW_FORMAT=COMPRESSED may help. In current row format, BLOB prefix of 768 bytes is stored inline. BatchUpdateException: Row size too large (> xxx). Changing some columns to TEXT or BLOB or using ROW_FORMAT=DYNAMIC or ROW_FORMAT=COMPRESSED may help. In current row format, BLOB prefix of 768 bytes is stored inline. MySQLSyntaxErrorException: Row size too large (> xxx). Changing some columns to TEXT or BLOB or using ROW_FORMAT=DYNAMIC or ROW_FORMAT=COMPRESSED may help. In current row format, BLOB prefix of 768 bytes is stored inline.DTS-RETRY-ERR-0680:Running in read-only mode
可能原因:目标MySQL数据库实例当前处于只读模式(read_only=ON),DTS无法写入数据。
解决方法:请检查目标数据库的状态,将其设置为可读写模式后,重新启动DTS任务。
报错示例:
DTS-100047: retry 4201 times, 43203 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: framework: DTS-30011: currentRunningSQL: /* DTS-full-ia8o8et1129xxxx */insert ignore into `xxx`.`xxx`() VALUES (),currentRow:null, reason: Running in read-only mode BatchUpdateException: Running in read-only mode SQLException: Running in read-only modeDTS-RETRY-ERR-0681:Schema (.*)? not found
可能原因:DTS访问数据库时,找不到指定的Schema(在某些数据库中等同于库或用户)。这可能是因为Schema不存在,或DTS账号没有访问该Schema的权限。
解决方法:请确保DTS配置的Schema名称正确,并且DTS账号已被授予访问该Schema的权限。
报错示例:
DTS-31009: In process of processing data (recordRange: 232142460080776xxxx) failed cause: FatalAnyAllException: common: DTS-100047: retry 0 times, 2 seconds, which exceed the supposed 43200 seconds DbException: Schema "xxx" not found [90079-193] JdbcSQLException: Schema "xxx" not found [90079-193]DTS-RETRY-ERR-0682:Server shutdown in progress
可能原因:源端MySQL数据库服务器正在关闭过程中,不再接受任何操作。
解决方法:请检查源数据库的状态。待其完成关闭并重启恢复服务后,再重新启动DTS任务。
报错示例:
DTS-31009: read source data error cause: MySQLNonTransientConnectionException: Server shutdown in progressDTS-RETRY-ERR-0683:set connection_id fail
可能原因:DTS在连接PolarDB的多主集群(Limitless)时,设置connection_id失败。这可能表明目标数据库版本或配置不支持此特定操作。
解决方法:此为DTS与特定数据库版本的兼容性问题,请确认数据库是否支持。
报错示例:
DTS-70032: set connection_id fail exceed retry time=4, threadid=4294963209, biId=264, adaptedBiId=9, fatal error, let's exitDTS-RETRY-ERR-0684:SQLException: could not find next log
可能原因:DTS在按顺序读取源MySQL的Binlog文件时,找不到下一个预期的文件(例如,从mysql-bin.000905之后找不到.000906)。这通常意味着Binlog文件已被数据库手动或自动清理。
解决方法:请延长源库的Binlog保留策略。您需要重新启动DTS任务或通过跳过位点的方式(有数据丢失风险)来恢复任务。
报错示例:
DTS-100047: retry 0 times, 1000 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: mysql-reader: DTS-52200: process error failed cause: IOException: java.sql.SQLException: could not find next log; the first event 'mysql-bin.000345' at 763652xxx, the last event read from './mysql-bin.000905' at 1073755xxx, the last byte read from './mysql-bin.000905' at 1073755xxx. SQLException: could not find next log; the first event 'mysql-bin.000345' at 763652xxx, the last event read from './mysql-bin.000905' at 1073755xxx, the last byte read from './mysql-bin.000905' at 1073755xxx. CriticalAnyAllException: mysql-reader: DTS-52200: process error failed IOException: java.sql.SQLException: could not find next log; the first event 'mysql-bin.000345' at 763652xxx, the last event read from './mysql-bin.000905' at 1073755xxx, the last byte read from './mysql-bin.000905' at 1073755xxx. SQLException: could not find next log; the first event 'mysql-bin.000345' at 763652xxx, the last event read from './mysql-bin.000905' at 1073755xxx, the last byte read from './mysql-bin.000905' at 1073755xxx.DTS-RETRY-ERR-0685:SQLException: Operand should contain 1 column
可能原因:DTS构造的SQL查询中,子查询返回了多于一列,而其上下文环境只允许单列。这通常发生在DTS配置了包含语法错误的过滤条件时。
解决方法:请检查DTS任务中为源对象配置的筛选条件,确保其符合SQL语法,特别是子查询的用法。
报错示例:
DTS-31009: In process of processing data (recordRange: 692535449456161xxxx) failed cause: FatalAnyAllException: common: DTS-100047: retry 4296 times, 43209 seconds, which exceed the supposed 43200 seconds CriticalAnyAllException: framework: DTS-30020: execute sql:/* DTS-full-h78o5f6927bxxxx */SELECT xxx FROM xxx WHERE ( xxx ) --> (null, null) record range id = 692535449456161xxx, fetchSize = -214748xxx SQLException: Operand should contain 1 column(s)DTS-RETRY-ERR-0686:SSLProtocolException: Received fatal alert: unexpected_message
可能原因:DTS与源端PostgreSQL数据库进行SSL加密通信时,收到了一个不符合SSL协议规范的意外消息,导致握手失败。
解决方法:请检查DTS与源库之间的网络是否存在异常。确认无误后,重新启动DTS任务。
报错示例:
DTS-52111: Increment Context Is Not Running..: cause: CriticalAnyAllException: postgresql-reader: DTS-52510: Fetch postgresql logical log failed FatalAnyAllException: common: DTS-100047: retry 0 times, 851577266 seconds, which exceed the supposed 600 seconds IOException: org.postgresql.util.PSQLException: Database connection failed when reading from copy PSQLException: Database connection failed when reading from copy SSLProtocolException: Received fatal alert: unexpected_messageDTS-RETRY-ERR-0687:STANDBY_MAX_DATA_DELAY of (.*)? seconds exceeded
可能原因:DTS连接的是一个配置了数据保护模式的Oracle备库(Standby)。主备库之间的同步延迟超过了STANDBY_MAX_DATA_DELAY参数设定的阈值,导致备库拒绝了查询以保证数据的一致性。
解决方法:请检查并解决Oracle主备库之间的延迟问题。或者,如果业务允许,可以考虑连接到主库进行同步。
报错示例:
java.sql.SQLException: ORA-03172: STANDBY_MAX_DATA_DELAY of xxx seconds exceededDTS-RETRY-ERR-0688:String or binary data would be truncated in table (.*)?, column '(.*)?. Truncated value
可能原因:源数据某个字段的长度超过了目标SQL Server表中对应列的最大长度,导致数据在写入时将被截断。
解决方法:请对比源表和目标表的结构。您需要加长目标表中对应列的长度(例如,从VARCHAR(50)增加到VARCHAR(200))以容纳源数据。修改表结构后,重新启动DTS任务。
报错示例:
java.sql.BatchUpdateException: String or binary data would be truncated in table 'xxx', column 'xxx'. Truncated value: 'xxx'.DTS-RETRY-ERR-0689:Subquery returned more than 1 value.
可能原因:DTS执行的SQL语句中包含一个子查询,该子查询作为表达式使用时返回了多于一个值,这在SQL中是不允许的。这通常是由于目标库上存在触发器或外键逻辑,在DTS写入时触发了有问题的子查询。
解决方法:请检查目标数据库上与同步表相关的触发器、外键等自动化逻辑,确保它们不会在DTS写入时产生此类错误。建议在同步/迁移期间禁用这些逻辑。
报错示例:
com.microsoft.sqlserver.jdbc.SQLServerException: Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression.DTS-RETRY-ERR-0690:Table has no partition for value
可能原因:DTS尝试写入的数据,其分区键的值在目标MySQL分区表中找不到对应的分区来存放。
解决方法:请根据源数据的分区键值,检查并修改目标表的分区定义,确保所有可能的数据都能被路由到合适的分区。例如,为未来的时间范围添加新的分区。
报错示例:
DTS-077100: Record Replicator error in table xxx. cause by [java.sql.SQLException: Table has no partition for value from column_list] About more information in [https://xxx].DTS-RETRY-ERR-0691:table not found
可能原因:DTS尝试向目标SelectDB(Doris)写入数据,但目标库中不存在指定的表。
解决方法:请检查DTS任务配置的同步对象是否正确,并确保目标数据库中已创建了对应的表。
报错示例:
java.io.IOException: Failed to stream load data to SelectDB.Key status is fail. Load result: {"Status":"Fail","Comment":"","BeginTxnTimeMs":0,"Message":"[NOT_FOUND]TStatus: errCode = 7, detailMessage = table not found, tableName=xxx","NumberUnselectedRows":0,"CommitAndPublishTimeMs":0,"Label":"DTS-SelectDB-Sinkc16c2c5d-xxx-xxx-xxx-5a647cb41cf1","LoadBytes":0,"StreamLoadPutTimeMs":0,"NumberTotalRows":0,"WriteDataTimeMs":0,"ReceiveDataTimeMs":0,"TxnId":-1,"LoadTimeMs":0,"TwoPhaseCommit":"false","ReadDataTimeMs":0,"NumberLoadedRows":0,"NumberFilteredRows":0}DTS-RETRY-ERR-0692:the database system is in recovery mode
可能原因:DTS尝试连接的PostgreSQL数据库当前正处于恢复模式,无法接受普通连接。
解决方法:请等待数据库完成恢复过程。待数据库恢复正常服务后,重新启动DTS任务。
报错示例:
org.postgresql.util.PSQLException: FATAL: the database system is in recovery modeDTS-RETRY-ERR-0693:the database system is not yet accepting connections
可能原因:DTS尝试连接的PostgreSQL数据库正在启动,但尚未达到可以接受连接的一致性恢复状态。
解决方法:请等待数据库完成启动过程。待数据库恢复正常服务后,重新启动DTS任务。
报错示例:
org.postgresql.util.PSQLException: FATAL: the database system is not yet accepting connections Detail: Consistent recovery state has not been yet reached.DTS-RETRY-ERR-0694:the database system is starting up
可能原因:DTS尝试连接的PostgreSQL数据库正在启动过程中,暂时无法接受连接。
解决方法:请等待数据库完成启动。待数据库恢复正常服务后,重新启动DTS任务。
报错示例:
org.postgresql.util.PSQLException: FATAL: the database system is starting upDTS-RETRY-ERR-0695:The index entry of length (.*)? bytes for the index (.*)? exceeds the maximum length of (.*)? bytes
可能原因:待插入或更新的行中,一个或多个列的值组合起来,使得索引条目的总长度超过了SQL Server允许的最大索引键长度(例如,对于非聚集索引是xxx字节)。
解决方法:请检查源端数据,看是否存在过长的值。您需要缩短相关字段的数据长度,或者重新设计目标表的索引,避免将过长的列包含在索引中。
报错示例:
com.microsoft.sqlserver.jdbc.SQLServerException: Operation failed. The index entry of length xxx bytes for the index 'xxx' exceeds the maximum length of xxx bytes.DTS-RETRY-ERR-0696:The INSERT permission was denied on the object (.*)?
可能原因:DTS用于连接目标SQL Server数据库的账号,没有在指定的表上执行INSERT操作的权限。
解决方法:请为DTS使用的目标库账号授予对相关表的INSERT权限。
报错示例:
java.sql.BatchUpdateException: The INSERT permission was denied on the object 'xxx', database 'xxx', schema 'xxx'.DTS-RETRY-ERR-0697:The SELECT would examine more than (.*)? rows
可能原因:DTS在PolarDB列存索引只读节点上执行的查询,预计扫描的行数超过了RDS_AP_THREHOLD参数设定的阈值,被数据库拒绝执行以防止性能问题。
解决方法:请调大目标PolarDB数据库的RDS_AP_THREHOLD参数值,或将会话设置为SET SQL_AP_SELECTS=ON。调整后,重新启动DTS任务。
报错示例:
DTS-30020: execute sql:/* DTS-full-d1gc8s3615zxxx */SELECT xxx FROM xxx --> (null, null) record range id = 271169604478094xxxx, fetchSize = -214748xxxx cause: SQLException: The SELECT would examine more than RDS_AP_THREHOLD rows; SET SQL_AP_SELECTS=ON or redirect to AP node.DTS-RETRY-ERR-0698:The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs
可能原因:DTS使用GTID模式进行复制,但它需要的起始GTID位点所在的Binlog文件已在源MySQL服务器上被清理(purged)。
解决方法:请立即调大源MySQL的Binlog保留周期。
报错示例:
DTS-100047: retry 0 times, 1000 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: mysql-reader: DTS-52200: process error failed cause: IOException: java.sql.SQLException: The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires. Replicate the missing transactions from elsewhere, or provision a new slave from backup. Consider increasing the master's binary log expiration period. The GTID set sent by the slave is '487e509d-316f-xxx-xxx-00163e257207:1-xxxx', and the missing transactions are '487e509d-316f-xxx-xxx-00163e257207:125941-xxx'. SQLException: The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires. Replicate the missing transactions from elsewhere, or provision a new slave from backup. Consider increasing the master's binary log expiration period. The GTID set sent by the slave is '487e509d-xxxx-xxx-xxx-00163e257207:1-125xxxx', and the missing transactions are '487e509d-xxx-xxx-xxxx-00163e257207:125941-12xxx'. CriticalAnyAllException: mysql-reader: DTS-52200: process error failed IOException: java.sql.SQLException: The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires. Replicate the missing transactions from elsewhere, or provision a new slave from backup. Consider increasing the master's binary log expiration period. The GTID set sent by the slave is '487e509d-xxx-xxx-xxx-00163e257207:1-125xxx', and the missing transactions are '487e509d-xxx-xxx-xxx-00163e257207:125941-12xxx'. SQLException: The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires. Replicate theDTS-RETRY-ERR-0699:The specified download session id does not exist.
可能原因:DTS用于从MaxCompute下载数据的会话(Session)ID已过期或失效。
解决方法:这是一个瞬时会话问题。请重新启动DTS任务,DTS会自动创建新的下载会话。
报错示例:
RequestId=20250626174326a181c40b04xxxx, ErrorCode=NoSuchDownload, ErrorMessage=The specified download session id does not exist.DTS-RETRY-ERR-0700:The specified table has been modified since the download initiated.
可能原因:在DTS从MaxCompute载入数据的过程中,目标表的结构被修改(例如执行了ALTER TABLE),导致下载会话失效。
解决方法:请不要在DTS全量同步期间对MaxCompute的目标表执行DDL操作。
报错示例:
DTS-31009: In process of processing data (recordRange: 788966514532706xxx) failed cause: CriticalAnyAllException: odps: DTS-11014: odps getRecordReader error,project:xxx table:xxx FatalAnyAllException: common: DTS-100047: retry 2535 times, 43206 seconds, which exceed the supposed 43200 seconds TunnelException: RequestId=202507290629070934c40b02fxxx, ErrorCode=TableModified, ErrorMessage=The specified table has been modified since the download initiated. Try initiate another download.