DTS-RETRY-ERR (0601–0700)

DTS-RETRY-ERR-0601: could not write to file

Possible cause: The source PostgreSQL database server has insufficient disk space. When DTS performs logical replication, the source database attempts to create a snapshot file. This operation fails because the device has no space left.

Solution: Free up disk space on the source PostgreSQL server to ensure sufficient storage for the logical replication operation. After you free up space, restart the DTS task.

Error example:

DTS-52111: Increment Context Is Not Running..: cause: CriticalAnyAllException: postgresql-reader: DTS-52510: Fetch postgresql logical log failed FatalAnyAllException: common: DTS-100047: retry 0 times, 5029990526 seconds, which exceed the supposed 43200 seconds IOException: org.postgresql.util.PSQLException: ERROR: could not write to file "pg_logical/snapshots/401-xxx.snap.xxx.tmp": No space left on device PSQLException: ERROR: could not write to file "pg_logical/snapshots/401-xxx.snap.xxx.tmp": No space left on device

DTS-RETRY-ERR-0602: CROSSSLOT Keys

Possible cause: DTS is attempting to execute a command that involves multiple keys in the destination Redis cluster, but these keys are hashed to different slots. This error usually occurs when the source and destination databases have inconsistent topologies. For example, the source is a Standard Edition instance and the destination is a Cluster Edition instance.

Solution: Ensure that the source and destination databases have the same topology.

Error example:

redis.clients.jedis.exceptions.JedisDataException: CROSSSLOT Keys in request don't hash to the same slot

DTS-RETRY-ERR-0603: Database (.*)? cannot be opened due to inaccessible files or insufficient memory or disk space. See the SQL Server errorlog for details.

Possible cause: The destination SQL Server database cannot be opened because of underlying resource issues. The specific cause might be inaccessible data files, insufficient server memory, or a full disk.

Solution: Log on to the destination database server and check the SQL Server error log for the detailed cause of the failure. After you resolve the file permission, memory, or disk space issues based on the log, restart the DTS task.

Error example:

DTS-100047: retry 112 times, 1208 seconds, which exceed the supposed 1200 seconds cause: SQLServerException: Database 'xxx' cannot be opened due to inaccessible files or insufficient memory or disk space. See the SQL Server errorlog for details.

DTS-RETRY-ERR-0604: Database (.*)? is in transition. Try the statement later.

Possible cause: The destination SQL Server database is in a transitional state, such as starting up, recovering, or failing over. It cannot temporarily accept new connections or operations.

Solution: Wait for the destination database to complete its current operation and return to a running state. After you confirm that the database is available, restart the DTS task.

Error example:

com.microsoft.sqlserver.jdbc.SQLServerException: Database 'xxx' is in transition. Try the statement later.

DTS-RETRY-ERR-0605: Database connection failed when reading from copy EOFException: null

Possible cause: The logical replication connection to the source PostgreSQL database was interrupted because the wal_sender_timeout value was exceeded. If DTS does not interact with the source database for a long time, for example, when it filters many transactions that do not require synchronization, the source database may actively close the connection because it did not receive a heartbeat.

Solution: Connect to the source database and set the wal_sender_timeout parameter to 0 to disable the timeout mechanism.

1. Execute the following SQL command: ALTER SYSTEM SET wal_sender_timeout = '0';

2. Reload the configuration: SELECT pg_reload_conf();

3. After you modify the configuration, restart the DTS task.

Error example:

xDTS-52111: Increment Context Is Not Running..: cause: CriticalAnyAllException: postgresql-reader: DTS-52510: Fetch postgresql logical log failed FatalAnyAllException: common: DTS-100047: retry 0 times, 1016336561 seconds, which exceed the supposed 43200 seconds IOException: org.postgresql.util.PSQLException: Database connection failed when reading from copy PSQLException: Database connection failed when reading from copy EOFException: null

DTS-RETRY-ERR-0606: date format not recognized

Possible cause: The source Oracle database contains data in a date format that DTS cannot recognize or parse. This causes the full data extraction to fail.

Solution: Based on the ROWID or data content in the error message, locate the abnormal data row in the source database and correct it to a standard date format. After you fix the data, restart the DTS task.

Error example:

 DTS-30020: execute sql:/* DTS-full-dl0y6l2a214xxxx */SELECT xxx FROM "xxx"."xxx" --> (null, null) record range id = 48754756125982xxxx, fetchSize = 1024 cause: SQLDataException: ORA-01821: date format not recognized

DTS-RETRY-ERR-0607: Disperse target record field (.*)? failed

Possible cause: While processing data, DTS cannot convert the value of a field from the source database to the type of the corresponding field in the destination database. This error usually occurs because the field types in the source and destination databases are incompatible, and DTS does not support this type of conversion.

Solution: Check the data types of the field in the source and destination databases. You can modify the destination table schema to make it compatible or exclude the field from synchronization in the DTS task. After you make the adjustments, restart the DTS task.

Error example:

DTS-71001: Disperse target record field collectInfo BYTES nullable as collectInfo failed for record databaseName: xxx tableName:xxx {offset [35699] operationType [UPDATE],checkpoint [0@35699@160288815@12568@175275215xxx]} cause: ConcurrentModificationException: null

DTS-RETRY-ERR-0608: disperser is null

Possible cause: When performing data type conversion, DTS cannot find a converter (disperser) to process a specific field. This error is usually caused by incompatible field types between the source and destination tables.

Solution: Check and compare the table schemas of the incompatible fields in the source and destination databases. You can modify the destination table field type to make it compatible with the source or exclude the field from synchronization in the DTS task. After you make adjustments, restart the DTS task.

Error example:

DTS-61001: disperser is null, can not find disperser for field billing_date

DTS-RETRY-ERR-0609: document size is larger than maximum

Possible cause: The data to be synchronized contains a single document that exceeds the 16 MB maximum size limit allowed by MongoDB. This prevents the document from being written to the destination database.

Solution: Handle the oversized document in the source database based on your business needs. For example, you can split or delete it.

Error example:

DTS-100047: retry 0 times, 0 seconds, which exceed the supposed 922337203685477xxxx seconds cause: SQLException: DTS full single insert oversize, throw exception and retry, caused by:Payload document size is larger than maximum of xxxx.

DTS-RETRY-ERR-0610: does not contain shard key

Possible cause: The destination MongoDB collection is a sharded collection, but a record to be synchronized is missing the required sharding key field. This prevents the data from being written.

Solution: Check the source data to ensure that all records to be synchronized include the sharding key field required by the destination table.

Example error:

DTS-100047: retry 0 times, 0 seconds, which exceed the supposed 922337203685477xxxx seconds cause: SQLException: com.mongodb.MongoWriteException: Write operation error on server 10.xxx.xxx.xxx:xxx. Write error: WriteError{code=61, message='document { xxx } does not contain shard key for pattern { brandId: 1.0, createTime: 1.0 }', details={}}. MongoWriteException: Write operation error on server 10.xxx.xxx.xxx:xxx. Write error: WriteError{code=61, message='document { xxx } does not contain shard key for pattern { brandId: 1.0, createTime: 1.0 }', details={}}.

DTS-RETRY-ERR-0611: Don't allow to dump binary log on standby instance

Possible cause: DTS is configured to connect to a standby or slave instance of the source MySQL database. The configuration of this standby instance prohibits external connections such as DTS from extracting its binary logs.

Solution: Modify the source database connection information in the DTS task to point to the primary or master instance of the source database.

Error example:

DTS-100047: retry 0 times, 1001 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: mysql-reader: DTS-52200: process error failed cause: IOException: java.sql.SQLException: Don't allow to dump binary log on standby instance SQLException: Don't allow to dump binary log on standby instance CriticalAnyAllException: mysql-reader: DTS-52200: process error failed IOException: java.sql.SQLException: Don't allow to dump binary log on standby instance SQLException: Don't allow to dump binary log on standby instance

DTS-RETRY-ERR-0612: DTS-50019

Error description: The DTS writer module cannot read data from the DTS cache module at the specified point in time.

Possible causes and solutions:

• Possible cause 1: The task was paused for too long. After the task is restarted, the data at the corresponding offset (for example, 2021-05-20 21:17:51) no longer exists in the DTS cache module.

  Note

  The error message displays a UNIX timestamp. You can use a search engine to find a UNIX timestamp converter and convert the timestamp to UTC.

  Solution 1:

  • Method 1: Check the logs used for incremental data parsing in the source database, such as binary logs or redo logs, to determine if their time range includes this point in time.

  • Method 2: Recreate the task. Do not pause the task for a long time. A task that is paused for more than six hours cannot be started successfully.

• Possible cause 2: The task failed and was not fixed for a long time. After the issue is fixed, the data at the corresponding offset no longer exists in the DTS cache module.

  Solution 2: Recreate the task. If a task fails, fix the issue and resume the task as soon as possible to prevent it from becoming unrecoverable.

• Possible cause 3: During full migration or full synchronization, DTS locally stores the last 24 hours or 50 GB of data logs from the source database for incremental data parsing. If the full migration or synchronization takes a long time, DTS may have already purged the cached data logs. This results in incomplete logs for incremental parsing.

  Solution 3: Recreate the task. Before you create the task, estimate the data volume. If the data volume is large, you can split it into multiple migration or synchronization tasks.

Error example:

DTS-31009: In process of processing data (recordRange: 15907394498750xxx) failed cause: FatalAnyAllException: common: 
DTS-100047: retry 0 times, 3 seconds, which exceed the supposed 43200 seconds CriticalAnyAllException: capture-as400: 
DTS-50019: Lost journal {timestampMs: 1752047995xxx} log {journalName: NBJRN, journalLibraryName: COASSYSTF1}

DTS-RETRY-ERR-0613: DTS-52061

Error description: The DTS writer module cannot read data from the DTS cache module at the specified point in time.

Possible causes and solutions:

• Possible cause 1: The task was paused for too long. After the task is restarted, the data at the corresponding offset (for example, 2021-05-20 21:17:51) no longer exists in the DTS cache module.

  Note

  The error message displays a UNIX timestamp. You can use a search engine to find a UNIX timestamp converter and convert the timestamp to UTC.

  Solution 1:

  • Method 1: Check the logs used for incremental data parsing in the source database, such as binary logs or redo logs, to determine if their time range includes this point in time.

  • Method 2: Recreate the task. Do not pause the task for a long time. A task that is paused for more than six hours cannot be started successfully.

• Possible cause 2: The task failed and was not fixed for a long time. After the issue is fixed, the data at the corresponding offset no longer exists in the DTS cache module.

  Solution 2: Recreate the task. If a task fails, fix the issue and resume the task as soon as possible to prevent it from becoming unrecoverable.

• Possible cause 3: During full migration or full synchronization, DTS locally stores the last 24 hours or 50 GB of data logs from the source database for incremental data parsing. If the full migration or synchronization takes a long time, DTS may have already purged the cached data logs. This results in incomplete logs for incremental parsing.

  Solution 3: Recreate the task. Before you create the task, estimate the data volume. If the data volume is large, you can split it into multiple migration or synchronization tasks.

Error example:

DTS-31009: In process of processing data (recordRange: 15907394498750xxxx) failed cause: FatalAnyAllException: common: 
DTS-100047: retry 0 times, 161765 seconds, which exceed the supposed 43200 seconds RuntimeException: AS400DB2Source:journalFetcher-1: failed cause retrieve capture-as400: 
DTS-52061: {ReceiverName: NBRCV9xxxx, ReceiverLibrary: COASSYSTF1}'s journal receiver is deleted, That journal log isn't continuous

DTS-RETRY-ERR-0615: DTS-52722

Possible cause: The database account configured for the DTS task lacks the permission to read the required tables.

Solution: Grant read permissions on the synchronization objects to the account used by DTS. After you grant the permissions, restart the DTS task.

Error example:

DTS-31009: In process of processing data (recordRange: 456002531641996xxxx) failed cause: FatalAnyAllException: common: 
DTS-100047: retry 0 times, 81170 seconds, which exceed the supposed 43200 seconds RuntimeException: AS400DB2Source:JournalEntryParser: parser entry failed, cause capture-as400: DTS-52722: Read table xxx.xxx info error, case by [SQL0551] Not authorized to object xxx in xxx type *FILE. Cause . . . . . : An operation was attempted on object xxx in xxx type *FILE. This operation cannot be performed without the required authority. Recovery . . . : Obtain the required authority from either the security officer, the object owner, or a user that is authorized to the QIBM_DB_SECADM function. If you are not authorized to a logical file, obtain the authority to the based-on files of the logical file. Try the operation again.

DTS-RETRY-ERR-0616: ERR DUMP payload version or checksum are wrong

Possible cause: The source and destination Redis versions are incompatible. This prevents DTS from migrating binary data directly using the RESTORE command.

Solution: This issue cannot be resolved by restarting the task. You can take one of the following measures:

• Adjust the version of the destination Redis instance to be compatible with or the same as the source version.

• Reconfigure the DTS task and select the full synchronization and incremental synchronization mode.

Error example:

DTS-100047: retry 614 times, 43208 seconds, which exceed the supposed 43200 seconds cause: JedisDataException: ERR DUMP payload version or checksum are wrong

DTS-RETRY-ERR-0617: ERR invalid DB index

Possible cause: DTS attempted to select an invalid DB in the destination Redis.

Solution: In the DTS task configuration, use the schema mapping feature to map the source DB to an allowed DB in the destination Redis.

Error example:

DTS-100047: retry 60 times, 601 seconds, which exceed the supposed 43200 seconds cause: RedisCannotRetryException: redis: DTS-10009: Exception in stage: [begin], errMsg: ERR invalid DB index JedisDataException: ERR invalid DB index

DTS-RETRY-ERR-0618: ERR-CODE: [TDDL-10000][ERR_X_PROTOCOL_BAD_PACKET]

Possible cause: The size of the data packet sent by DTS exceeds the packet limit of the destination database.

Solution: Log on to the destination database and increase the value of the packet parameter.

Error example:

java.sql.BatchUpdateException: [19d7e170d4402000][26.xxx.xxx.xxx:xxx][pro_scrm]ERR-CODE: [TDDL-10000][ERR_X_PROTOCOL_BAD_PACKET] X-NIO-Client /26.xxx.xxx.xxx:xxx to /10.xxx.xxx.xxx:xxx sent packet length (81341097) exceeds the allowed maximum (67108864).

DTS-RETRY-ERR-0619: ERR-CODE: [TDDL-5311][ERR_GLOBAL_SECONDARY_INDEX_MODIFY_GSI_TABLE_DIRECTLY]

Possible cause: DTS attempted to directly modify a global secondary index (GSI) table, but the destination database does not allow this operation.

Solution: This issue cannot be resolved by restarting the task. A global secondary index must be updated through its primary table. Check your synchronization object configuration to ensure that you are synchronizing the primary table, not the GSI table.

Error example:

java.sql.BatchUpdateException: [1a0e2bffcfb3xxxx][172.xxx.xxx.xxx:xxx][sto_sitems]ERR-CODE: [TDDL-5311][ERR_GLOBAL_SECONDARY_INDEX_MODIFY_GSI_TABLE_DIRECTLY] Does not support modify global secondary index table 'xxxx'

DTS-RETRY-ERR-0622: Error on Send.This server does not host this topic-partition

Possible cause: The data partition metadata information within DTS is out of date. This error is usually caused by an internal component, such as DStore, restarting or migrating.

Solution: This is an internal DTS error. You can try to restart the DTS task. This forces a refresh of the metadata information.

Error example:

DTS-100047: retry 0 times, 0 seconds, which exceed the supposed 43200 seconds cause: RuntimeException: Error on Send.This server does not host this topic-partition. UnknownTopicOrPartitionException: This server does not host this topic-partition.

DTS-RETRY-ERR-0623: estimated need client-output-buffer

Possible cause: The client output buffer allocated by the source Redis for DTS is insufficient. This causes the connection to be actively closed to protect the source database.

Solution: Based on the recommended value for estimated need client-output-buffer in the error message, increase the value of this configuration in the source Redis.

Error example:

DTS-10015: Slave connection closed by source redis, Reach 90.00% hard-client-output-buffer-limit, current size: 1453056xxx, hard limit: 1610612xxx.RDB parse progress: 28.97%, already cost time: 209, estimated finish cost time: 721, remaining time: 512, current client-output-buffer: 1453056xxx, estimated need client-output-buffer: 478x.xxMB.

DTS-RETRY-ERR-0624: Expiring (.*)? record(s) (.*)?

Possible cause: A message that DTS sent to the destination Kafka cluster did not receive a confirmation for a long time, which caused the message to time out. This error may be caused by an abnormal Kafka broker or the DTS IP address whitelist not being configured on all brokers.

Solution:

• Check the health status of all brokers in the destination Kafka cluster.

• Confirm that the DTS IP address whitelist has been added to all broker nodes in the Kafka cluster.

• Check for high latency or packet loss in the network connection between DTS and the Kafka cluster.

Error example:

DTS-100047: retry 107 times, 43338 seconds, which exceed the supposed 43200 seconds cause: ExecutionException: com.xxx.kafka_thin_client1d0d0.common.errors.TimeoutException: Expiring xxx record(s) for dls_hr_ram_t_xxx_tp-22: 391390 ms has passed since last append TimeoutException: Expiring xxx record(s) for dls_hr_ram_t_xxx_tp-22: 391390 ms has passed since last append

DTS-RETRY-ERR-0625: Failed to calculate delete bitmap

Possible cause: When DTS writes data to the destination SelectDB using StreamLoad, a timeout or error occurs within SelectDB while it calculates the delete bitmap. This indicates that the destination SelectDB instance may have health issues or internal lock contention.

Solution: Check the running status, logs, and load of the destination SelectDB instance. After you confirm that the database is back to normal, restart the DTS task.

Error example:

java.io.IOException: Failed to stream load data to SelectDB.Key status is fail. Load result: {"Status":"Fail","Comment":"","BeginTxnTimeMs":28,"Message":"[DELETE_BITMAP_LOCK_ERROR]TStatus: errCode = 301, detailMessage = Failed to calculate delete bitmap. Timeout. Unfinished mark: 392787086xxx=23524707046653xxx","NumberUnselectedRows":0,"CommitAndPublishTimeMs":0,"Label":"DTS-SelectDB-Sink2ea489f7-1684-xxxx-xxxx-9c6adc2e36d0","LoadBytes":2285539,"StreamLoadPutTimeMs":17,"NumberTotalRows":10240,"WriteDataTimeMs":70024,"ReceiveDataTimeMs":118,"TxnId":2352470704665xxxx,"LoadTimeMs":220693,"TwoPhaseCommit":"false","ReadDataTimeMs":1,"NumberLoadedRows":10240,"NumberFilteredRows":0}

DTS-RETRY-ERR-0626: Failed to create upload session with tunnel endpoint

Possible cause: DTS cannot create a data upload channel to the destination MaxCompute table. This error is usually caused by network issues or the destination service being unavailable.

Solution: Check the network connection between DTS and the MaxCompute service. After you confirm that the network is clear and the MaxCompute service is normal, restart the DTS task.

Error example:

DTS-70004: execute upload failed, may try it again cause: TransactionReplicateException: transaction-replicate: DTS-70003: 5 execute transaction has excess max transaction retry time [100] cause:odps: DTS-11014: Failed to get the upload session for Prd_IOT_DataWorks.ods_t_ord_affiliated_deduction_log caused by ErrorCode=Local Error, ErrorMessage=Failed to create upload session with tunnel endpoint xxx ErrorCode=Local Error, ErrorMessage=Failed to create upload session with tunnel endpoint xxx CriticalAnyAllException: odps: DTS-11014: Failed to get the upload session for Prd_IOT_DataWorks.ods_t_ord_affiliated_deduction_log caused by ErrorCode=Local Error, ErrorMessage=Failed to create upload session with tunnel endpoint xxx ErrorCode=Local Error, ErrorMessage=Failed to create upload session with tunnel endpoint xxx...

DTS-RETRY-ERR-0627: failed to parse field

Possible cause: The data of a field in the source database does not match the mapping type defined in the destination Elasticsearch (ES). This causes ES to fail to parse the data and reject the write operation. For example, this error occurs if you write a string to a field that is defined as a long type.

Solution: Check the source data and the destination ES mapping to ensure data type compatibility. You can choose to:

• Correct the source data.

• Modify the destination ES index mapping and then re-import the data.

Error example:

DTS-100047: retry 15 times, 43 seconds, which exceed the supposed 922337203685477xxxx seconds cause: IOException: es: DTS-10035: [1:412] failed to parse field [xxx.xxx] of type [long] in document with id 'xxx'. Preview of field's value: 'xxx', For input string: "xxx" CriticalAnyAllException: es: DTS-10035: [1:412] failed to parse field [xxx.xxx] of type [long] in document with id 'xxx'. Preview of field's value: 'xxx', For input string: "xxx"

DTS-RETRY-ERR-0628: fetch target schema from target table (.*)? failed

Possible cause: DTS cannot obtain the table schema information of the destination database. This may be because the destination table does not exist, or the database account used by DTS does not have sufficient permissions to view the table schema.

Solution:

1. Confirm that the table exists in the destination database.

2. Check and ensure that the DTS synchronization account has DESCRIBE or equivalent permissions on the table.

3. After you confirm, restart the DTS task.

Error example:

DTS-100047: retry 0 times, 0 seconds, which exceed the supposed 43200 seconds cause: AnyCoreException: framework: DTS-31008: fetch target schema from target table xxx.xxx failed

DTS-RETRY-ERR-0629: file (.*)? cannot be read at this time ORA-01110: data file (.*)?

Possible cause: When reading from the source Oracle database, DTS cannot read a data file (.dbf). This is usually because the DTS account lacks access permissions to the tablespace where the file is located, or the database is recovering and the file is offline.

Solution: Check the status of the source Oracle database to ensure that it is open and running normally. Also, confirm that the account used by DTS has permission to access the required tablespaces. After you confirm, restart the DTS task.

Error example:

DTS-30020: execute sql:/* DTS-full-xa7m7590u2xxxx */SELECT xxx FROM xxx.xxx --> (null, null) record range id = 25182463197225xxxx, fetchSize = 1024 cause: SQLException: ORA-00376: file xxx cannot be read at this time ORA-01110: data file 13: 'xxx'

DTS-RETRY-ERR-0630: get table read lock timeout

Possible cause: When DTS writes data to the destination SelectDB using StreamLoad, it fails because it cannot obtain a read lock on the table within the timeout period. This usually indicates that there are long-running DDL operations or other blocking tasks on the destination table.

Solution: Check the destination SelectDB instance for long-running operations such as ALTER TABLE. After the blocking tasks are complete, restart the DTS task.

Error example:

java.io.IOException: Failed to stream load data to SelectDB.Key status is fail. Load result: {"Status":"Fail","Comment":"","BeginTxnTimeMs":11,"Message":"[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = get table read lock timeout, database=xxx,table=xxx","NumberUnselectedRows":0,"CommitAndPublishTimeMs":0,"Label":"DTS-SelectDB-Sink1b0aec5d-xxxx-xxxx-b9f1-9a1e62dde118","LoadBytes":0,"StreamLoadPutTimeMs":60000,"NumberTotalRows":0,"WriteDataTimeMs":0,"ReceiveDataTimeMs":0,"TxnId":2581391489973xxxx,"LoadTimeMs":0,"TwoPhaseCommit":"false","ReadDataTimeMs":0,"NumberLoadedRows":0,"NumberFilteredRows":0}

DTS-RETRY-ERR-0631: given ts is smaller than oldest message's ts, some binlogs may lose

Possible cause: DTS attempts to pull data from a specified start timestamp in the source Kafka (used for TiDB Binlog), but the earliest message in Kafka is later than that timestamp. This means the required incremental logs have been purged.

Solution: Check the retention policy for TiDB Binlog data in the source Kafka and ensure that the retention period is long enough.

Error example:

DTS-31009: In process of processing data (recordRange: 776029846907125xxxx) failed cause: FatalAnyAllException: common: DTS-100047: retry 0 times, 0 seconds, which exceed the supposed 43200 seconds RuntimeException: seek timestamp for topic [xxx] with timestamp [1747724390xxx] failed RuntimeException: given ts is smaller than oldest message's ts, some binlogs may lose, given ts: 1747724390xxx, oldest ts: 1747726918xxx

DTS-RETRY-ERR-0632: got sequence entry (.*)? for toast chunk (.*)?

Possible cause: An internal logic error occurred in the source PostgreSQL database when processing large object (TOAST) data. This caused DTS to receive out-of-order or unexpected log entries. This is a database-specific error.

Solution: This is a database internal error. Check the health status and error logs of the source PostgreSQL database.

Error example:

DTS-52111: Increment Context Is Not Running..: cause: CriticalAnyAllException: postgresql-reader: DTS-52510: Fetch postgresql logical log failed FatalAnyAllException: common: DTS-100047: retry 58 times, 600947 seconds, which exceed the supposed 600 seconds IOException: org.postgresql.util.PSQLException: ERROR: got sequence entry xxx for toast chunk xxx instead of seq 0 PSQLException: ERROR: got sequence entry xxx for toast chunk xxx instead of seq 0

DTS-RETRY-ERR-0633: Got timeout reading communication packets

Possible cause: A read timeout occurred on the network connection between DTS and the source MySQL database. This is usually caused by high network latency, instability, or a heavily loaded database server that responds slowly.

Solution: Troubleshoot the network link quality between DTS and the source database. If the network is normal, check the load on the database server. Restarting the DTS task can resolve transient network issues.

Error example:

com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Got timeout reading communication packets

DTS-RETRY-ERR-0635: Host (.*)? is not allowed to connect to this (.*)? server

Possible cause: The IP address whitelist of the destination database does not include the IP address of the DTS server, which causes the database to reject the connection.

Solution: Add the IP address of the DTS server to the IP address whitelist of the destination database instance.

Error example:

java.sql.SQLException: null, message from server: "Host '100.xxx.xxx.xxx' is not allowed to connect to this xxx server"

DTS-RETRY-ERR-0636: ignore with global index not supported

Possible cause: The table in the destination PolarDB-X 2.0 database has a global index, and the INSERT IGNORE statement used by DTS is not supported in this case.

Solution: This is a database limitation. If your business allows, you can consider deleting the global index from the destination table or changing the partitioned table to a non-partitioned table. After you modify the database schema, restart the DTS task.

Error example:

DTS-30011: currentRunningSQL: /* DTS-full-ch7c7pjz16mxxxx */insert ignore into `xxx`.`xxx`() VALUES (),currentRow:,
reason: ignore with global index not supported cause: BatchUpdateException: ignore with global index not supported SQLException: ignore with global index not supported

DTS-RETRY-ERR-0638: immediate shutdown in progress - no operations are permitted

Possible cause: The Oracle database that DTS is trying to connect to is performing an immediate shutdown, during which no new operations are accepted.

Solution: Wait for the Oracle database to complete the shutdown and restart. After the database resumes normal service, restart the DTS task.

Error example:

java.sql.SQLRecoverableException: ORA-01089: immediate shutdown in progress - no operations are permitted

DTS-RETRY-ERR-0639: insufficient privileges

Possible cause: The account that DTS uses to connect to the source Oracle database lacks necessary permissions, such as the permission to access system data dictionary views like sys.USER$.

Solution: For more information, see Limits and preparations for Oracle databases. Grant all required permissions to the dedicated DTS account for the source Oracle database.

Error example:

DTS-52110: OracleRecordExtractor Init Error: oracle-reader: DTS-52050: Mirror Meta error, cause: CriticalAnyAllException: oracle-reader: DTS-52050: Mirror Meta error, CriticalAnyAllException: oracle-reader: DTS-52050: Mirror meta failed CriticalAnyAllException: oracle-reader: DTS--0002: execute sql SELECT USER#,NAME,TYPE# FROM sys.USER$ failed, case by common: DTS-100047: retry 4307 times, 43200410 seconds, which exceed the supposed 43200 seconds FatalAnyAllException: common: DTS-100047: retry 4307 times, 43200410 seconds, which exceed the supposed 43200 seconds SQLSyntaxErrorException: ORA-01031: insufficient privileges

DTS-RETRY-ERR-0640: Insufficient result space to convert uniqueidentifier value to char.

Possible cause: When DTS reads data from SQL Server, the internal buffer space for storing the result of converting a uniqueidentifier type is insufficient. This may be an internal issue caused by DTS not synchronizing metadata in time after a DDL change.

Solution: Confirm whether a DDL change has been made and try to restart the DTS task.

Error example:

DTS-31009: read source data error cause: SQLServerException: Insufficient result space to convert uniqueidentifier value to char.

DTS-RETRY-ERR-0641: invalid input syntax for type money

Possible cause: The source and destination databases have different units for the money type. This causes the currency format string written by DTS (such as ￥118,971.00) to be rejected by the destination database.

Solution: Check and unify the currency format settings on both the source and destination databases. After you fix the settings, restart the DTS task.

Error example:

DTS-100047: retry 180 times, 1802 seconds, which exceed the supposed 1800 seconds cause: CriticalAnyAllException: framework: DTS-30011: currentRunningSQL: /* DTS-full-jw3c85ox14cxxxx */insert into "xxx"."xxx"() values () on conflict do nothing,currentRow:, 
reason: Batch entry 0 /* DTS-full-jw3c85ox14cxxxx */insert into "xxx"."xxx"() values () on conflict do nothing was aborted: ERROR: invalid input syntax for type money: "￥118,971.00" Call getNextException to see other errors in the batch. BatchUpdateException: Batch entry 0 /* DTS-full-jw3c85ox14cxxx */insert into "xxx"."xxx"() values () on conflict do nothing was aborted: ERROR: invalid input syntax for type money: "￥118,971.00" Call getNextException to see other errors in the batch. PSQLException: ERROR: invalid input syntax for type money: "￥118,971.00"

DTS-RETRY-ERR-0642: invalid username/password; logon denied

Possible cause: The username or password used by DTS to connect to the Oracle database is incorrect.

Solution: Check and update the Oracle database account and password in the DTS task configuration to ensure that they are correct and valid. After you confirm, restart the DTS task.

Error example:

java.sql.SQLException: ORA-01017: invalid username/password; logon denied

DTS-RETRY-ERR-0643: (.*)? is an identity column defined as GENERATED ALWAYS

Possible cause: A column in the destination PostgreSQL table is defined as a GENERATED ALWAYS identity column. This type of column does not allow explicit value insertion. However, DTS is attempting to write a value from the source database to this column.

Solution: This issue cannot be resolved by restarting the task. You need to modify the destination table schema. For example, you can change the column to GENERATED BY DEFAULT or exclude this column from synchronization in the DTS column mapping.

Error example:

DTS-30011: currentRunningSQL: /* DTS-full-nezi7395167xxxx */insert into "xxx"."xxx"() values () on conflict do nothing,currentRow:xxx, reason: Batch entry 0 /* DTS-full-nezi7395167xxx */insert into "xxx"."xxx"() values (xxx) on conflict do nothing was aborted: ERROR: cannot insert a non-DEFAULT value into column "id" Detail: Column "id" is an identity column defined as GENERATED ALWAYS. Hint: Use OVERRIDING SYSTEM VALUE to override. Call getNextException to see ...

DTS-RETRY-ERR-0644: (.*)? is INTEGER, but it's defined as STRING in table meta

Possible cause: The type of a primary key column in the data to be synchronized (for example, INTEGER) does not match the metadata type defined in the destination Tablestore (for example, STRING).

Solution: Check and unify the source data type with the destination Tablestore table schema definition. After you fix the issue, restart the DTS task.

Error example:

DTS-100047: retry 0 times, 1 seconds, which exceed the supposed 43200 seconds cause: ClientException: The type of primary key column 'xxx' is INTEGER, but it's defined as STRING in table meta.

DTS-RETRY-ERR-0645: java.net.UnknownHostException

Possible cause: The DTS server cannot resolve the domain name configured for the source database through DNS.

Solution: Check whether the connection address of the source database is correct. Also, confirm that the domain name can be resolved normally in the network environment where the DTS server is located.

Error example:

java.sql.SQLException: java.net.UnknownHostException: database-instance-1.c96cqe6wxxxx.ap-southeast-1.rds.xxx.com

DTS-RETRY-ERR-0646: java.sql.SQLException: Binary log is not open

Possible cause: The source database did not enable binary logging as expected.

Solution: Check and ensure that the binary logging feature is correctly enabled on the source database. After you confirm, restart the DTS task.

Error example:

DTS-100047: retry 0 times, 1101 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: mysql-reader: DTS-52200: process error failed cause: IOException: java.sql.SQLException: Binary log is not open SQLException: Binary log is not open CriticalAnyAllException: mysql-reader: DTS-52200: process error failed IOException: java.sql.SQLException: Binary log is not open SQLException: Binary log is not open

DTS-RETRY-ERR-0647: java.sql.SQLException: Not Support Unusing binary logging.

Possible cause: The source MySQL database has not enabled binary logging, which is required for DTS incremental synchronization.

Solution: Log on to the source MySQL database, check and enable binary logging, and then restart the DTS task.

Error example:

java.sql.SQLException: Not Support Unusing binary logging.

DTS-RETRY-ERR-0648: java.sql.SQLException: The requested gtid transaction no longer exists and may have been cleaned up

Possible cause: During incremental synchronization, the GTID offset required by DTS no longer exists in the source MySQL binary log files. This is usually because the binary log files have been automatically purged by the database.

Solution: Extend the binary log retention period of the source database. After you fix the issue, you need to reconfigure the DTS task to synchronize from the beginning.

Error example:

DTS-100047: retry 0 times, 1101 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: mysql-reader: DTS-52200: process error failed cause: IOException: java.sql.SQLException: The requested gtid transaction no longer exists and may have been cleaned up SQLException: The requested gtid transaction no longer exists and may have been cleaned up CriticalAnyAllException: mysql-reader: DTS-52200: process error failed IOException: java.sql.SQLException: The requested gtid transaction no longer exists and may have been cleaned up SQLException: The requested gtid transaction no longer exists and may have been cleaned up

DTS-RETRY-ERR-0649: java.sql.SQLNonTransientConnectionException: Got timeout reading communication packets

Possible cause: A network timeout occurred when DTS was reading the response from the source MySQL database. This may be caused by network latency or high database load.

Solution: Check the network condition between DTS and the source database, and investigate the performance load of the source database. Restarting the DTS task can resolve transient issues.

Error example:

java.sql.SQLNonTransientConnectionException: Got timeout reading communication packets

DTS-RETRY-ERR-0650: java.sql.SQLNonTransientConnectionException: Server shutdown in progress

Possible cause: The source MySQL database server is shutting down and is no longer accepting any operations.

Solution: Check the status of the source database. After it has completed its shutdown and restart process and resumed service, restart the DTS task.

Error example:

java.sql.SQLNonTransientConnectionException: Server shutdown in progress

DTS-RETRY-ERR-0651: LeaderNotAvailableException

Possible cause: The destination Kafka cluster is undergoing a partition migration, which makes the partition temporarily unavailable for writing. This is usually triggered by a cluster node restart or network fluctuations.

Solution: This issue is usually temporary. You can restart the DTS task. DTS will retry the connection and retrieve the latest partition leader information.

Error example:

java.util.concurrent.ExecutionException: com.xxx.kafka_thin_client1d0d0.common.errors.LeaderNotAvailableException: There is no leader for this topic-partition as we are in the middle of a leadership election.

DTS-RETRY-ERR-0652: listener does not currently know of SID given in connect descriptor

Possible cause: The listener of the source Oracle database does not recognize the system ID (SID) specified in the DTS connection request. This may be because the SID is configured incorrectly, the database instance is not running, or it is not registered with the listener.

Solution: Check the connection information of the source database to ensure that the SID is correct. Also, confirm that both the Oracle database and its listener service are running normally. After you confirm, restart the DTS task.

Error example:

java.sql.SQLException: Listener refused the connection with the following error: ORA-12505, TNS:listener does not currently know of SID given in connect descriptor

DTS-RETRY-ERR-0653: MySQLSyntaxErrorException: Result consisted of more than one row

Possible cause: When DTS executed an operation that was expected to affect only a single row in the destination MySQL database, the database returned multiple rows. This error usually indicates that the destination table contains duplicate data, which is most likely caused by a missing primary key or unique constraint.

Solution: Check the destination table, and locate and clean up the duplicate data based on the business primary key. Also, ensure that the destination table has a correct primary key or unique index constraint.

Error example:

DTS-100047: retry 3106 times, 43203 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: framework: DTS-30011: currentRunningSQL: /* DTS-full-y5jg7pd1138xxxx */insert ignore into `xxx`.`xxx`() VALUES (),currentRow:xxx, reason: Result consisted of more than one row BatchUpdateException: Result consisted of more than one row MySQLSyntaxErrorException: Result consisted of more than one row

DTS-RETRY-ERR-0654: new row for relation (.*)? violates check constraint

Possible cause: A record from the source does not satisfy a CHECK constraint defined on the destination PostgreSQL table.

Solution: Locate and correct the data in the source database that does not comply with the business rule. Alternatively, if the business logic allows, you can consider modifying or removing the CHECK constraint from the destination table. After you adjust the data or table schema, restart the DTS task.

Error example:

DTS-100047: retry 4247 times, 43207 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: framework: DTS-30011: currentRunningSQL: /* DTS-full-j87z7m2b16dxxxx */insert into "xxx"."xxx"() values () on conflict do nothing,currentRow:xxx, reason: Batch entry 32 /* DTS-full-j87z7m2b16dxxxx */insert into "xxx"."xxx"() values () on conflict do nothing was aborted: ERROR: new row for relation "xxx" violates check constraint "xxx_constraint" Detail: Failing row contains (). Call getNextException to see other errors in the batch. 
BatchUpdateException: Batch entry 32 /* DTS-full-j87z7m2b16dxxxx */insert into "xxx"."xxx"() values (xxx) on conflict do nothing was aborted: ERROR: new row for relation "xxx" violates check constraint "xxx_constraint" Detail: Failing row contains (). Call getNextException to see other errors in the batch. PSQLException: ERROR: new row for relation "xxx" violates check constraint "xxx_constraint" Detail: Failing row contains ().

DTS-RETRY-ERR-0655: No more data to read from socket

Possible cause: The network connection between DTS and the Oracle database was unexpectedly closed. This could be caused by an abnormal process on the database side, a network device such as a firewall cutting off the session due to an idle timeout, or a transient network interruption.

Solution: Check the network stability between DTS and the Oracle server, and confirm that no firewall is interrupting persistent connections. Also, check the Oracle alert log for server-side errors. Restarting the DTS task can usually resolve transient issues.

Error example:

DTS-30020: execute sql:/* DTS-full-etzf7t4x24xxxxx */SELECT /*+ index("xxx" SYS_C0044122) */ xxx FROM xxx WHERE xxx --> (null, [Lcom.alibaba.amp.any.record.value.Value;@20510xxx) record range id = 712551168722599xxxx, fetchSize = 1024 cause: SQLRecoverableException: No more data to read from socket

DTS-RETRY-ERR-0656: no privileges on tablespace (.*)?

Possible cause: The account connected to the destination Oracle database lacks the permission to create objects in the specified tablespace.

Solution: Grant permissions on the relevant tablespace to the destination database account used by DTS.

Error example:

DTS-100047: retry 4314 times, 43201 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: framework: DTS-30011: currentRunningSQL: /* DTS-full-r87i6i04j33xxxx */insert into "xxx"."xxx"() VALUES (),currentRow:xxx, reason: ORA-01950: no privileges on tablespace 'xxx' BatchUpdateException: ORA-01950: no privileges on tablespace 'xxx'

DTS-RETRY-ERR-0657: no privileges on tablespace (.*)?

Possible cause: The account connected to the destination database does not have permission to create objects in the specified tablespace.

Solution: Grant permissions on the relevant tablespace to the destination database account used by DTS.

Error example

DTS-30011: currentRunningSQL: /* DTS-full-q0yy62nq16fxxx */insert into "xxx"."xxx"() VALUES (),currentRow:null, reason: ORA-01950: no privileges on tablespace 'xxx' cause: BatchUpdateException: ORA-01950: no privileges on tablespace 'xxx'

DTS-RETRY-ERR-0658: No route to host (Host unreachable)

Possible cause: DTS cannot find a network route to the source database host, which causes the connection to fail. This is usually caused by incorrect network configuration, firewall policies, or a lack of routing between subnets.

Solution: Check the network configuration and connectivity between the DTS server and the source database server. You can try using the ping or telnet command on the DTS server to test the reachability of the destination host.

Error example:

java.sql.SQLException: java.net.NoRouteToHostException: No route to host (Host unreachable)

DTS-RETRY-ERR-0659: Not allowed to perform current operation on async materialized view

Possible cause: DTS attempted to write data to an async materialized view in the destination SelectDB (Doris) using StreamLoad, but this operation is not allowed. Data must be written to the base table.

Solution: Modify the DTS task configuration to set the synchronization object to the base table that corresponds to the async materialized view, not the materialized view itself.

Error example:

DTS-70003: 29 execute transaction has excess max transaction retry time [64] cause:Failed to stream load data to table: xxx.xxx status is fail. Load result: {"Status":"Fail","Comment":"","BeginTxnTimeMs":0,"Message":"[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = Not allowed to perform current operation on async materialized view","NumberUnselectedRows":0,"CommitAndPublishTimeMs":0,"Label":"DTS-SelectDB-Sink4b5a1dcd-xxx-xxx-88ad-d9ce08bcf343","LoadBytes":0,"StreamLoadPutTimeMs":0,"NumberTotalRows":0,"WriteDataTimeMs":0,"ReceiveDataTimeMs":0,"TxnId":-1,"LoadTimeMs":0,"TwoPhaseCommit":"false","ReadDataTimeMs":0,"NumberLoadedRows":0,"NumberFilteredRows":0} cause: IOException: Failed to stream load data to table: xxx.xxx status is fail. Load result: {"Status":"Fail","Comment":"","BeginTxnTimeMs":0,"Message":"[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = Not allowed to perform current operatio...

DTS-RETRY-ERR-0660: not enough space. (NOT_ENOUGH_SPACE)

Possible cause: The destination ClickHouse instance or the node it is on has insufficient disk space. It cannot reserve the required memory or write data for the write operation.

Solution: Check the storage capacity of the destination ClickHouse instance and free up or expand the disk space.

Error example:

DTS-100047: retry 1044 times, 43202 seconds, which exceed the supposed 43200 seconds cause: TransactionReplicateException: transaction-replicate: DTS-70003: 5 execute transaction has excess max transaction retry time [64] cause:transaction-replicate: DTS-70004: EXEC statement failed, may try it again RecoverableAnyAllException: transaction-replicate: DTS-70004: EXEC statement failed, may try it again SQLException: ClickHouseEngine(5): meet error, insertSQL:insert into `xxx`.`xxx` () values (), error detail message:Code: 243. DB::Exception: Cannot reserve 1.00 MiB, not enough space. (NOT_ENOUGH_SPACE) (version 23.8.16.1) , server ClickHouseNode [uri=http://10.xxx.xxx.xxx:xxx/x_magnet_dev]@-184683xxxx SQLException: Code: 243. DB::Exception: Cannot reserve 1.00 MiB, not enough space. (NOT_ENOUGH_SPACE) (version 23.8.16.1) , server ClickHouseNode [uri=http://10.xxx.xxx.xxx:xxx/x_magnet_dev]@-184683xxx IOException: Code: 243. DB::Exception: Cannot reserve 1.00 MiB, not enough space. (NOT_ENOUGH_SPACE) (version 23.8.16.1)

DTS-RETRY-ERR-0661: not support exactly-once with thread id mode

Possible cause: The DTS task is configured with the Exactly-Once write feature, but GTID is not enabled on the source database. This mode relies heavily on GTID to ensure transaction uniqueness.

Solution: Enable the GTID feature for the source database.

Error example:

DTS-11005: not support exactly-once with thread id mode

DTS-RETRY-ERR-0662: Not support fetch backup log MissActiveLogException

Possible cause: When DTS reads incremental logs from the source database, it finds that the required logs, including active and backup logs, have been purged.

Solution: Extend the transaction log backup retention period of the source database.

Error example:

DTS-100047: retry 0 times, 107903 seconds, which exceed the supposed 43200 seconds cause: RecoverableAnyAllException: dts-k-src: DTS-52110: SQLServerRecordExtractor Init Error: sqlserver-reader: DTS-52061: Failed to seek sqlserver position CriticalAnyAllException: sqlserver-reader: DTS-52061: Failed to seek sqlserver position CriticalAnyAllException: sqlserver-reader: DTS-52411: Not support fetch backup log MissActiveLogException: ERROR_MISS_TRANSACTION_LOG(timestamp:1749775487)

DTS-RETRY-ERR-0663: Object (.*)? in library (.*)? not found

Possible cause: DTS cannot find the specified object, such as a Journal log file, in the source database. This usually means that the logs required by DTS have been purged or moved.

Solution: Check the log management policy of the source database to ensure that the log retention period is long enough.

Error example:

DTS-31009: In process of processing data (recordRange: 456002531641996xxxx) failed cause: FatalAnyAllException: common: DTS-100047: retry 0 times, 276393 seconds, which exceed the supposed 43200 seconds RuntimeException: AS400DB2Source:journalFetcher-0: failed cause false; rc=0x501 CPF9801: Object xxx in library xxx not found.

DTS-RETRY-ERR-0664: ODPS partition exception - maximum (.*)? partitions allowed, or more than (.*)? partitions created

Possible cause: The number of partitions in the destination MaxCompute table has reached the upper limit, or a single operation attempted to create more partitions than the single-time limit.

Solution: Check your partition strategy to avoid creating too many partitions. You may need to clean up unnecessary historical partitions or optimize the design of the partition fields.

Error example:

DTS-70004: execute upload failed, may try it again cause: TransactionReplicateException: transaction-replicate: DTS-70003: 17 execute transaction has excess max transaction retry time [100] cause:ODPS-0110061: Failed to run ddltask - Modify DDL meta encounter exception : ODPS-0123031:ODPS partition exception - maximum xxx partitions allowed, or more than xxx partitions created OdpsException: ODPS-0110061: Failed to run ddltask - Modify DDL meta encounter exception : ODPS-0123031:ODPS partition exception - maximum xxx partitions allowed, or more than xxx partitions created

DTS-RETRY-ERR-0665: Operation is not allowed under READ-ONLY mode!

Possible cause: The destination AnalyticDB for MySQL 3.0 instance is currently in read-only mode, and DTS cannot write data.

Solution: Check the status of the destination AnalyticDB for MySQL 3.0 instance.

Error example:

java.sql.SQLException: [xxx, 202507301xxxx41921680681130315111xxxx] Operation is not allowed under READ-ONLY mode!

DTS-RETRY-ERR-0666: ORACLE initialization or shutdown in progress

Possible cause: The Oracle database that DTS is trying to connect to is undergoing initialization or shutdown and is temporarily unavailable.

Solution: Wait for the Oracle database to complete its startup or shutdown process and resume normal service. Then, restart the DTS task.

Error example:

java.sql.SQLRecoverableException: ORA-01033: ORACLE initialization or shutdown in progress

DTS-RETRY-ERR-0669: org.bson.json.JsonParseException

Possible cause: The filter condition configured for the MongoDB source in the DTS task is not a valid JSON format string.

Solution: Check and correct the JSON statement in the DTS task filter condition to ensure that its syntax is correct.

Error example:

org.bson.json.JsonParseException: Invalid JSON input. Position: 2. Character: '“'.

DTS-RETRY-ERR-0670: PROCEDURE (.*)? does not exist

Possible cause: The data synchronized by DTS triggered an operation in the destination MySQL database, such as a trigger, which attempted to call a non-existent stored procedure.

Solution: Check the destination database to ensure that the required stored procedure has been created, or remove the logic, such as triggers, that calls the stored procedure. After you fix the issue, restart the DTS task.

Error example:

DTS-100047: retry 4290 times, 43205 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: framework: DTS-30011: currentRunningSQL: /* DTS-full-lsqb6q1p18txxx */insert ignore into `xxx`.`xxx`() VALUES (),currentRow:xxx, reason: PROCEDURE xxx.xxx does not exist BatchUpdateException: PROCEDURE xxx.xxx does not exist SQLSyntaxErrorException: PROCEDURE xxx.xxx does not exist

DTS-RETRY-ERR-0671: Project: (.*)? Slot Quota Exceeded

Possible cause: The compute resources (slots) consumed by the destination MaxCompute have exceeded its quota.

Solution: Check the resource usage of the MaxCompute project. You can wait for other tasks to release resources or increase the slot quota for the project. After you confirm, restart the DTS task.

Error example:

DTS-70004: execute upload failed, may try it again cause: TransactionReplicateException: transaction-replicate: DTS-70003: 12 execute transaction has excess max transaction retry time [100] cause:odps: DTS-11014: Failed to get the upload session for xxx.xxx caused by RequestId=20250601131605b381c40b05d6xxxx, ErrorCode=SlotExceeded, ErrorMessage=Region: cn-zhangjiakou Project: xxx Slot Quota Exceeded. Total:300, Acquired:349RequestId=20250601131605b381c40b05d6xxxx, ErrorCode=SlotExceeded, ErrorMessage=Region: cn-zhangjiakou Project: xxxx Slot Quota Exceeded. Total:300, Acquired:349 CriticalAnyAllException: odps: DTS-11014: Failed to get the upload session for xxx.xxx caused by RequestId=20250601131605b381c40b05d6xxxx, ErrorCode=SlotExceeded, ErrorMessage=Region: cn-zhangjiakou Project: xxx Slot Quota Exceeded. Total:3...

DTS-RETRY-ERR-0672: Query was selected to stop by OvercommitTracker

Possible cause: The destination ClickHouse instance terminated the DTS query because of its internal OvercommitTracker mechanism, which was triggered by memory overcommitment.

Solution: Check the memory usage and load of the destination ClickHouse instance. You may need to optimize queries, increase node memory, or upgrade the instance type.

Error example:

java.sql.BatchUpdateException: Code: 241. DB::Exception: Memory limit (total) exceeded: would use xxx GiB (attempt to allocate chunk of xxx bytes), maximum: xxx GiB. OvercommitTracker decision: Query was selected to stop by OvercommitTracker.: (avg_value_size_hint = xxx, avg_chars_size = xxx, limit = xxx): (while reading column order_sn): (while reading from part /clickhouse/xxx/xxx/store/xxx/c3ff3852-5f5d-xxx-xxx-95e34f91fdf8/all_xxx/ in table xxx (c3ff3852-5f5d-xxx-xxx-95e34f91fdf8) located on disk default of type local, from mark 192 with max_rows_to_read = xxx): While executing MergeTreeThread: while pushing to view xxx (2b1cfb37-6694-xxx-xxx-785513b23836). (MEMORY_LIMIT_EXCEEDED) (version 23.8.16.1) , server ClickHouseNode [uri=http://10.xxx.xxx.xxx:xxx/default]@742123xxx

DTS-RETRY-ERR-0673: READONLY You can't write against a read only replica.

Possible cause: DTS is configured to write data to a read-only Redis replica node.

Solution: Modify the destination configuration of the DTS task to point to the Redis master node.

Error example:

redis.clients.jedis.exceptions.JedisDataException: READONLY You can't write against a read only replica.

DTS-RETRY-ERR-0674: RecordTooLargeException

Possible cause: The size of a message that DTS is trying to send to the destination Kafka exceeds Kafka's maximum message delivery size limit.

Solution: Check the source data to confirm if there are records that contain oversized fields, such as very large TEXT or BLOB fields. If the data is normal, increase the maximum message delivery size limit of the destination Kafka. You can also configure object column mapping in the DTS task to exclude the fields that are causing the message to be too large.

Error example:

DTS-100047: retry 4102 times, 43206 seconds, which exceed the supposed 43200 seconds cause: ExecutionException: com.xxx.kafka_thin_xxx.common.errors.RecordTooLargeException: The request included a message larger than the max message size the server will accept. RecordTooLargeException: The request included a message larger than the max message size the server will accept.

DTS-RETRY-ERR-0675: rejected execution of primary operation

Possible cause: The destination Elasticsearch has triggered a circuit breaker mechanism because of excessive resource pressure, such as high CPU, memory, or disk usage, and has rejected the write request from DTS.

Solution: Check the health status and resource load of the destination Elasticsearch. You may need to scale out the cluster, optimize indexes, or reduce the write rate.

Error example:

DTS-100047: retry 15 times, 219 seconds, which exceed the supposed 922337203685477xxxx seconds cause: IOException: es: DTS-10035: Index error: rejected execution of primary operation [coordinating_and_primary_bytes=xxx, replica_bytes=0, all_bytes=xxx, primary_operation_bytes=xxx, max_coordinating_and_primary_bytes=xxx] CriticalAnyAllException: es: DTS-10035: Index error: rejected execution of primary operation [coordinating_and_primary_bytes=xxx, replica_bytes=0, all_bytes=141360231, primary_operation_bytes=xxx, max_coordinating_and_primary_bytes=xxx]

DTS-RETRY-ERR-0676: Requests rate limit exceeded

Possible cause: The write rate from DTS to the destination Elasticsearch instance has exceeded the service's request frequency limit.

Solution: Set a reasonable synchronization rate limit in the DTS task configuration, or upgrade the destination Elasticsearch instance to obtain higher throughput capacity.

Error example:

DTS-100047: retry 15 times, 79 seconds, which exceed the supposed 922337203685477xxx seconds cause: IOException: es: DTS-10035: Index error: Error from remote service: {"error": {"code": "Throttling.RateQuota", "message": "Requests rate limit exceeded, please try again later."}} CriticalAnyAllException: es: DTS-10035: Index error: Error from remote service: {"error": {"code": "Throttling.RateQuota", "message": "Requests rate limit exceeded, please try again later."}}

DTS-RETRY-ERR-0677: rollback segment number (.*)? with name "(.*)?" too small

Possible cause: This is an Oracle ORA-01555: snapshot too old error. The full query that was executed by DTS took too long. During this time, many data changes occurred in the source database. As a result, Oracle could not use the UNDO information to construct a consistent data view as of the query start time.

Solution: Increase the size of the UNDO tablespace in the source Oracle database.

Error example:

DTS-31009: In process of processing data (recordRange: 868490004694617xxxx) failed cause: FatalAnyAllException: common: DTS-100047: retry 0 times, 48910 seconds, which exceed the supposed 43200 seconds CriticalAnyAllException: framework: DTS-31009: read source data error SQLException: ORA-01555: snapshot too old: rollback segment number xxx with name "xxx" too small

DTS-RETRY-ERR-0678: routing is required

Possible cause: The destination Elasticsearch index requires a _routing value to be provided when documents are written, but the DTS task has not configured a corresponding routing rule.

Solution: In the DTS task, configure the correct routing field based on your Elasticsearch index design.

Error example:

DTS-100047: retry 15 times, 15 seconds, which exceed the supposed 922337203685477xxxx seconds cause: IOException: es: DTS-10035: routing is required for [xxx]/[xxx] CriticalAnyAllException: es: DTS-10035: routing is required for [xxx]/[xxx]

DTS-RETRY-ERR-0679: Row size too large

Possible cause: The total size of a row that DTS tried to write exceeds the maximum row size limit of the destination MySQL table.

Solution: Check and optimize the source and destination table schemas. You can change some long VARCHAR fields to TEXT or BLOB types, because the data in these types does not fully count toward the row size limit. After you modify the table schema, restart the DTS task.

Error example:

DTS-100047: retry 112 times, 1203 seconds, which exceed the supposed 1200 seconds cause: CriticalAnyAllException: framework: DTS-30011: currentRunningSQL: /* DTS-full-dedl7sabm8gxxxx */insert ignore into `xxx`.`xxx`() VALUES (),currentRow:xxx, reason: Row size too large (> xxx). Changing some columns to TEXT or BLOB or using ROW_FORMAT=DYNAMIC or ROW_FORMAT=COMPRESSED may help. In current row format, BLOB prefix of 768 bytes is stored inline. BatchUpdateException: Row size too large (> xxx). Changing some columns to TEXT or BLOB or using ROW_FORMAT=DYNAMIC or ROW_FORMAT=COMPRESSED may help. In current row format, BLOB prefix of 768 bytes is stored inline. MySQLSyntaxErrorException: Row size too large (> xxx). Changing some columns to TEXT or BLOB or using ROW_FORMAT=DYNAMIC or ROW_FORMAT=COMPRESSED may help. In current row format, BLOB prefix of 768 bytes is stored inline.

DTS-RETRY-ERR-0680: Running in read-only mode

Possible cause: The destination MySQL database instance is currently in read-only mode (read_only=ON), and DTS cannot write data.

Solution: Check the status of the destination database, set it to read-write mode, and then restart the DTS task.

Error example:

DTS-100047: retry 4201 times, 43203 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: framework: DTS-30011: currentRunningSQL: /* DTS-full-ia8o8et1129xxxx */insert ignore into `xxx`.`xxx`() VALUES (),currentRow:null, reason: Running in read-only mode BatchUpdateException: Running in read-only mode SQLException: Running in read-only mode

DTS-RETRY-ERR-0681: Schema (.*)? not found

Possible cause: When DTS accesses the database, it cannot find the specified schema, which is equivalent to a database or user in some databases. This may be because the schema does not exist, or the DTS account does not have permission to access it.

Solution: Ensure that the schema name configured in DTS is correct and that the DTS account has been granted permission to access the schema.

Error example:

DTS-31009: In process of processing data (recordRange: 232142460080776xxxx) failed cause: FatalAnyAllException: common: DTS-100047: retry 0 times, 2 seconds, which exceed the supposed 43200 seconds DbException: Schema "xxx" not found [90079-193] JdbcSQLException: Schema "xxx" not found [90079-193]

DTS-RETRY-ERR-0682: Server shutdown in progress

Possible cause: The source MySQL database server is shutting down and is no longer accepting any operations.

Solution: Check the status of the source database. After it has completed its shutdown and restart process and resumed service, restart the DTS task.

Error example:

DTS-31009: read source data error cause: MySQLNonTransientConnectionException: Server shutdown in progress

DTS-RETRY-ERR-0683: set connection_id fail

Possible cause: DTS failed to set the connection_id when connecting to a PolarDB Multi-master Cluster (Limitless) Edition. This may indicate that the destination database version or configuration does not support this specific operation.

Solution: This is a compatibility issue between DTS and a specific database version. Confirm whether the database supports this operation.

Error example:

DTS-70032: set connection_id fail exceed retry time=4, threadid=4294963209, biId=264, adaptedBiId=9, fatal error, let's exit

DTS-RETRY-ERR-0684: SQLException: could not find next log

Possible cause: When reading the source MySQL binary log files in sequence, DTS cannot find the next expected file. For example, it cannot find .000906 after mysql-bin.000905. This usually means that the binary log files have been manually or automatically purged by the database.

Solution: Extend the binary log retention policy of the source database. You need to restart the DTS task or resume the task by skipping the offset, which carries a risk of data loss.

Error example:

DTS-100047: retry 0 times, 1000 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: mysql-reader: DTS-52200: process error failed cause: IOException: java.sql.SQLException: could not find next log; the first event 'mysql-bin.000345' at 763652xxx, the last event read from './mysql-bin.000905' at 1073755xxx, the last byte read from './mysql-bin.000905' at 1073755xxx. SQLException: could not find next log; the first event 'mysql-bin.000345' at 763652xxx, the last event read from './mysql-bin.000905' at 1073755xxx, the last byte read from './mysql-bin.000905' at 1073755xxx. CriticalAnyAllException: mysql-reader: DTS-52200: process error failed IOException: java.sql.SQLException: could not find next log; the first event 'mysql-bin.000345' at 763652xxx, the last event read from './mysql-bin.000905' at 1073755xxx, the last byte read from './mysql-bin.000905' at 1073755xxx. SQLException: could not find next log; the first event 'mysql-bin.000345' at 763652xxx, the last event read from './mysql-bin.000905' at 1073755xxx, the last byte read from './mysql-bin.000905' at 1073755xxx.

DTS-RETRY-ERR-0685: SQLException: Operand should contain 1 column

Possible cause: In the SQL query constructed by DTS, a subquery returned more than one column, while its context only allows a single column. This usually occurs when DTS is configured with a filter condition that contains a syntax error.

Solution: Check the filter condition configured for the source object in the DTS task and ensure that it complies with SQL syntax, especially the usage of subqueries.

Error example:

DTS-31009: In process of processing data (recordRange: 692535449456161xxxx) failed cause: FatalAnyAllException: common: DTS-100047: retry 4296 times, 43209 seconds, which exceed the supposed 43200 seconds CriticalAnyAllException: framework: DTS-30020: execute sql:/* DTS-full-h78o5f6927bxxxx */SELECT xxx FROM xxx WHERE ( xxx ) --> (null, null) record range id = 692535449456161xxx, fetchSize = -214748xxx SQLException: Operand should contain 1 column(s)

DTS-RETRY-ERR-0686: SSLProtocolException: Received fatal alert: unexpected_message

Possible cause: During SSL encrypted communication between DTS and the source PostgreSQL database, an unexpected message that does not conform to the SSL protocol was received, which caused the handshake to fail.

Solution: Check for any abnormalities in the network between DTS and the source database. After you confirm, restart the DTS task.

Error example:

DTS-52111: Increment Context Is Not Running..: cause: CriticalAnyAllException: postgresql-reader: DTS-52510: Fetch postgresql logical log failed FatalAnyAllException: common: DTS-100047: retry 0 times, 851577266 seconds, which exceed the supposed 600 seconds IOException: org.postgresql.util.PSQLException: Database connection failed when reading from copy PSQLException: Database connection failed when reading from copy SSLProtocolException: Received fatal alert: unexpected_message

DTS-RETRY-ERR-0687: STANDBY_MAX_DATA_DELAY of (.*)? seconds exceeded

Possible cause: DTS is connected to an Oracle standby database that is configured with a data protection mode. The synchronization latency between the primary and standby databases has exceeded the threshold set by the STANDBY_MAX_DATA_DELAY parameter. This caused the standby database to reject the query to ensure data consistency.

Solution: Check and resolve the latency issue between the primary and standby Oracle databases. Alternatively, if your business allows, you can consider connecting to the primary database for synchronization.

Error example:

java.sql.SQLException: ORA-03172: STANDBY_MAX_DATA_DELAY of xxx seconds exceeded

DTS-RETRY-ERR-0688: String or binary data would be truncated in table (.*)?, column '(.*)?. Truncated value

Possible cause: The length of a field in the source data exceeds the maximum length of the corresponding column in the destination SQL Server table. This causes the data to be truncated when it is written.

Solution: Compare the source and destination table schemas. You need to increase the length of the corresponding column in the destination table (for example, from VARCHAR(50) to VARCHAR(200)) to accommodate the source data. After you modify the table schema, restart the DTS task.

Error example:

java.sql.BatchUpdateException: String or binary data would be truncated in table 'xxx', column 'xxx'. Truncated value: 'xxx'.

DTS-RETRY-ERR-0689: Subquery returned more than 1 value.

Possible cause: An SQL statement executed by DTS contains a subquery that returned more than one value when used as an expression, which is not allowed in SQL. This is often caused by triggers or foreign key logic on the destination database that triggers a problematic subquery when DTS writes data.

Solution: Check the automated logic on the destination database related to the synchronized tables, such as triggers and foreign keys. Ensure that they do not produce this type of error when DTS writes data. We recommend that you disable this logic during synchronization or migration.

Error example:

com.microsoft.sqlserver.jdbc.SQLServerException: Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression.

DTS-RETRY-ERR-0690: Table has no partition for value

Possible cause: The value of the partition key for the data that DTS is trying to write does not have a corresponding partition in the destination MySQL partitioned table.

Solution: Based on the partition key values of the source data, check and modify the partition definition of the destination table to ensure that all possible data can be routed to an appropriate partition. For example, you can add new partitions for future time ranges.

Error example:

DTS-077100: Record Replicator error in table xxx. cause by [java.sql.SQLException: Table has no partition for value from column_list] About more information in [https://xxx].

DTS-RETRY-ERR-0691: table not found

Possible cause: DTS tried to write data to the destination SelectDB (Doris), but the specified table does not exist in the destination database.

Solution: Check whether the synchronization object configured in the DTS task is correct, and ensure that the corresponding table has been created in the destination database.

Error example:

java.io.IOException: Failed to stream load data to SelectDB.Key status is fail. Load result: {"Status":"Fail","Comment":"","BeginTxnTimeMs":0,"Message":"[NOT_FOUND]TStatus: errCode = 7, detailMessage = table not found, tableName=xxx","NumberUnselectedRows":0,"CommitAndPublishTimeMs":0,"Label":"DTS-SelectDB-Sinkc16c2c5d-xxx-xxx-xxx-5a647cb41cf1","LoadBytes":0,"StreamLoadPutTimeMs":0,"NumberTotalRows":0,"WriteDataTimeMs":0,"ReceiveDataTimeMs":0,"TxnId":-1,"LoadTimeMs":0,"TwoPhaseCommit":"false","ReadDataTimeMs":0,"NumberLoadedRows":0,"NumberFilteredRows":0}

DTS-RETRY-ERR-0692: the database system is in recovery mode

Possible cause: The PostgreSQL database that DTS is trying to connect to is currently in recovery mode and cannot accept normal connections.

Solution: Wait for the database to complete the recovery process. After the database resumes normal service, restart the DTS task.

Error example:

org.postgresql.util.PSQLException: FATAL: the database system is in recovery mode

DTS-RETRY-ERR-0693: the database system is not yet accepting connections

Possible cause: The PostgreSQL database that DTS is trying to connect to is starting up but has not yet reached a consistent recovery state where it can accept connections.

Solution: Wait for the database to complete its startup process. After the database resumes normal service, restart the DTS task.

Error example:

org.postgresql.util.PSQLException: FATAL: the database system is not yet accepting connections Detail: Consistent recovery state has not been yet reached.

DTS-RETRY-ERR-0694: the database system is starting up

Possible cause: The PostgreSQL database that DTS is trying to connect to is in the process of starting up and cannot temporarily accept connections.

Solution: Wait for the database to finish starting up. After the database resumes normal service, restart the DTS task.

Error example:

org.postgresql.util.PSQLException: FATAL: the database system is starting up

DTS-RETRY-ERR-0695: The index entry of length (.*)? bytes for the index (.*)? exceeds the maximum length of (.*)? bytes

Possible cause: In the row to be inserted or updated, the combined values of one or more columns cause the total length of the index entry to exceed the maximum index key length that is allowed by SQL Server (for example, xxx bytes for a non-clustered index).

Solution: Check the source data for any excessively long values. You need to shorten the data length of the relevant fields or redesign the index of the destination table to avoid including overly long columns in the index.

Error example:

com.microsoft.sqlserver.jdbc.SQLServerException: Operation failed. The index entry of length xxx bytes for the index 'xxx' exceeds the maximum length of xxx bytes.

DTS-RETRY-ERR-0696: The INSERT permission was denied on the object (.*)?

Possible cause: The account used by DTS to connect to the destination SQL Server database does not have permission to perform INSERT operations on the specified table.

Solution: Grant INSERT permission on the relevant tables to the destination database account used by DTS.

Error example:

java.sql.BatchUpdateException: The INSERT permission was denied on the object 'xxx', database 'xxx', schema 'xxx'.

DTS-RETRY-ERR-0697: The SELECT would examine more than (.*)? rows

Possible cause: A query executed by DTS on a PolarDB columnstore index read-only node is expected to scan more rows than the threshold set by the RDS_AP_THREHOLD parameter. The database rejected the query to prevent performance issues.

Solution: Increase the value of the RDS_AP_THREHOLD parameter for the destination PolarDB database, or set the session to SET SQL_AP_SELECTS=ON. After you make the adjustment, restart the DTS task.

Error example:

DTS-30020: execute sql:/* DTS-full-d1gc8s3615zxxx */SELECT xxx FROM xxx --> (null, null) record range id = 271169604478094xxxx, fetchSize = -214748xxxx cause: SQLException: The SELECT would examine more than RDS_AP_THREHOLD rows; SET SQL_AP_SELECTS=ON or redirect to AP node.

DTS-RETRY-ERR-0698: The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs

Possible cause: DTS is using GTID mode for replication, but the binary log file that contains the required starting GTID offset has been purged on the source MySQL server.

Solution: Immediately increase the binary log retention period of the source MySQL database.

Error example:

DTS-100047: retry 0 times, 1000 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: mysql-reader: DTS-52200: process error failed cause: IOException: java.sql.SQLException: The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires. Replicate the missing transactions from elsewhere, or provision a new slave from backup. Consider increasing the master's binary log expiration period. The GTID set sent by the slave is '487e509d-316f-xxx-xxx-00163e257207:1-xxxx', and the missing transactions are '487e509d-316f-xxx-xxx-00163e257207:125941-xxx'. SQLException: The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires. Replicate the missing transactions from elsewhere, or provision a new slave from backup. Consider increasing the master's binary log expiration period. The GTID set sent by the slave is '487e509d-xxxx-xxx-xxx-00163e257207:1-125xxxx', and the missing transactions are '487e509d-xxx-xxx-xxxx-00163e257207:125941-12xxx'. CriticalAnyAllException: mysql-reader: DTS-52200: process error failed IOException: java.sql.SQLException: The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires. Replicate the missing transactions from elsewhere, or provision a new slave from backup. Consider increasing the master's binary log expiration period. The GTID set sent by the slave is '487e509d-xxx-xxx-xxx-00163e257207:1-125xxx', and the missing transactions are '487e509d-xxx-xxx-xxx-00163e257207:125941-12xxx'. SQLException: The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires. Replicate the

DTS-RETRY-ERR-0699: The specified download session id does not exist.

Possible cause: The session ID used by DTS to download data from MaxCompute has expired or is invalid.

Solution: This is a transient session issue. You can restart the DTS task. DTS will automatically create a new download session.

Error example:

RequestId=20250626174326a181c40b04xxxx, ErrorCode=NoSuchDownload, ErrorMessage=The specified download session id does not exist.

DTS-RETRY-ERR-0700: The specified table has been modified since the download initiated.

Possible cause: During the process of DTS loading data from MaxCompute, the schema of the destination table was modified (for example, by executing ALTER TABLE). This caused the download session to become invalid.

Solution: Do not perform DDL operations on the destination table in MaxCompute during a DTS full synchronization.

Error example:

DTS-31009: In process of processing data (recordRange: 788966514532706xxx) failed cause: CriticalAnyAllException: odps: DTS-11014: odps getRecordReader error,project:xxx table:xxx FatalAnyAllException: common: DTS-100047: retry 2535 times, 43206 seconds, which exceed the supposed 43200 seconds TunnelException: RequestId=202507290629070934c40b02fxxx, ErrorCode=TableModified, ErrorMessage=The specified table has been modified since the download initiated. Try initiate another download.