DTS-RETRY-ERR-0401 to 0500
This topic describes the error messages (DTS-RETRY-ERR-0401 to DTS-RETRY-ERR-0500) that are returned when a DTS task fails, and provides solutions for each error.
In this topic, the **(.*)?** regular expression represents variable content in error messages.
Error index
| Database type | Error codes |
|---|---|
| SQL Server | 0401, 0402, 0403, 0404, 0405, 0407, 0408, 0409, 0410, 0411, 0412, 0414, 0416, 0418, 0419, 0420, 0421, 0424, 0426, 0427, 0428, 0429, 0433, 0434, 0435, 0481 |
| Oracle | 0415, 0422, 0423, 0436 |
| MySQL / PolarDB-X | 0437, 0441, 0442, 0456, 0459, 0460, 0462, 0463, 0493, 0495 |
| Redis | 0450, 0451, 0452, 0453, 0454, 0455, 0457, 0458, 0461, 0464, 0465, 0466, 0467, 0468, 0470 |
| ClickHouse | 0438, 0444, 0448 |
| MongoDB | 0473, 0474, 0475, 0476, 0477, 0478, 0479, 0480 |
| AnalyticDB MySQL | 0447, 0469, 0471, 0472 |
| Elasticsearch | 0483, 0484, 0485, 0486, 0487, 0488, 0489 |
| Function Compute | 0490 |
| Greenplum / PostgreSQL | 0491, 0492, 0499, 0500 |
| OceanBase / Doris | 0482, 0494 |
| MaxCompute | 0496, 0497, 0498 |
| General | 0417 |
SQL Server
DTS-RETRY-ERR-0401: CDC partition (.)? capture instance is not found
Possible cause: The CDC capture instance in the source database has been cleaned up.
Solution:
-
Method 1: Contact the DBA of the source database to restore the capture instance, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 1000 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52340: Serial transaction log failed CriticalAnyAllException: sqlserver-reader: DTS-52421: CDC partition 7**** capture instance is not found, LSN=00****
DTS-RETRY-ERR-0402: com.microsoft.sqlserver.jdbc.SQLServerException: Read timed out
Possible cause: DTS timed out while reading data from the source database.
Solution: Ensure that the source database is accessible and that DTS can read data from it normally, and then restart the DTS task.
Example:
com.microsoft.sqlserver.jdbc.SQLServerException: Read timed out ClientConnectionId:eb622896-230******
DTS-RETRY-ERR-0403: Database cdc is disable mode
Possible cause: CDC is disabled in the source database.
Solution: Enable CDC in the source database, and then restart the DTS task.
Example:
DTS-100047: retry 0 times, 12011 seconds, which exceed the supposed 43200 seconds cause: RecoverableAnyAllException: dts-k-src: DTS-52110: SQLServerRecordExtractor Init Error: sqlserver-reader: DTS-52405: Database cdc is disable mode CriticalAnyAllException: sqlserver-reader: DTS-52405: Database cdc is disable mode
DTS-RETRY-ERR-0404: ERROR_LOAD_CDC_DATA,Capture Instance
Possible cause: DTS failed to load the CDC instance in the source database.
Solution: Ensure that the CDC job in the source database is running normally and that the number of tables with CDC enabled in the DTS task does not exceed the recommended limit of 1,000. Then restart the DTS task.
Example:
DTS-100047: retry 0 times, 1000 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: ExecutionException: java.lang.RuntimeException: ERROR_LOAD_CDC_DATA,Capture Instance **** RuntimeException: ERROR_LOAD_CDC_DATA,Capture Instance{dbo_BAS_Location_dts_cap_20241205160405,dbo_BAS_Codes_dts_cap_2024120812****,dbo_INV_LOT_LOC_ID_dts_cap_2024120516****}
DTS-RETRY-ERR-0405: execute sql DBCC TRACEOFF (.)? failed
Possible cause: The source database account used by the DTS task has insufficient permissions.
Solution:
-
Method 1: Grant the necessary permissions to the database account based on the error message, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task using a database account with sufficient permissions.
Example:
DTS-52110: SQLServerRecordExtractor Init Error: sqlserver-reader: DTS-52061: Failed to seek sqlserver position cause: CriticalAnyAllException: sqlserver-reader: DTS-52061: Failed to seek sqlserver position CriticalAnyAllException: sqlserver-reader: DTS-52411: Fetch backup log failed CriticalAnyAllException: sqlserver-reader: DTS--0001: execute sql DBCC TRACEOFF(2537,-1) failed, case by common: DTS-100047: retry 0 times, 0 seconds, which exceed the supposed 43200 seconds FatalAnyAllException: common: DTS-100047: retry 0 times, 0 seconds, which exceed the supposed 43200 seconds SQLServerException: Database *** does not exist. Make sure that the name is entered correctly.
DTS-RETRY-ERR-0407: Fetch RDS SQL Server backup log failed, fetch lsn
Possible cause: The transaction logs in the source database have been cleared.
Solution:
-
Method 1: Contact the DBA of the source database to restore the logs, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 281445 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52410: Sqlserver log fetch failed CriticalAnyAllException: sqlserver-reader: DTS-52411: Fetch RDS SQL Server backup log failed, fetch lsn = ****
DTS-RETRY-ERR-0408: get table (.)? detail info failed
| Possible cause | Solution |
|---|---|
| The destination table does not exist. | Ensure that the destination database is accessible, that the required tables exist, and that the destination database account has sufficient permissions. Then restart the DTS task. |
| The destination database account has insufficient permissions. | Method 1: Grant the necessary permissions to the account based on the error message, and then restart the DTS task. Method 2: Reconfigure the DTS task using an account with sufficient permissions. |
Example:
DTS-31007: get table **** detail info failed after retrying. cause: FatalAnyAllException: common: DTS-100047: retry 4307 times, 43204 seconds, which exceed the supposed 43200 seconds RecoverableAnyAllException: replicate-sqlserver: DTS-70002: get table [[DC_CRM_SIT].[dbo].[Test****]] detail info failed. SQLServerException: Invalid object name 'DC_CRM_SIT.dbo.Test****'.
DTS-RETRY-ERR-0409: History cdc (.)? capture instance is deleted
Possible cause: The CDC capture instance in the source database has been cleaned up.
Solution:
-
Method 1: Contact the DBA of the source database to restore the capture instance, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 102730 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52340: Serial transaction log failed CriticalAnyAllException: sqlserver-reader: DTS-52422: History cdc **** capture instance is deleted, LSN=0000****
DTS-RETRY-ERR-0410: Incorrect syntax near the keyword
Possible cause: The DTS task configuration contains an error. This commonly occurs when filter conditions or data processing (ETL) statements include a syntax error.
Solution: Reconfigure the DTS task. If the task uses filter conditions or ETL statements, verify the syntax is correct.
Example:
DTS-31009: In process of processing data (recordRange: 573855793005****) failed cause: FatalAnyAllException: common: DTS-100047: retry 4309 times, 43201 seconds, which exceed the supposed 43200 seconds CriticalAnyAllException: framework: DTS-30020: execute sql:/* DTS-full-h**** --> (null, null) record range id = 5738557930056****, fetchSize = 10000 SQLServerException: Incorrect syntax near the keyword 'FROM'.
DTS-RETRY-ERR-0411: Input parameter LSN can't be null or empty
Possible cause: The transaction logs in the source database have been cleared, leaving DTS without a valid LSN to resume from.
Solution:
-
Method 1: Contact the DBA of the source database to restore the logs, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 1000 seconds, which exceed the supposed 7200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52410: Sqlserver log fetch failed IOException: sqlserver-reader: DTS-52415: Input parameter LSN can't be null or empty CriticalAnyAllException: sqlserver-reader: DTS-52415: Input parameter LSN can't be null or empty
DTS-RETRY-ERR-0412: invalid backup log
Possible cause: The transaction logs in the source database have been cleared.
Solution:
-
Method 1: Contact the DBA of the source database to restore the logs, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 1002 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52410: Sqlserver log fetch failed CriticalAnyAllException: sqlserver-reader: DTS-52413: invalid backup log
DTS-RETRY-ERR-0414: Login failed for user (.*)?
Possible cause: DTS cannot connect to the source database. This may be due to incorrect credentials, a server upgrade, or access restrictions.
Solution: Ensure that DTS can connect to and read data from the source database normally, and then restart the DTS task.
Example:
DTS-100047: retry 0 times, 1000 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52410: Sqlserver log fetch failed IOException: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user '****'. Reason: Server is in script upgrade mode. Only administrator can connect at this time. ClientConnectionId:a6e3cd3f-9ee9-4065-84e0-7**** SQLServerException: Login failed for user '****'. Reason: Server is in script upgrade mode. Only administrator can connect at this time. ClientConnectionId:a6e3cd3f-9ee9-4065-84e0-7****
DTS-RETRY-ERR-0416: Not found capture log at (.*)?
Possible cause: The CDC in the source database has been cleaned up.
Solution:
-
Method 1: Contact the DBA of the source database to resolve the issue, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 11010 seconds, which exceed the supposed 43200 seconds cause: IllegalArgumentException: Not found capture log at {timestamp:173527****}
DTS-RETRY-ERR-0418: Not support fetch backup log MissActiveLogException: ERROR_MISS_TRANSACTION_LOG
Possible cause: The transaction logs in the source database have been truncated or cleared.
Solution:
-
Method 1: Contact the DBA of the source database to restore the logs, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 14054 seconds, which exceed the supposed 43200 seconds cause: RecoverableAnyAllException: dts-k-src: DTS-52110: SQLServerRecordExtractor Init Error: sqlserver-reader: DTS-52061: Failed to seek sqlserver position CriticalAnyAllException: sqlserver-reader: DTS-52061: Failed to seek sqlserver position CriticalAnyAllException: sqlserver-reader: DTS-52411: Not support fetch backup log MissActiveLogException: ERROR_MISS_TRANSACTION_LOG(timestamp:17376****)
DTS-RETRY-ERR-0419: Operating system error 38
Possible cause: The transaction logs in the source database have been cleared, and the log file has reached the end of file.
Solution:
-
Method 1: Contact the DBA of the source database to restore the logs, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 1302 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52410: Sqlserver log fetch failed IOException: com.microsoft.sqlserver.jdbc.SQLServerException: Operating system error 38(Reached the end of the file.) on file "****" during SQLServerLogMgr::CheckLogBlockReadComplete. SQLServerException: Operating system error 38(Reached the end of the file.) on file "****" during SQLServerLogMgr::CheckLogBlockReadComplete.
DTS-RETRY-ERR-0420: operation:LOP_MODIFY_COLUMNS context:LCX_HEAP
Possible cause: The SQL Server Incremental Synchronization Mode parameter is set to Incremental Synchronization Based on Logs of Source Database (Heap tables are not supported), but the migration or synchronization objects include heap tables, which are not supported in this mode.
Solution: Reconfigure the DTS task and set the SQL Server Incremental Synchronization Mode parameter to a mode that supports heap tables.
Example:
DTS-100047: retry 0 times, 1001 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52340: Serializer redo log entry failed RuntimeException: updateBackEntry: currentLsn:000535b0:00046fa0:**** operation:LOP_MODIFY_COLUMNS context:LCX_HEAP transactionID:0000:1a0**** allocUnitName: partitionId:7205****
DTS-RETRY-ERR-0421: operation:LOP_MODIFY_ROW context:LCX_HEAP
Possible cause: The SQL Server Incremental Synchronization Mode parameter is set to Incremental Synchronization Based on Logs of Source Database (Heap tables are not supported), but the migration or synchronization objects include heap tables.
Solution: Reconfigure the DTS task and set the SQL Server Incremental Synchronization Mode parameter to a mode that supports heap tables.
Example:
DTS-100047: retry 0 times, 1103 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52340: Serializer redo log entry failed RuntimeException: updateBackEntry: currentLsn:00e0a112:00010a50:**** operation:LOP_MODIFY_ROW context:LCX_HEAP transactionID:0000:742****
DTS-RETRY-ERR-0424: serializeRow fail rowsetId (.)? LSN = (.*)?. case by 12 ArrayIndexOutOfBoundsException: 12
Possible cause: DTS failed to parse the transaction logs in the source database.
Solution:
-
Method 1: Contact the DBA of the source database to investigate and resolve the log parsing issue, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 1001 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52340: Serializer redo log entry failed CriticalAnyAllException: sqlserver-reader: DTS-52340: serializeRow fail rowsetId = 720575941715****, LSN = 00010c5c:00012****. case by 12 ArrayIndexOutOfBoundsException: 12
DTS-RETRY-ERR-0426: SQL Server (.)? table cdc query back failed
Possible cause: The CDC feature in the source SQL Server database is in an abnormal state.
Solution:
-
Method 1: Contact the DBA of the source database to restore normal CDC operation, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 1001 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52340: Serial transaction log failed CriticalAnyAllException: sqlserver-reader: DTS-52343: SQL Server bigtree.dbo.intlroomstock table cdc query back failed CriticalAnyAllException: sqlserver-reader: DTS-52343: SQL Server bigtree.dbo.intlroomstock table cdc query back failed
DTS-RETRY-ERR-0427: SQL Server agent service not running, please restart it
Possible cause: The SQL Server Agent service in the source database is not running.
Solution: Contact the DBA of the source database to start the SQL Server Agent service, and then restart the DTS task.
Example:
DTS-100047: retry 0 times, 1100 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52405: SQL Server agent service not running, please restart it
DTS-RETRY-ERR-0428: SQL Server did not return a response
Possible cause: The transaction logs in the source database have been cleared, causing the connection to drop.
Solution:
-
Method 1: Contact the DBA of the source database to restore the logs, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 5003 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52410: Sqlserver log fetch failed IOException: com.microsoft.sqlserver.jdbc.SQLServerException: SQL Server did not return a response. The connection has been closed. SQLServerException: SQL Server did not return a response. The connection has been closed.
DTS-RETRY-ERR-0429: Sqlserver log fetch failed
Possible cause: The transaction logs in the source database have been cleared.
Solution:
-
Method 1: Contact the DBA of the source database to restore the logs, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 1001 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52410: Sqlserver log fetch failed IOException: java.lang.NumberFormatException: For input string:****
DTS-RETRY-ERR-0431: Operating system error (.*)?occurred on file(.*)?during SQLServerLogMgr::CheckLogBlockReadComplete. End of file reached.
Possible cause: The logs of the source database have been cleared.
Solution:
-
Method 1: Contact the DBA of the source database to resolve the issue, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
common: DTS-100047: retry 0 times, 1200 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52410: Sqlserver log fetch failed IOException: com.microsoft.sqlserver.jdbc.SQLServerException: Operating system error 38 (Reached end of file.) occurred during SQLServerLogMgr::CheckLogBlockReadComplete for file "****". SQLServerException: Operating system error 38 (Reached end of file.) occurred during SQLServerLogMgr::CheckLogBlockReadComplete for file "****".
DTS-RETRY-ERR-0433: The SELECT permission was denied on the
Possible cause: The source database account used by the DTS task lacks SELECT permission on system objects.
Solution:
-
Method 1: Grant the necessary permissions to the database account based on the error message, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task using a database account with sufficient permissions.
Example:
DTS-100047: retry 0 times, 1001 seconds, which exceed the supposed 7200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52410: Sqlserver log fetch failed IOException: com.microsoft.sqlserver.jdbc.SQLServerException: The SELECT permission was denied on the object '****', database '****', schema 'sys'. SQLServerException: The SELECT permission was denied on the object '****', database '****', schema 'sys'.
DTS-RETRY-ERR-0434: uildPartitionMeta rowsetId=(.*)?
Possible cause: DTS failed to parse the transaction logs in the source database.
Solution:
-
Method 1: Contact the DBA of the source database to investigate and resolve the log parsing issue, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 1001 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: sqlserver-reader: DTS-52340: Serializer redo log entry failed CriticalAnyAllException: sqlserver-reader: DTS-52420: uildPartitionMeta rowsetId=72057594083****8. case by null
DTS-RETRY-ERR-0435: UNSupport sqlserver simple recover mode
Possible cause: The log recovery mode of the source SQL Server database is not set to Full recovery mode, which is required for incremental data capture.
Solution: Contact the DBA of the source database to change the log recovery mode to Full, and then reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 11410 seconds, which exceed the supposed 43200 seconds cause: RecoverableAnyAllException: dts-k-src: DTS-52110: SQLServerRecordExtractor Init Error: sqlserver-reader: DTS-52401: UNSupport sqlserver simple recover mode CriticalAnyAllException: sqlserver-reader: DTS-52401: UNSupport sqlserver simple recover mode
DTS-RETRY-ERR-0481: because it does not exist or you do not have permissions.
| Possible cause | Solution |
|---|---|
| The destination table does not exist. | Ensure that the destination database is accessible, that the required tables exist, and that the destination database account has sufficient permissions. Then restart the DTS task. |
| The destination database account has insufficient permissions. | Method 1: Grant the necessary permissions to the account based on the error message, and then restart the DTS task. Method 2: Reconfigure the DTS task using an account with sufficient permissions. |
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: SQLServerException: Cannot find the object "ManyStores_GoodsHelper.dbo.CacheData" because it does not exist or you do not have permissions.
Oracle
DTS-RETRY-ERR-0415: Miss redo log file
Possible cause: The redo logs in the source Oracle database have been cleared.
Solution:
-
Method 1: Contact the DBA of the source database to restore the redo logs, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-52111: Increment Context Is Not Running..: cause: CriticalAnyAllException: oracle-reader: DTS-52320: Fetch oracle log failed FatalAnyAllException: common: DTS-100047: retry 0 times, 100 seconds, which exceed the supposed 43200 seconds CriticalAnyAllException: oracle-reader: DTS-52322: Miss redo log file
DTS-RETRY-ERR-0422: oracle nodes changes, restart it
Possible cause: The available nodes in the source Oracle RAC cluster have changed, disrupting the DTS connection.
Solution: Restart the DTS task. If the error persists, contact Alibaba Cloud technical support.
Example:
DTS-100047: retry 0 times, 1101 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: RecoverableAnyAllException: oracle-reader: DTS-52320: oracle nodes changes, restart it
DTS-RETRY-ERR-0423: Serializer redo log entry failed
Possible cause: DTS failed to parse the redo logs in the source Oracle database.
Solution:
-
Method 1: Contact the DBA of the source database to investigate and resolve the log issue, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 1001 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: CriticalAnyAllException: oracle-reader: DTS-52340: Serializer redo log entry failed ArrayIndexOutOfBoundsException: 1
DTS-RETRY-ERR-0436: Valid thread redo log file in rac cluster
Possible cause: The available nodes in the source Oracle RAC cluster have changed, making redo log files inaccessible.
Solution: Restart the DTS task. If the error persists, contact Alibaba Cloud technical support.
Example:
DTS-52111: Increment Context Is Not Running..: cause: CriticalAnyAllException: oracle-reader: DTS-52320: Fetch oracle log failed FatalAnyAllException: common: DTS-100047: retry 0 times, 4404 seconds, which exceed the supposed 43200 seconds CriticalAnyAllException: oracle-reader: DTS-52322: Valid thread redo log file in rac cluster
MySQL / PolarDB-X
DTS-RETRY-ERR-0437: OffsetOutOfRangeException: Offsets out of range with no configured reset policy for partitions
Possible cause: The current offset of the DTS task falls outside the available offset range in the incremental data collection module, typically because the corresponding data has already been overwritten.
Solution:
-
Method 1: Reconfigure the DTS task.
-
Method 2: Use the modify offset feature to reset the offset to within the available range.
Modifying the offset may result in data loss.
Example:
capture-dstore: DTS-51009: real poll source record failed
cause:
OffsetOutOfRangeException: Offsets out of range with no configured reset policy for partitions: {cn_g****_dts_version2-0=34****}
DTS-RETRY-ERR-0441: com.fasterxml.jackson.core.JsonParseException: Illegal unquoted character
Possible cause: The task configuration contains unescaped special characters. This commonly occurs in the DbList parameter when the DTS task is configured using OpenAPI.
Solution: Identify and fix the invalid characters based on the error message, and then restart the DTS task or use OpenAPI to start the DTS task.
Example:
DTS-100047: retry 0 times, 11009 seconds, which exceed the supposed 43200 seconds cause: RecoverableAnyAllException: dts-k-src: DTS-52110: MySQLRecordExtractor Init Error: null:reason=com.fasterxml.jackson.core.JsonParseException: Illegal unquoted character ((CTRL-CHAR, code 10)): has to be escaped using backslash to be included in name at [Source:****; line: 1, column: 11] ReaderException: null:reason=com.fasterxml.jackson.core.JsonParseException: Illegal unquoted character ((CTRL-CHAR, code 10)): has to be escaped using backslash to be included in name at [Source:**** line: 1, column: 11] JsonParseException: Illegal unquoted character ((CTRL-CHAR, code 10)): has to be escaped using backslash to be included in name at [Source:**** line: 1, column: 11]
DTS-RETRY-ERR-0442: mysql row image valid failed, miss
Possible cause: The Binlog settings in the source MySQL database are incorrect, causing DTS to fail to validate row images.
Solution: Check and correctly configure the Binlog settings in the source database, and then restart the DTS task.
Example:
DTS-100047: retry 0 times, 1000 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: mysql-reader: DTS-52230: mysql-reader: DTS-52230: mysql row image valid failed, miss 2, event: Header:****
DTS-RETRY-ERR-0456: unsupported encoding: utf8mb3
Possible cause: The current DTS task does not support the utf8mb3 character encoding used in the source database.
Solution: Contact the DBA of the source database to change the character encoding to a supported format, and then reconfigure the DTS task.
Example:
DTS-073602: Record Converter failed for the record of table drds.multi_****. cause by [Unsupported encoding: utf8mb3] About more information in ****
DTS-RETRY-ERR-0459: nvalid event type
Possible cause: The Binlog in the source database contains an abnormal event type that DTS cannot parse.
Solution:
-
Method 1: Contact the DBA of the source database to investigate and resolve the Binlog issue, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 1001 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: mysql-reader: DTS-52200: process error failed cause: IOException: java.sql.SQLException: nvalid event type [binlog-102****] SQLException: nvalid event type [binlog-102****] CriticalAnyAllException: mysql-reader: DTS-52200: process error failed IOException: java.sql.SQLException: nvalid event type [binlog-102****] SQLException: nvalid event type [binlog-102****]
DTS-RETRY-ERR-0460: valid binlog checksum
Possible cause: The Binlog in the source database has failed checksum validation, indicating data corruption.
Solution:
-
Method 1: Contact the DBA of the source database to investigate and resolve the Binlog issue, and then restart the DTS task.
-
Method 2: Reconfigure the DTS task.
Example:
DTS-100047: retry 0 times, 1001 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: dts-k-src: DTS-52111: Increment Context Is Not Running..: mysql-reader: DTS-52200: process error failed cause: IOException: reader: DTS-52232: Failed to valid binlog checksum. cause: ****
DTS-RETRY-ERR-0462: polardbx (.)? server error by
Possible cause: The source PolarDB-X instance is in an abnormal state and cannot serve requests.
Solution: Ensure that the PolarDB-X instance is running normally and that DTS can connect to and read data from it, and then restart the DTS task.
Example:
DTS-52110: MySQLRecordExtractor Init Error: mysql-reader: DTS--0001: execute sql show binary logs failed, case by common: DTS-100047: retry 0 times, 100 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: mysql-reader: DTS--0001: execute sql show binary logs failed, case by common: DTS-100047: retry 0 times, 100 seconds, which exceed the supposed 43200 seconds FatalAnyAllException: common: DTS-100047: retry 0 times, 100 seconds, which exceed the supposed 43200 seconds SQLException: ****[polardbx]ERR-CODE: [TDDL-4700][ERR_SERVER] server error by UNAVAILABLE: io exception
DTS-RETRY-ERR-0463: polardbx (.)? UNAVAILABLE
Possible cause: The source PolarDB-X instance is unavailable.
Solution: Ensure that the PolarDB-X instance is running normally and that DTS can connect to and read data from it, and then restart the DTS task.
Example:
DTS-52110: MySQLRecordExtractor Init Error: mysql-reader: DTS--0001: execute sql show binary logs failed, case by common: DTS-100047: retry 0 times, 0 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: mysql-reader: DTS--0001: execute sql show binary logs failed, case by common: DTS-100047: retry 0 times, 0 seconds, which exceed the supposed 43200 seconds FatalAnyAllException: common: DTS-100047: retry 0 times, 0 seconds, which exceed the supposed 43200 seconds SQLException: ****[polardbx]UNAVAILABLE: io exception
DTS-RETRY-ERR-0493: MySQLQueryInterruptedException: Query execution was interrupted
| Possible cause | Solution |
|---|---|
| DTS was executing a query in the database when it was interrupted, possibly due to an external kill command or a policy restriction. | Remove any restrictions that prevent DTS from executing queries in the database, and then restart the DTS task. |
| The database has insufficient remaining resources (such as memory or CPU), causing the DTS task to fail. | Upgrade the database specifications, and then restart the DTS task. |
Example:
DTS-31009: In process of processing data (recordRange: 123456) failed cause: FatalAnyAllException: common: DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds CriticalAnyAllException: framework: DTS-31009: read source data error MySQLQueryInterruptedException: Query execution was interrupted
DTS-RETRY-ERR-0495: sent packet length (.)? exceeds the allowed maximum
Possible cause: A single data packet written to the destination PolarDB-X instance exceeds the max_allowed_packet limit configured on the instance.
Solution: Increase the max_allowed_packet parameter value on the destination PolarDB-X instance to accommodate the largest expected packet size, and then restart the DTS task.
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: BatchUpdateException: **** sent packet length (71542218) exceeds the allowed maximum (67108864). SQLException: [19d663b3ba00****][10.88.XX.XX:XXX][sassrada]ERR-CODE: [TDDL-10000][ERR_X_PROTOCOL_BAD_PACKET] X-NIO-Client **** sent packet length (71542218) exceeds the allowed maximum (67108864).
Redis
DTS-RETRY-ERR-0450: wrong redis password
Possible cause: The Redis account or password configured in the DTS task is incorrect.
Solution: Reconfigure the DTS task with the correct Redis account and password.
Example:
DTS-077002: Record Replicator initialization error. cause by [wrong redis password.] About more information in ****
DTS-RETRY-ERR-0451: ERR bad lua script for redis cluster
Possible cause: The Lua script being synchronized to the destination Redis cluster does not meet the cluster's requirements for Lua scripts.
Solution: Modify the Lua script in the source database to comply with the Lua script specifications, or disable the script_check_enable parameter in the destination database. Then restart the DTS task or reconfigure the DTS task.
Example:
java.lang.RuntimeException: redis.clients.jedis.exceptions.JedisDataException: ERR bad lua script for redis cluster, first parameter of redis.call/redis.pcall must be a single literal string
DTS-RETRY-ERR-0452: JedisMovedDataException: MOVED
Possible cause: DTS cannot write data to the destination Redis instance, possibly due to a cluster slot migration or connectivity issue.
Solution: Ensure that DTS can write data to the destination Redis instance normally, and then restart the DTS task.
Example:
DTS-100047: retry 25 times, 1845 seconds, which exceed the supposed 1800 seconds cause: JedisMovedDataException: MOVED ****
DTS-RETRY-ERR-0453: seek by gbid failed
| Possible cause | Solution |
|---|---|
| The source database does not support breakpoint resumption. | Reconfigure the DTS task. |
| The DTS task failed for an extended period and the Binlog in the source database has been deleted. |
Example:
DTS-31009: In process of processing data (recordRange: 27873049260****) failed cause: FatalAnyAllException: common: DTS-100047: retry 60 times, 600073 seconds, which exceed the supposed 43200 seconds RedisServerException: redis: DTS-10015: redis: DTS-10017: Command item length not start with *, -ERR seek by gbid failed UnexpectedFormatException: redis: DTS-10017: Command item length not start with *, -ERR seek by gbid failed
DTS-RETRY-ERR-0454: Command item length not start with
Possible cause: The source database executed a non-standard or malformed Redis command that DTS cannot process.
Solution: Restart the DTS task. If the error persists, contact Alibaba Cloud technical support.
Example:
DTS-31009: In process of processing data (recordRange: 27873049260****) failed cause: RedisServerException: redis: DTS-10015: redis: DTS-10017: Command item length not start with *, 4 UnexpectedFormatException: redis: DTS-10017: Command item length not start with *, 4
DTS-RETRY-ERR-0457: unsupported RDB_TYPE_MODULE_
Possible cause: The source Redis database contains data in a module-specific RDB format that DTS does not support.
Solution: Reconfigure the DTS task and exclude objects that use the unsupported RDB format.
Example:
DTS-31009: In process of processing data (recordRange: 446037242650873****) failed cause: FatalAnyAllException: common: DTS-100047: retry 60 times, 606 seconds, which exceed the supposed 43200 seconds RedisServerException: redis: DTS-10015: redis: DTS-10019: [unsupported RDB_TYPE_MODULE_2][exhash---]key:****
DTS-RETRY-ERR-0458: redis aof is off
Possible cause: AOF (Append-Only File) persistence is not enabled in the source Redis instance, which is required for incremental data migration.
Solution: Manually enable AOF by setting the appendonly parameter to yes, and then restart the DTS task.
Example:
DTS-31009: In process of processing data (recordRange: 13936524630****) failed cause: FatalAnyAllException: common: DTS-100047: retry 60 times, 606 seconds, which exceed the supposed 43200 seconds RedisServerException: redis: DTS-10015: redis: DTS-10017: Command item length not start with *, -ERR redis aof is off UnexpectedFormatException: redis: DTS-10017: Command item length not start with *, -ERR redis aof is off
DTS-RETRY-ERR-0461: No enum constant com.alibaba.amp.any.redis.writer.ExtendRedisCommand
Possible cause: DTS encountered an unrecognized command type in the source Redis instance that it cannot parse.
Solution: Restart the DTS task. If the error persists, contact Alibaba Cloud technical support.
Example:
DTS-31009: In process of processing data (recordRange: 657948551556366****) failed cause: FatalAnyAllException: common: DTS-100047: retry 0 times, 2392099 seconds, which exceed the supposed 43200 seconds IllegalArgumentException: No enum constant com.alibaba.amp.any.redis.writer.ExtendRedisCommand.
DTS-RETRY-ERR-0464: redis temporary failure
Possible cause: The destination Redis instance is in an abnormal state and is returning temporary failure errors.
Solution: Contact the DBA of the destination database to restore normal Redis operation, and then restart the DTS task.
Example:
DTS-100047: retry 60 times, 600 seconds, which exceed the supposed 43200 seconds cause: RedisCannotRetryException: redis: DTS-10009: Exception in stage: [begin], errMsg: ERR redis temporary failure (ErrorCode 7002) JedisDataException: ERR redis temporary failure (ErrorCode 7002)
DTS-RETRY-ERR-0465: server closed connection
Possible cause: The destination Redis instance closed the connection unexpectedly.
Solution: Contact the DBA of the destination database to investigate and resolve the issue, and then restart the DTS task.
Example:
DTS-31009: In process of processing data (recordRange: 27873049260****) failed cause: FatalAnyAllException: common: DTS-100047: retry 60 times, 606 seconds, which exceed the supposed 43200 seconds RedisServerException: redis: DTS-10015: redis: DTS-10017: Command item length not start with *, -ERR server closed connection UnexpectedFormatException: redis: DTS-10017: Command item length not start with *, -ERR server closed connection
DTS-RETRY-ERR-0466: redis (.)? Connection timed out
Possible cause: The connection to the destination Redis instance timed out, indicating the instance is in an abnormal state or network issues exist.
Solution: Contact the DBA of the destination database to restore connectivity, and then restart the DTS task.
Example:
DTS-31009: In process of processing data (recordRange: 27873049260****) failed cause: FatalAnyAllException: common: DTS-100047: retry 60 times, 8141 seconds, which exceed the supposed 43200 seconds RedisClientInitiationException: redis: DTS-10010: null ConnectException: Connection timed out (Connection timed out)
DTS-RETRY-ERR-0467: JedisDataException: ERR unknown command
Possible cause: A command compatibility issue exists between the source and destination Redis instances due to version differences. For example, the source instance uses the SET PXAT command, which is supported only in Redis 6.2 and above, but the destination instance runs an earlier version.
Solution: Contact the DBA of the destination Redis instance to upgrade the instance version to one that supports the commands used by the source, and then restart the DTS task.
Example:
DTS-100047: retry 606 times, 43244 seconds, which exceed the supposed 43200 seconds cause: JedisDataException: ERR unknown command 'BF.LOADCHUNK'
DTS-RETRY-ERR-0468: JedisDataException: ERR syntax error
Possible cause: A command compatibility issue exists between the source and destination Redis instances due to version differences. For example, the source instance uses the SET PXAT command, which is supported only in Redis 6.2 and above, but the destination instance runs an earlier version.
Solution: Contact the DBA of the destination Redis instance to upgrade the instance version to one that supports the commands used by the source, and then restart the DTS task.
Example:
DTS-100047: retry 616 times, 43210 seconds, which exceed the supposed 43200 seconds cause: JedisDataException: ERR syntax error
DTS-RETRY-ERR-0470: ERR binlog-get need full sync for client
Possible cause: The DTS task is performing incremental synchronization only (without full synchronization or migration), but the append-only file (AOF) in the source Redis instance has been cleared, making incremental resumption impossible.
Solution: Restart the DTS task. If the error persists, reconfigure the DTS task.
Example:
DTS-31009: In process of processing data (recordRange: 359634877462609****) failed cause: FatalAnyAllException: common: DTS-100047: retry 60 times, 606 seconds, which exceed the supposed 43200 seconds RedisServerException: redis: DTS-10015: redis: DTS-10009: redis: DTS-10013: -ERR binlog-get need full sync for client(****) RedisCannotRetryException: redis: DTS-10009: redis: DTS-10013: -ERR binlog-get need full sync for client(****)) RedisReFullSyncException: redis: DTS-10013: -ERR binlog-get need full sync for client****
DTS-RETRY-ERR-0455: binlog-get client request seq (.)? bigger than
Possible cause: The DTS service has encountered an internal exception that caused the request sequence number to exceed the server sequence number.
Solution: Restart the DTS task. If the error persists, contact Alibaba Cloud technical support.
Example:
DTS-31009: In process of processing data (recordRange: 712574516379569****) failed cause: FatalAnyAllException: common: DTS-100047: retry 60 times, 606 seconds, which exceed the supposed 1800 seconds RedisServerException: redis: DTS-10015: redis: DTS-10009: redis: DTS-10013: -ERR binlog-get client request seq 18960**** bigger than server seq 163**** RedisCannotRetryException: redis: DTS-10009: redis: DTS-10013: -ERR binlog-get client request seq 18960**** bigger than server seq 163**** RedisReFullSyncException: redis: DTS-10013: -ERR binlog-get client request seq 18960**** bigger than server seq 163****
ClickHouse
DTS-RETRY-ERR-0438: DISABLE_WRITE
Possible cause: Write operations to the destination ClickHouse database are disabled, either because the instance is in an abnormal state or write operations have been explicitly restricted.
Solution: Contact the DBA of the destination database to enable write operations, and then restart the DTS task.
Example:
DTS-100047: retry 3438 times, 43207 seconds, which exceed the supposed 43200 seconds cause: TransactionReplicateException: transaction-replicate: DTS-70003: 2 execute transaction has excess max transaction retry time [64] cause:transaction-replicate: DTS-70004: EXEC statement failed, may try it again RecoverableAnyAllException: transaction-replicate: DTS-70004: EXEC statement failed, may try it again SQLException: ClickHouseEngine(2): meet error, insertSQL:insert into ****, error detail message:Code: 2002. DB::Exception: The write operations on the database instance is disabled. (DISABLE_WRITE) (version 23.8.16.1) , server ClickHouseNode [uri=http:****]@-1110610050
DTS-RETRY-ERR-0444: ClickHouse error null returned from null
Possible cause: DTS cannot connect to the destination ClickHouse instance.
Solution: Ensure that DTS can connect to the destination ClickHouse instance normally, and then restart the DTS task.
Example:
java.sql.SQLException: HTTP response -1 null (ClickHouse error null returned from null), server ClickHouseNode ****
DTS-RETRY-ERR-0448: NO_SUCH_COLUMN_IN_TABLE
Possible cause: The table structure in the destination database is inconsistent with the source. The destination table is missing one or more columns.
Solution: Based on the error message, manually add the missing columns to the corresponding table in the destination database, and then restart the DTS task.
Example:
DTS-100047: retry 3336 times, 43211 seconds, which exceed the supposed 43200 seconds cause: TransactionReplicateException: transaction-replicate: DTS-70003: 2 execute transaction has excess max transaction retry time [64] cause:transaction-replicate: DTS-70004: EXEC statement failed, may try it again RecoverableAnyAllException: transaction-replicate: DTS-70004: EXEC statement failed, may try it again SQLException: ClickHouseEngine(2): meet error, insertSQL:insert into **** SQLException: Code: 16. DB::Exception: No such column newest_che**** in table ****
MongoDB
DTS-RETRY-ERR-0473: MongoNotPrimaryException
Possible cause: The destination MongoDB node is not a Primary node. Write operations can only be performed on the Primary node.
Solution: Reconfigure the DTS task to use the Primary node or the high availability address of the MongoDB instance as the destination database.
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: SQLException: com.mongodb.MongoNotPrimaryException: Command failed with error 10107 (NotMaster): **** MongoNotPrimaryException: Command failed with error 10107 (NotMaster): 'not master' on server 218.17.XX.XX:XXX. The full response is ****
DTS-RETRY-ERR-0474: Connection timed out
Possible cause: The network connection between DTS and the database timed out.
Solution: Ensure that the database is running normally and that DTS can connect to it. Check the connection settings, whitelist configuration, and account credentials. Then restart the DTS task.
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: SQLException: com.mongodb.MongoSocketOpenException: Exception opening socket MongoSocketOpenException: Exception opening socket ConnectException: Connection timed out (Connection timed out)
DTS-RETRY-ERR-0475: key too large to index
Possible cause: A document being written to the destination MongoDB instance contains an index key that exceeds the size limit.
Solution: Modify or remove the index size restriction in the destination database, and then restart the DTS task.
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: SQLException: com.mongodb.MongoWriteException: Write operation error on server 10.30.XX.XX:XXX. Write error: WriteError{code=17280, message='WiredTigerIndex::insert: key too large to index, failing 4789 ****, details={}}. MongoWriteException: Write operation error on server 10.30.201.77:13960. Write error: WriteError{code=17280, message='WiredTigerIndex::insert: key too large to index, failing 4789 ****
DTS-RETRY-ERR-0476: Timed out after 3600000 ms while waiting to connect
Possible cause: The network connection between DTS and the database timed out after 3,600,000 ms.
Solution: Ensure that the database is running normally and that DTS can connect to it. Check the connection settings, whitelist configuration, and account credentials. Then restart the DTS task.
Example:
DTS-31007: fetch and merge schemas failed cause: RuntimeException: java.sql.SQLException: com.mongodb.MongoTimeoutException: Timed out after 3600000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=10.45.XX.XX:XXX, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoCommandException: Command failed with error 96 (OperationFailed): 'fail to satisfy horizon { horizonName: [ "public" ] }' on server 10.45.XX.XX:XXX. ...}}]}
DTS-RETRY-ERR-0477: operation was interrupted
| Possible cause | Solution |
|---|---|
| DTS was executing a command in the database when the operation was interrupted, possibly due to a policy or external kill. | Remove any restrictions that prevent DTS from executing commands in the database, and then restart the DTS task. |
| The database has insufficient remaining resources (such as memory or CPU). | Upgrade the database specifications, and then restart the DTS task. |
Example:
DTS-31009: In process of processing data (recordRange: 123456) failed cause: FatalAnyAllException: common: DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds RuntimeException: com.mongodb.MongoQueryException: Query failed with error code 11602 and error message 'operation was interrupted' on server 10.48.XX.XX:XXX MongoQueryException: Query failed with error code 11602 and error message 'operation was interrupted' on server 10.48.XX.XX:XXX
DTS-RETRY-ERR-0478: Couldn't get a connection within the time limit
Possible cause: The network connection between DTS and the database timed out.
Solution: Ensure that the database is running normally and that DTS can connect to it. Check the connection settings, whitelist configuration, and account credentials. Then restart the DTS task.
Example:
DTS-31009: In process of processing data (recordRange: 123456) failed cause: FatalAnyAllException: common: DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds RuntimeException: com.mongodb.MongoQueryException: Query failed with error code 202 and error message 'Couldn't get a connection within the time limit' on server 10.48.XX.XX:XXX MongoQueryException: Query failed with error code 202 and error message 'Couldn't get a connection within the time limit' on server 10.48.XX.XX:XXX
DTS-RETRY-ERR-0479: Connection was closed
| Possible cause | Solution |
|---|---|
| The connection between DTS and the database was interrupted, possibly due to a policy restriction. | Remove any restrictions that prevent DTS from connecting to the database, ensure that DTS can connect normally, and then restart the DTS task. |
| The database has insufficient remaining resources. | Upgrade the database specifications, and then restart the DTS task. |
Example:
DTS-31009: In process of processing data (recordRange: 123456) failed cause: FatalAnyAllException: common: DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds RuntimeException: com.mongodb.MongoQueryException: Query failed with error code 6 and error message 'Error connecting to 192.168.XX.XX:XXX :: caused by :: Connection was closed' on server 10.48.XX.XX:XXX MongoQueryException: Query failed with error code 6 and error message 'Error connecting to 192.168.XX.XX:XXX :: caused by :: Connection was closed' on server 10.48.XX.XX:XXX
DTS-RETRY-ERR-0480: stream truncated
| Possible cause | Solution |
|---|---|
| The connection between DTS and the database was interrupted. | Remove any restrictions that prevent DTS from connecting to the database, ensure that DTS can connect normally, and then restart the DTS task. |
| The database has insufficient remaining resources. | Upgrade the database specifications, and then restart the DTS task. |
Example:
DTS-31009: In process of processing data (recordRange: 123456) failed cause: FatalAnyAllException: common: DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds RuntimeException: com.mongodb.MongoQueryException: Query failed with error code 9001 and error message 'stream truncated' on server 10.48.XX.XX:XXX MongoQueryException: Query failed with error code 9001 and error message 'stream truncated' on server 10.48.XX.XX:XXX
AnalyticDB MySQL
DTS-RETRY-ERR-0447: ResourceNotFoundException
Possible cause: The destination table or project does not exist in the destination database.
Solution: Based on the error message, manually create the missing table or project in the destination database with the same or compatible structure as the source, and then restart the DTS task.
Example:
DTS-31007: get table **** detail info failed after retrying. cause: FatalAnyAllException: common: DTS-100047: retry 4233 times, 43203 seconds, which exceed the supposed 43200 seconds RecoverableAnyAllException: replicate-datahub: DTS-70002: get table **** detail info failed. ResourceNotFoundException: [httpStatus:404, requestId:202412310740249ed53a0a51e2**** errorCode:NoSuchProject, errorMessage:The specified project does not exist.]
DTS-RETRY-ERR-0469: STATUS_READ_ONLY
| Possible cause | Solution |
|---|---|
| The destination database is in read-only mode. | Set the destination database to read-write mode, and then restart the DTS task. |
| The destination database is a read-only replica. | Use a read-write destination database and reconfigure the DTS task. |
Example:
Record Replicator error in table **** by [java.sql.SQLException: ****[cz_delivery]ERR-CODE: [TDDL-4633][ERR_DB_STATUS_READ_ONLY] Database is read-only, only read sql are supported. ] About more information in ****
DTS-RETRY-ERR-0471: multi-statement is disabled
Possible cause: The Multi-Statement feature is not enabled in the destination AnalyticDB MySQL 3.0 cluster, which DTS requires to execute batch SQL operations.
Solution: Run the following command in the destination AnalyticDB MySQL 3.0 cluster to enable the Multi-Statement feature, and then restart the DTS task.
SET ADB_CONFIG ALLOW_MULTI_QUERIES=true;
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: RuntimeException: Table **** with columns **** and batch exception is {[9001, 202504190815280210170210580315106****] multi-statement is disabled} SQLException: [9001, 202504190815280210170210580315106****] multi-statement is disabled
DTS-RETRY-ERR-0472: ColumnNotFoundException
Possible cause: The table structure in the destination database is inconsistent with the source. The destination table is missing one or more columns.
Solution: Based on the error message, manually add the missing columns to the corresponding table in the destination database, and then restart the DTS task.
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: LindormException: Failed doing UPSERT on ****, caused by com.alibaba.lindorm.client.exception.ColumnNotFoundException: Column **** not found for table ****
Elasticsearch
DTS-RETRY-ERR-0483: Document contains at least one immense term
Possible cause: A field value in the data being written to Elasticsearch exceeds the maximum term size limit of 32,766 bytes (UTF-8 encoding).
Solution:
-
Method 1: Based on the error message, modify the analyzer configuration of the destination Elasticsearch instance to prevent terms exceeding the size limit, and then restart the DTS task.
-
Method 2:
-
Synchronization task: Use the modify synchronization objects feature to remove the objects causing the error, and then restart the DTS task.
-
Migration task: Reconfigure the migration task and use the correct data processing statement.
-
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: IOException: es: DTS-10035: Document contains at least one immense term in field="****" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped.
DTS-RETRY-ERR-0484: Illegal pattern character
Possible cause: The data being written to Elasticsearch does not match the expected format, causing a parsing failure.
Solution:
-
Method 1: Based on the error message and the table structure of the source database, update the field mapping in the destination Elasticsearch instance to accommodate the data, and then restart the DTS task.
-
Method 2:
-
Synchronization task: Use the modify synchronization objects feature to remove the objects causing the error, and then restart the DTS task.
-
Migration task: Reconfigure the migration task and use the correct data processing statement.
-
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: IllegalArgumentException: Illegal pattern character 't'
DTS-RETRY-ERR-0485: failed to parse
Possible cause: The data being written to Elasticsearch does not match the expected field type, causing a parsing failure.
Solution:
-
Method 1: Based on the error message and the table structure of the source database, update the field mapping in the destination Elasticsearch instance to accommodate the data, and then restart the DTS task.
-
Method 2:
-
Synchronization task: Use the modify synchronization objects feature to remove the objects causing the error, and then restart the DTS task.
-
Migration task: Reconfigure the migration task and use the correct data processing statement.
-
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: IOException: es: DTS-10035: [1:39] failed to parse field [topic_id] of type [integer] in document with id '2'. Preview of field's value: 'topic_1', For input string: "topic_1"
DTS-RETRY-ERR-0486: out of range of
Possible cause: A numeric field value in the data being written to Elasticsearch exceeds the valid range for the configured field type (for example, a value that overflows an integer field).
Solution:
-
Method 1: Based on the error message and the table structure of the source database, update the field mapping in the destination Elasticsearch instance (for example, change the field type to
long) to accommodate the data, and then restart the DTS task. -
Method 2:
-
Synchronization task: Use the modify synchronization objects feature to remove the objects causing the error, and then restart the DTS task.
-
Migration task: Reconfigure the migration task and use the correct data processing statement.
-
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: IOException: es: DTS-10035: [1:7] failed to parse field [id] of type [integer] in document with id '190480000000000****'. Preview of field's value: '190480000000000****', [1:26] Numeric value (190480000000000****) out of range of int (-2147483648 - 2147483647)
DTS-RETRY-ERR-0487: rejected execution of processing
Possible cause: The write thread pool queue in the destination Elasticsearch instance is full, causing new write requests to be rejected.
Solution: Ensure that the destination Elasticsearch instance is running normally and that DTS can connect to it, and then restart the DTS task.
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: IOException: es: DTS-10035: rejected execution of processing of [15623****][indices:data/write/bulk[s][p]]: request: BulkShardRequest [[mysql_binlog_wxwork_message_****_dts][0]] containing [660] requests, ...on EsThreadPoolExecutor[name = XOh****/write, queue capacity = 200,...]
DTS-RETRY-ERR-0488: listener timeout after waiting
Possible cause: The network connection between DTS and the database timed out while waiting for a response.
Solution: Ensure that the database is running normally and that DTS can connect to it. Check the connection settings, whitelist configuration, and account credentials. Then restart the DTS task.
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: IOException: java.io.IOException: listener timeout after waiting for [30000] ms IOException: listener timeout after waiting for [30000] ms
DTS-RETRY-ERR-0489: Failed to parse
Possible cause: The data being written to Elasticsearch does not match the expected field type or structure, causing a parsing failure.
Solution:
-
Method 1: Based on the error message and the table structure of the source database, update the field mapping in the destination Elasticsearch instance to accommodate the data, and then restart the DTS task.
-
Method 2:
-
Synchronization task: Use the modify synchronization objects feature to remove the objects causing the error, and then restart the DTS task.
-
Migration task: Reconfigure the migration task and use the correct data processing statement.
-
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: IOException: org.elasticsearch.client.ResponseException: POST **** Request { "error" : { "root_cause" : [ { "type" : "parsing_exception", "reason" : "Failed to parse object: expecting token of type [VALUE_NUMBER] but found [END_OBJECT]" ...} ], "type" : "mapper_parsing_exception", "reason" : "failed to parse" }, "status" : 400 }
Function Compute
DTS-RETRY-ERR-0490: write record to fc failed
Possible cause: DTS cannot write data to the destination Function Compute (FC) service.
Solution: Based on the error messages from both DTS and FC, identify and resolve the issue, and then restart the DTS task.
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: fc: DTS-66805: write record to fc failed
Greenplum / PostgreSQL
DTS-RETRY-ERR-0491: cancelled by global deadlock detector
Possible cause: A deadlock occurred while DTS was writing data to the destination database. This can happen when other applications or processes are concurrently writing to the same tables.
Solution: Ensure that no other write operations are causing deadlocks in the destination database, and then restart the DTS task.
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: full-greenplum: DTS-65708: copy error, ...PSQLException: ERROR: canceling statement due to user request: "cancelled by global deadlock detector"
DTS-RETRY-ERR-0492: tuple concurrently updated
Possible cause: A concurrent write conflict occurred: another transaction updated the same row at the same time as DTS, causing DTS to fail.
Solution: Ensure that no other write operations are running concurrently in the destination database, and then restart the DTS task.
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: full-greenplum: DTS-65708: copy error, ...PSQLException: ERROR: tuple concurrently updated (heapam.c:5053)
DTS-RETRY-ERR-0499: failed to set the requested LC_COLLATE value
Possible cause: A character set collation compatibility issue exists between the destination PolarDB for PostgreSQL (Compatible with Oracle) cluster and the source Oracle database.
Solution: Contact Alibaba Cloud technical support for the PolarDB for PostgreSQL (Compatible with Oracle) cluster for assistance.
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: framework: DTS-30011: currentRunningSQL: ...PSQLException: ERROR: failed to set the requested LC_COLLATE value ['****']
DTS-RETRY-ERR-0500: invalid input syntax for type timestamp
Possible cause: The ETL data processing statement is incorrect, causing the ETL execution to fail when processing timestamp fields.
Solution:
-
Synchronization task: Based on the error message, check and correct the data processing statement, and then restart the DTS task.
-
Migration task: Reconfigure the migration task with the correct data processing statement.
Example:
DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds cause: CriticalAnyAllException: full-postgresql: DTS-65708: copy error, ...PSQLException: ERROR: invalid input syntax for type timestamp: "2:22:38" Where: COPY acc_syncattendance, line 1, column synctime: "2:22:38"
OceanBase / Doris
DTS-RETRY-ERR-0482: Timeout, query has reached the maximum query timeout (.)? maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again.
Possible cause: DTS timed out while executing a query operation in the database.
Solution: Increase the ob_query_timeout or query_timeout parameter value in the database to extend the query timeout, and then restart the DTS task.
Example:
DTS-31009: In process of processing data (recordRange: 123456) failed cause: FatalAnyAllException: common: DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds CriticalAnyAllException: framework: DTS-31009: read source data error SQLException: Timeout, query has reached the maximum query timeout: 1800000000(us), maybe you can adjust the session variable ob_query_timeout or query_timeout hint, and try again.
DTS-RETRY-ERR-0494: query is timeout, killed by timeout checker
Possible cause: DTS timed out while executing a query operation in the source database, and the query was killed by the database's timeout checker.
Solution: Increase the SQL query timeout setting in the source database, and then restart the DTS task.
Example:
DTS-31009: In process of processing data (recordRange: 123456) failed cause: FatalAnyAllException: common: DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds CriticalAnyAllException: framework: DTS-31009: read source data error SQLException: errCode = 2, detailMessage = query is timeout, killed by timeout checker
MaxCompute
DTS-RETRY-ERR-0496: split number should
Possible cause: The count value of a table in the MaxCompute instance returns a negative number, causing the task split calculation to fail.
Solution: Contact the DBA of the destination database to resolve the negative count value, and then restart the DTS task.
Example:
DTS-31004: split number should > 1 expectSliceNumber=[-279285].
DTS-RETRY-ERR-0497: Table not found
Possible cause: The destination table does not exist in the MaxCompute project.
Solution: Based on the error message, manually create the missing table in the destination MaxCompute project with the same or compatible structure as the source, and then restart the DTS task.
Example:
DTS-31007: fetch and merge schemas failed cause: RuntimeException: com.aliyun.odps.ReloadException: ODPS-0130131:Table not found - **** table not found ReloadException: ODPS-0130131:Table not found - **** table not found NoSuchObjectException: ODPS-0130131:Table not found - ****
DTS-RETRY-ERR-0498: The download session is expired and timed out. Too much time elapsed since it was created.
Possible cause: The destination MaxCompute download session expired before the operation completed.
Solution: Ensure that the destination MaxCompute is operating normally and can be connected properly, and then restart the DTS task.
Example:
DTS-31009: In process of processing data (recordRange: 123456) failed cause: CriticalAnyAllException: odps: DTS-11014: odps getRecordReader error,**** FatalAnyAllException: common: DTS-100047: retry 1 times, 43200 seconds, which exceed the supposed 43200 seconds TunnelException: RequestId=20250306031732d1e1db0b047****, ErrorCode=StatusConflict, ErrorMessage=The download session is expired.Session timed out. Too much time elapsed since it was created.
General
DTS-RETRY-ERR-0417: Not support db empty
Possible cause: The DTS service encountered an intermittent exception.
Solution: Retry or restart the DTS task. If the error persists, contact Alibaba Cloud technical support.
Example:
DTS-100047: retry 0 times, 11108 seconds, which exceed the supposed 43200 seconds cause: RuntimeException: Not support db empty.