Considerations and limits when synchronizing from a MySQL source database

Type

Description

Source database limits

• The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

• If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

• Binary logs:

  • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

    Important

    • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

    • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

  • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

    Note

    To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

• Do not run DDL operations that modify primary keys or add comments—such as ALTER TABLE table_name COMMENT='Table comment';—while synchronization is running. Otherwise, the DDL operation fails.

• Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

  Note

  During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

• Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

  Note

  If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

• If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

  Note

  Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

• MySQL compatibility: When DTS synchronizes data from a MySQL-family source database, it relies on the standard MySQL protocol and Binlog format, and behaviors incompatible with standard MySQL are not supported. If your source database claims MySQL compatibility but behaves differently from MySQL (for example, when OceanBase is connected as a source in MySQL mode, the timestamp of the trailing rotate event in the Binlog is 0, which differs from MySQL), the DTS synchronization task may fail.

Other limits

• DTS does not support synchronizing prefix indexes. If your source database contains prefix indexes, synchronization may fail.

• If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

• DTS does not support synchronizing indexes, partitions, views, stored procedures, functions, triggers, or foreign keys.

• If a primary key or unique key conflict occurs while the task is running:

  • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

    • During full data synchronization, DTS retains the destination record and skips the source record.

    • During incremental synchronization, DTS overwrites the destination record with the source record.

  • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

• The destination database must contain a custom primary key. Or, in the Configurations for Databases, Tables, and Columns step, configure a Primary Key Column. Otherwise, synchronization may fail.

• If you synchronize to AnalyticDB for MySQL (2.0), the source database objects must not contain AnalyticDB for MySQL (2.0) Reserved words (Version 2.0). Otherwise, synchronization or DDL synchronization may fail.

• Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

• Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

• If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

  You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

• For table-level synchronization, if no data other than DTS writes to the destination AnalyticDB for MySQL, you can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

• Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

• If a DDL operation fails in the destination database, the DTS task continues to run. Check the failed DDL in the task logs. For instructions, see View task logs.

• If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

  Note

  ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

• If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

  Note

  Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

Special cases

• For a self-managed MySQL source database:

  • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

  • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

    Note

    If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

  • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

• For an ApsaraDB RDS for MySQL source database:

  • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

Type

Description

Source database limits

• The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

• If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

• Binary logs:

  • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

    Important

    • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

    • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

  • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

    Note

    To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

• Do not run DDL operations that modify primary keys or add comments—such as ALTER TABLE table_name COMMENT='Table comment';—while synchronization is running. Otherwise, the DDL operation fails.

• Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

  Note

  During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

• Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

  Note

  If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

• If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

  Note

  Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

Other limits

• DTS does not support synchronizing prefix indexes. If your source database contains prefix indexes, synchronization may fail.

• If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

• DTS does not support synchronizing indexes, partitions, views, stored procedures, functions, triggers, or foreign keys.

• If a primary key or unique key conflict occurs while the task is running:

  • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

    • During full data synchronization, DTS retains the destination record and skips the source record.

    • During incremental synchronization, DTS overwrites the destination record with the source record.

  • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

• The destination database must contain a custom primary key. Or, in the Configurations for Databases, Tables, and Columns step, configure a Primary Key Column. Otherwise, synchronization may fail.

• Due to usage limits in AnalyticDB for MySQL(3.0), if disk usage exceeds 80% in an AnalyticDB for MySQL(3.0) cluster, DTS tasks may experience delays or errors. Estimate required space based on your synchronization objects and ensure sufficient storage in the destination cluster.

• If the destination AnalyticDB for MySQL 3.0 cluster is backing up while the DTS task runs, the task fails.

• Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

• Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

• If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

  You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

• For table-level synchronization, if no data other than DTS writes to the destination AnalyticDB for MySQL(3.0), you can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

• Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

• If a DDL operation fails in the destination database, the DTS task continues to run. Check the failed DDL in the task logs. For instructions, see View task logs.

• If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

  Note

  ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

• If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

  Note

  Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

Special cases

• For a self-managed MySQL source database:

  • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

  • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

    Note

    If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

  • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

• For an ApsaraDB RDS for MySQL source database:

  • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

Type

Description

Source database limits

• The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

• If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

• Binary logs:

  • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

    Important

    • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

    • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

  • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

    Note

    To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

• Do not run DDL operations that modify primary keys or add comments—such as ALTER TABLE table_name COMMENT='Table comment';—while synchronization is running. Otherwise, the DDL operation fails.

• If your source database contains date values of 0000-00-00 00:00:00, the task may fail.

  Note

  DTS converts this date to null in the destination database. Temporarily change the source data to 0001-01-01 00:00:00 or set the corresponding destination column to allow null values.

• Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

  Note

  During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

• Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

  Note

  If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

• If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

  Note

  Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

• MySQL compatibility: When DTS synchronizes data from a MySQL-family source database, it relies on the standard MySQL protocol and Binlog format, and behaviors incompatible with standard MySQL are not supported. If your source database claims MySQL compatibility but behaves differently from MySQL (for example, when OceanBase is connected as a source in MySQL mode, the timestamp of the trailing rotate event in the Binlog is 0, which differs from MySQL), the DTS synchronization task may fail.

Other limits

• Synchronization object requirements:

  • Only table-level synchronization is supported.

  • DTS does not support synchronizing VARBIT, GEOMETRY, ARRAY, UUID, TSQUERY, TSVECTOR, TXID_SNAPSHOT, or POINT data types.

  • DTS does not support synchronizing prefix indexes. If your source database contains prefix indexes, synchronization may fail.

  • DTS does not support synchronizing partitions, views, stored procedures, functions, triggers, foreign keys, or indexes.

• If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

• If a primary key or unique key conflict occurs while the task is running:

  • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

    • During full data synchronization, DTS retains the destination record and skips the source record.

    • During incremental synchronization, DTS overwrites the destination record with the source record.

  • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

• If the table to synchronize has a primary key, the primary key column in the destination table must match the source table. If the table to synchronize lacks a primary key, the primary key column in the destination table must match the distribution key.

• The unique key in the destination table—including the primary key column—must include all columns in the distribution key.

• Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

• Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

• If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

  You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

• Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

• The destination table does not support append-optimized (AO) tables.

• If you use column mapping and do not synchronize full tables—or if the source and destination table schemas do not match—data in columns missing from the destination side is lost.

• If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

  Note

  ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

• If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

  Note

  Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

Special cases

• For a self-managed MySQL source database:

  • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

  • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

    Note

    If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

  • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

• For an ApsaraDB RDS for MySQL source database:

  • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

Type

Description

Source database limits

• The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

• If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

• Binary logs:

  • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

    Important

    • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

    • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

  • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

    Note

    To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

• Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

  Note

  During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

• Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

  Note

  If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

• If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

  Note

  Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

• MySQL compatibility: When DTS synchronizes data from a MySQL-family source database, it relies on the standard MySQL protocol and Binlog format, and behaviors incompatible with standard MySQL are not supported. If your source database claims MySQL compatibility but behaves differently from MySQL (for example, when OceanBase is connected as a source in MySQL mode, the timestamp of the trailing rotate event in the Binlog is 0, which differs from MySQL), the DTS synchronization task may fail.

Other limits

• A single String field in DataHub supports up to 2 MB.

• DTS does not support synchronizing indexes, partitions, views, stored procedures, functions, triggers, or foreign keys.

• If your source table contains a record_id column, use field mapping to rename it in the destination instance. Otherwise, the task fails. For instructions, see Database and table column name mapping.

• Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

• If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

  Note

  ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

• If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

  Note

  Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

Special cases

• For a self-managed MySQL source database:

  • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

  • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

    Note

    If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

  • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

• For an ApsaraDB RDS for MySQL source database:

  • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

Type

Description

Source database limits

• The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

• If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

• Binary logs:

  • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

    Important

    • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

    • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

  • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

    Note

    To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

• Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

  Note

  During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

• Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

  Note

  If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

• If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

  Note

  Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

• MySQL compatibility: When DTS synchronizes data from a MySQL-family source database, it relies on the standard MySQL protocol and Binlog format, and behaviors incompatible with standard MySQL are not supported. If your source database claims MySQL compatibility but behaves differently from MySQL (for example, when OceanBase is connected as a source in MySQL mode, the timestamp of the trailing rotate event in the Binlog is 0, which differs from MySQL), the DTS synchronization task may fail.

Other limits

• DTS does not support synchronizing indexes, partitions, views, stored procedures, functions, triggers, or foreign keys.

• You cannot synchronize data to Elasticsearch indexes that contain parent-child relationships or Join field types. Doing so may cause task errors or query failures in the destination.

• Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

• Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

• If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

• If your source table needs to add a column, first modify the corresponding mapping in the Elasticsearch instance. Then run the DDL operation in the source MySQL database. Finally, pause and restart the DTS synchronization task.

• Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

• If data synchronized from MySQL to Elasticsearch contains null characters, DTS converts them to LONG type values. This causes the task to fail.

• If data synchronized from MySQL to Elasticsearch contains location information with reversed latitude and longitude values, writing to Elasticsearch fails.

• Development and test specifications of Elasticsearch instances are not supported.

• If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

  Note

  ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

• If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

  Note

  Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

Special cases

• For a self-managed MySQL source database:

  • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

  • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

    Note

    If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

  • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

• For an ApsaraDB RDS for MySQL source database:

  • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

Type

Description

Source database limits

• The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

• If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

• Binary logs:

  • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

    Important

    • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

    • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

  • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

    Note

    To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

• Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

  Note

  During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

• Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

  Note

  If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

• If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

  Note

  Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

• MySQL compatibility: When DTS synchronizes data from a MySQL-family source database, it relies on the standard MySQL protocol and Binlog format, and behaviors incompatible with standard MySQL are not supported. If your source database claims MySQL compatibility but behaves differently from MySQL (for example, when OceanBase is connected as a source in MySQL mode, the timestamp of the trailing rotate event in the Binlog is 0, which differs from MySQL), the DTS synchronization task may fail.

Other limits

• Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

• Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

• If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

  You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

• DTS does not support synchronizing indexes, partitions, views, stored procedures, functions, triggers, or foreign keys.

• MaxCompute does not support primary key constraints. If DTS retries due to network issues or other reasons, duplicate records may appear in MaxCompute.

• Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

• If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

  Note

  ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

• If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

  Note

  Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

Special cases

• For a self-managed MySQL source database:

  • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

  • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

    Note

    If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

  • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

• For an ApsaraDB RDS for MySQL source database:

  • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

Type

Description

Source database limits

• Tables without a primary key cannot be synchronized.

• If you synchronize data at the table level and need to edit objects, such as mapping table or column names, a single data synchronization task supports a maximum of 1,000 tables. If you exceed this limit, an error is reported after you submit the task. In this case, split the tables into multiple tasks or configure a task to synchronize the entire database.

• Binary logs:

  • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

    Important

    • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

    • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

  • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

    Note

    To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

• Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

  Note

  During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

• Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

  Note

  If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

• If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

  Note

  Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

Other limits

• f the DDL statements of the source RDS MySQL instance do not follow standard MySQL syntax, the synchronization task may fail or data may be lost.

• If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

• When you use DMS or the gh-ost tool to perform online DDL operations on the source, DTS synchronizes only the original DDL statements to the destination. In this scenario, DTS does not need to synchronize a large amount of temporary table data, but this may cause tables to be locked at the destination.

  Note

  DTS does not support synchronizing online DDL changes made with tools like pt-online-schema-change. If such changes exist at the source, data may be lost at the destination, or the synchronization instance may fail.

• Synchronization of INDEX, PARTITION, VIEW, PROCEDURE, FUNCTION, TRIGGER, and FK is supported.

• The RENAME TABLE operation is not supported.

• If a primary key or unique key conflict occurs while the task is running:

  • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

    • During full data synchronization, DTS retains the destination record and skips the source record.

    • During incremental synchronization, DTS overwrites the destination record with the source record.

  • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

• Time-type data in ApsaraDB for ClickHouse has range limitations. If the time data in RDS MySQL is outside this range, the time synchronized to ApsaraDB for ClickHouse will be incorrect. For information about the range limitations, see Time information.

• The Partition Key cannot be a nullable field. Otherwise, the synchronization task will fail.

  Note

  Only fields of the BIGINT, INT, TIMESTAMP, DATETIME, and DATE types are supported as partition keys.

• The number of databases to be synchronized does not exceed the limit of ApsaraDB for ClickHouse, which is 256.

• The names of the databases, tables, and columns to be synchronized comply with the naming conventions of ApsaraDB for ClickHouse. For more information about the conventions, see Object naming conventions.

• During schema synchronization, DTS adds the _sign, _is_deleted, and _version fields to the destination tables. If you do not select Schema Synchronization when you configure Synchronization Types, you must manually create the tables for receiving data at the destination and add the extra fields to the tables. For information about table creation requirements and field details, see Table and field information.

• Before synchronizing data, evaluate the performance of the source and destination databases. We recommend that you perform data synchronization during off-peak hours. Otherwise, initial full data synchronization will consume read and write resources of the source and destination databases, which may increase database loads.

• If you synchronize one or more tables instead of the entire database, do not use tools like pt-online-schema-change to perform online DDL operations on the synchronization objects in the source database. Otherwise, the synchronization will fail.

  You can use Data Management (DMS) to perform online DDL operations. For more information, see Change schemas without locking tables.

• During DTS synchronization, do not allow any data other than that from DTS to be written to the destination database. Otherwise, data inconsistency between the source and destination databases will occur.

• If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

  Note

  ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

• If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

  Note

  Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

Special cases

• For a self-managed MySQL source database:

  • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

  • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

    Note

    If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

  • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

• For an ApsaraDB RDS for MySQL source database:

  • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

Type

Description

Source database limits

• The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

• If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

• Binary logs:

  • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

    Important

    • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

    • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

  • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

    Note

    To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

• Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

  Note

  During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

• Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

  Note

  If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

• If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

  Note

  Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

• MySQL compatibility: When DTS synchronizes data from a MySQL-family source database, it relies on the standard MySQL protocol and Binlog format, and behaviors incompatible with standard MySQL are not supported. If your source database claims MySQL compatibility but behaves differently from MySQL (for example, when OceanBase is connected as a source in MySQL mode, the timestamp of the trailing rotate event in the Binlog is 0, which differs from MySQL), the DTS synchronization task may fail.

Other limits

• Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

• Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

• If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

  You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

• Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

• If a primary key or unique key conflict occurs while the task is running:

  • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

    • During full data synchronization, DTS retains the destination record and skips the source record.

    • During incremental synchronization, DTS overwrites the destination record with the source record.

  • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

• Names of tables or columns to synchronize must comply with Tablestore naming conventions:

  • Names consist of uppercase and lowercase letters, digits, or underscores (_), and must begin with a letter or underscore.

  • Names must be 1 to 255 characters long.

• If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

  Note

  ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

• If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

  Note

  Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

Special cases

• For a self-managed MySQL source database:

  • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

  • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

    Note

    If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

  • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

• For an ApsaraDB RDS for MySQL source database:

  • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

Type

Description

Source database limits

• The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

• If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

• Binary logs:

  • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

    Important

    • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

    • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

  • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

    Note

    To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

• Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

  Note

  During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

• Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

  Note

  If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

• If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

  Note

  Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

• MySQL compatibility: When DTS synchronizes data from a MySQL-family source database, it relies on the standard MySQL protocol and Binlog format, and behaviors incompatible with standard MySQL are not supported. If your source database claims MySQL compatibility but behaves differently from MySQL (for example, when OceanBase is connected as a source in MySQL mode, the timestamp of the trailing rotate event in the Binlog is 0, which differs from MySQL), the DTS synchronization task may fail.

Other limits

• Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

• Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

• If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

  You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

• DTS does not support synchronizing indexes, partitions, views, stored procedures, functions, triggers, or foreign keys.

• Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases.

• If the destination Kafka cluster scales out or scales in during synchronization, restart the task.

• If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

  Note

  ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

• If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

  Note

  Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

Special cases

• For a self-managed MySQL source database:

  • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

  • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

    Note

    If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

  • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

• For an ApsaraDB RDS for MySQL source database:

  • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

Type

Description

Source database limits

• The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

• If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

• Binary logs:

  • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

    Important

    • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

    • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

  • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

    Note

    To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

• Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

  Note

  During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

• Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

  Note

  If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

• If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

  Note

  Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

• MySQL compatibility: When DTS synchronizes data from a MySQL-family source database, it relies on the standard MySQL protocol and Binlog format, and behaviors incompatible with standard MySQL are not supported. If your source database claims MySQL compatibility but behaves differently from MySQL (for example, when OceanBase is connected as a source in MySQL mode, the timestamp of the trailing rotate event in the Binlog is 0, which differs from MySQL), the DTS synchronization task may fail.

Other limits

• If a primary key or unique key conflict occurs while the task is running:

  • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

    • During full data synchronization, DTS retains the destination record and skips the source record.

    • During incremental synchronization, DTS overwrites the destination record with the source record.

  • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

• Synchronization object requirements:

  • DTS does not support synchronizing BIT, VARBIT, GEOMETRY, ARRAY, UUID, TSQUERY, TSVECTOR, or TXID_SNAPSHOT data types.

  • DTS does not support synchronizing prefix indexes. If your source database contains prefix indexes, synchronization may fail.

• If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

• Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

• Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

• If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

  You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

• Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

• If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

  Note

  ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

• If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

  Note

  Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

Special cases

• For a self-managed MySQL source database:

  • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

  • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

    Note

    If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

  • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

• For an ApsaraDB RDS for MySQL source database:

  • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

Type

Description

Source database limits

• The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

• If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

• Binary logs:

  • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

    Important

    • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

    • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

  • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

    Note

    To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

• Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

  Note

  During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

• Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

  Note

  If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

• If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

  Note

  Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

• MySQL compatibility: When DTS synchronizes data from a MySQL-family source database, it relies on the standard MySQL protocol and Binlog format, and behaviors incompatible with standard MySQL are not supported. If your source database claims MySQL compatibility but behaves differently from MySQL (for example, when OceanBase is connected as a source in MySQL mode, the timestamp of the trailing rotate event in the Binlog is 0, which differs from MySQL), the DTS synchronization task may fail.

Other limits

• Multi-column attribute changes are not supported.

• Changes to basic data types are not supported.

• If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

• If your self-managed Oracle database uses Real Application Clusters (RAC), do not configure ScanIP. Instead, configure only one virtual IP address (VIP) in the connection information. With this configuration, RAC node failover is not supported.

• If a primary key or unique key conflict occurs while the task is running:

  • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

    • During full data synchronization, DTS retains the destination record and skips the source record.

    • During incremental synchronization, DTS overwrites the destination record with the source record.

  • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

• If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

  Note

  ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

• If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

  Note

  Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

Special cases

• For a self-managed MySQL source database:

  • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

  • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

    Note

    If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

  • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

• For an ApsaraDB RDS for MySQL source database:

  • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

Type

Description

Source database limits

• The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

• If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

• Binary logs:

  • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

    Important

    • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

    • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

  • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

    Note

    To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

• Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

  Note

  During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

• Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

  Note

  If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

• If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

  Note

  Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

• MySQL compatibility: When DTS synchronizes data from a MySQL-family source database, it relies on the standard MySQL protocol and Binlog format, and behaviors incompatible with standard MySQL are not supported. If your source database claims MySQL compatibility but behaves differently from MySQL (for example, when OceanBase is connected as a source in MySQL mode, the timestamp of the trailing rotate event in the Binlog is 0, which differs from MySQL), the DTS synchronization task may fail.

Other limits

• Synchronization object requirements:

  • Only table-level synchronization is supported.

  • DTS does not support synchronizing BIT, VARBIT, GEOMETRY, LINESTRING, POLYGON, ARRAY, UUID, TSQUERY, TSVECTOR, TXID_SNAPSHOT, or POINT data types.

  • DTS does not support synchronizing prefix indexes. If your source database contains prefix indexes, synchronization may fail.

  • DTS does not support synchronizing partitions, views, stored procedures, functions, or triggers.

• If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

• If a primary key or unique key conflict occurs while the task is running:

  • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

    • During full data synchronization, DTS retains the destination record and skips the source record.

    • During incremental synchronization, DTS overwrites the destination record with the source record.

  • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

• Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

• Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

• If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

  You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

• Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

• If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

  Note

  ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

• For full or incremental data synchronization tasks, if the source tables to be synchronized contain foreign keys, triggers, or event triggers, DTS temporarily sets the session_replication_role parameter to replica at the session level. If the destination database account does not have the required permissions, you must manually set the session_replication_role parameter to replica in the destination database. During this period, if cascade update or delete operations occur in the source database while session_replication_role is set to replica, data inconsistency may occur. After the DTS task is released, change the session_replication_role parameter back to origin.

• If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

  Note

  Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

Special cases

• For a self-managed MySQL source database:

  • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

  • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

    Note

    If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

  • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

• For an ApsaraDB RDS for MySQL source database:

  • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

  • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.