Data migration and synchronization FAQ
Answers to common questions about data migration and data synchronization for Tair and Redis instances using Data Transmission Service (DTS).
-
What is the difference between data migration and data synchronization by using DTS?
-
Can I obtain SYNC/PSYNC command permissions to migrate data with third-party tools?
-
Can a DTS precheck list the specific duplicate keys in the destination database?
-
What do I do if data latency occurs when DTS synchronizes data to a Redis instance in proxy mode?
What are data migration and data synchronization?
Data migration copies key-value pairs from a Tair or open source Redis instance to another Tair or open source Redis instance. The task runs once and stops automatically after all data is transferred. Use it when you need a one-time, bulk transfer—for example, moving data from a self-managed Redis instance to a Tair instance on Alibaba Cloud.
Data synchronization continuously replicates key-value pairs from a source instance to a destination instance in real time. The task keeps running to maintain data consistency between the two instances. Use it when you need ongoing replication—for example, for active geo-redundancy or disaster recovery.
Both operations replicate data to the destination instance without affecting the source instance.
How do data migration and data synchronization work?
Both features rely on DTS to capture and apply changes between instances. For a detailed explanation of the underlying architecture, see System architecture and design concepts.
What is the difference between data migration and data synchronization by using DTS?
The table below compares key dimensions to help you choose the right approach.
DTS data synchronization supports more Tair scenarios and features than data migration. If your use case allows, prefer data synchronization. For more information, see Data synchronization.
| Dimension | Data migration | Data synchronization |
|---|---|---|
| Typical use case | One-time transfer from on-premises databases, self-managed databases on ECS instances, or third-party cloud platforms to Tair instances on Alibaba Cloud | Continuous, real-time replication between Tair instances—for active geo-redundancy, disaster recovery, or data analytics |
| Supported scenarios | Support for data migration | Support for data synchronization |
| Key features | Supports migration across self-managed databases in virtual private clouds (VPCs) owned by different Alibaba Cloud accounts | Supports duplicate keys handling modes, bidirectional synchronization between instances, and data synchronization across self-managed databases in VPCs owned by different Alibaba Cloud accounts |
| Billing | Schema migration and full data migration are free. Incremental data migration (optional) is billed by duration (regardless of the amount of data migrated); only pay-as-you-go is supported. | Schema migration and full data migration are free. After incremental migration starts, billing is by duration; both pay-as-you-go and subscription are supported. |
What do I do if an error occurs during a precheck?
Before a migration or synchronization task starts, DTS runs a precheck on the source and destination databases to confirm the task can proceed. If the precheck fails, consult the following resources to identify and fix the issue:
Can I obtain SYNC/PSYNC command permissions to migrate data with third-party tools?
Alibaba Cloud Redis and Tair instances disable the SYNC and PSYNC commands by default, and this cannot be enabled. As a result, third-party migration or synchronization tools (such as redis-shake in sync/psync mode, or DTS services from other cloud providers) cannot perform incremental synchronization against Alibaba Cloud Redis instances over the SYNC/PSYNC protocol.
To migrate data from an Alibaba Cloud Redis instance to an external environment, use Alibaba Cloud DTS.
When the source database is an ApsaraDB for Tair instance, you do not need to obtain PSYNC command permissions on the source database. You can perform incremental migration by using DTS instead.
How long does data migration or synchronization take?
The duration cannot be precisely estimated. The following orders of magnitude are for reference only: small data volumes (below the GB level) usually complete within minutes; medium data volumes (several GB) usually complete within tens of minutes; large data volumes (tens of GB or more) may take several hours.
The actual duration is affected by factors such as data volume, bandwidth, DTS instance class, and instance load, and may vary considerably.
Can a DTS precheck list the specific duplicate keys in the destination database?
A DTS precheck does not list the specific duplicate keys in the destination database. It only reports that the destination database is not empty. If you need to identify the duplicate data, query the destination database yourself and compare the results against the source database.
What do I do if data latency occurs when DTS synchronizes data to a Redis instance in proxy mode?
A Redis instance in proxy mode performs syntax checks on Lua scripts, and the script_check_enable parameter is enabled by default. If the source runs a Lua script that does not meet the syntax requirements of proxy mode—for example, the first argument of redis.call or redis.pcall is a variable instead of a string literal—incremental synchronization in DTS stalls because the destination instance returns an error. This shows up as continuously growing synchronization latency.
To resolve this, disable script_check_enable in the parameter settings of the destination instance in proxy mode, and then restart the DTS synchronization task. The latency gradually catches up after the write throughput on the source instance decreases.
How do I configure the whitelist of the destination Tair instance for cross-region synchronization by using DTS?
When you use DTS for cross-region data synchronization (for example, from China (Shanghai) to China (Zhangjiakou)), you must add the CIDR blocks of the DTS servers in both the source region and the destination region to the whitelist of the destination Tair instance. This ensures that DTS can connect to the destination instance and complete synchronization.