You can use migration tools, such as DataWorks/DataX and Tablestore's Tunnel Service, to perform full migration, incremental migration, or real-time synchronization of Tablestore data without affecting your online services. This topic describes how to select the appropriate migration tool.
Select a migration tool
When you migrate data, select an appropriate migration tool for your business. The following table describes the migration tools that Tablestore supports.
|
Migration tool |
Description |
Scenarios |
|
DataWorks data integration is a stable, efficient, and elastic data synchronization platform. It is built on the DataX offline data synchronization tool. You can use it for data migration and synchronization between disparate data sources, such as MySQL, Oracle, and SQL Server. DataWorks/DataX abstracts the data synchronization process for different data sources into a Reader plugin that reads data from a source and a Writer plugin that writes data to a destination. For more information, see DataWorks data integration or DataX. |
||
|
Tunnel Service is an integrated service for full and incremental data synchronization. It is built on Tablestore data APIs. You can use it for data migration and synchronization if the source table is a Tablestore data table. Tunnel Service provides three types of distributed, real-time data consumption channels: incremental, full, and a combination of full and incremental. You can create a data channel for a data table to easily consume and process its historical and new data. For more information, see Tunnel Service. |
||
|
Data Transmission Service (DTS) is a real-time data streaming service provided by Alibaba Cloud. It supports data exchange between data sources such as relational databases (RDBMS), NoSQL databases, and online analytical processing (OLAP) systems. DTS supports data synchronization, migration, subscription, integration, and processing. This service helps you build a secure, scalable, and highly available data architecture. For more information, see What is Data Transmission Service (DTS)?. |
||
|
canal provides incremental data subscription and consumption features by parsing incremental logs from MySQL databases. It is an open source Change Data Capture (CDC) tool from Alibaba. It retrieves and parses MySQL binlog data, and then sends data changes downstream. You can use canal to achieve real-time synchronization from MySQL to other databases. For more information, see the official canal website. |
||
|
Tapdata Cloud is a real-time data service from Tapdata that provides data replication and distribution. It offers millisecond-level real-time data synchronization and data fusion services across clouds, regions, and various data source types. For more information, see What is Tapdata?. |
DataWorks/DataX
You can use DataWorks/DataX to migrate and synchronize databases to Tablestore. You can also use it to migrate and synchronize Tablestore data across instances or accounts.
-
Migrate database data to Tablestore
DataWorks/DataX provides stable and efficient data synchronization between disparate data sources. You can migrate various databases to Tablestore, as shown in the following figure.
NoteFor more information about the data sources and read/write plugins that DataWorks and DataX support, see Data sources and read/write plugins supported by DataWorks or Data sources and read/write plugins supported by DataX.
-
Migrate and synchronize Tablestore data across instances or accounts
You can configure the Tablestore Reader and Writer plugins in DataWorks/DataX to replicate data from data tables or time series tables. The process is shown in the following figures. The following table describes the Tablestore plugins.
Plugin
Description
Reads data from Tablestore tables. You can specify a timestamp range to extract incremental data.
Exports incremental data from Tablestore tables.
Writes data to Tablestore.
Tunnel Service
You can use Tunnel Service to easily build efficient and elastic data replication solutions.
Data Transmission Service (DTS)
DTS synchronizes ongoing data changes between two data sources. Data synchronization is often used to transfer data from online transactional processing (OLTP) to online analytical processing (OLAP). Data synchronization includes two phases:
-
Initial synchronization: DTS starts collecting incremental data, then loads the structure and historical data from the source database to the destination database.
-
Real-time data synchronization: DTS synchronizes ongoing data changes and keeps the source and destination databases in sync.
canal
When you use canal to synchronize data, the Deployer service pulls binlog data from the upstream source and records offsets. The Client-Adapter service connects to the Deployer to retrieve the parsed data and then sends the data to the destination database.
Tapdata Cloud
When you use Tapdata Cloud to synchronize data, the Tapdata agent retrieves task information from Tapdata Cloud Manager (TCM). The agent uses streaming technology to retrieve data from the source system, transform the data, and send it to the destination system.
Billing
Billing rules vary based on the migration tool you use:
-
When you use migration tools or other products, such as DTS and the Alibaba Cloud IoT Platform, to access Tablestore, you are charged for the read and write throughput consumed by your requests.
-
Using Tunnel Service does not incur additional costs. When you consume data from Tunnel Service, you are charged for the read throughput based on the amount of data that you pull.
For more information, see CU mode (formerly pay-as-you-go).