What are data ingestion solutions-Data Management-阿里云帮助中心

A project space for data warehouse development consists of data ingestion solutions and business scenarios. In a data ingestion solution, you can add and run multiple real-time synchronization and migration tasks. These tasks perform full or near-real-time incremental migration and integration of data. This fulfills the data migration and integration requirements for business scenarios within the project space.

You can create a data ingestion solution to synchronize data from your online databases based on the required synchronization efficiency and data volume. Data ingestion solutions include the following:

Real-time synchronization task: A data transmission service based on offline data integration. For more information about how to create a real-time synchronization task, see Create a real-time synchronization task.
Real-time migration task: A data transmission service based on Data Transmission Service (DTS). For more information about how to create a real-time migration task, see Create a real-time migration task.

Note

For data synchronization and migration services that are based on DTS, you can use DTS checkpoint nodes to check the synchronization status. For more information, see Configure a DTS checkpoint node.
For more information about the differences between data migration and data synchronization, see What are the differences between data migration and data synchronization?.

Create a real-time synchronization task

Log in to DMS 5.0.
In the top navigation bar, choose Data+AI > Data Development > Data Warehouse Development.

Note
If you use the DMS console in simple mode, move the pointer over the icon in the upper-left corner and choose All Features > Data+AI > Data Development > Data Warehouse Development.
Click the name of the target project space.
In the navigation pane on the left of the project space, choose > Data Ingestion Solutions > Real-time Synchronization.
Note
Alternatively, you can right-click Data Ingestion Solutions in the navigation pane on the left side of the project space and choose New > Real-time Synchronization.
Create and purchase a real-time synchronization task.

Create a real-time migration task

Log in to DMS 5.0.
In the top navigation bar, choose Data+AI > Data Development > Data Warehouse Development.

Note
If you use the DMS console in simple mode, move the pointer over the icon in the upper-left corner and choose All Features > Data+AI > Data Development > Data Warehouse Development.
Click the name of the target project space.
In the navigation pane on the left of the project space, choose > Data Ingestion Solutions > Real-time Migration.
Note
In the project space, you can also right-click Data Ingestion Solutions in the navigation pane on the left and choose New > Real-time Migration.
Create and purchase a real-time migration task.

What are the differences between data migration and data synchronization?

Item	Data migration	Data synchronization
Scenarios	This feature is mainly used for cloud migration, such as migrating on-premises databases, self-managed databases on ECS instances, or third-party cloud databases to Alibaba Cloud databases. This is a one-time task. After the migration is complete, you can release the instance.	This feature is mainly used for real-time data synchronization between two data sources. It applies to scenarios such as active geo-redundancy, data disaster recovery, cross-border data synchronization, query and report offloading, cloud business intelligence (BI), and real-time data warehouses. After you create a data synchronization task, it continuously synchronizes data between the specified data sources to ensure data consistency.
Supported databases	For more information, see Overview of migration solutions.	For more information, see Overview of data synchronization solutions.
Supported database deployment locations	Self-managed database with a public IP address Self-managed database without a public IP address or port number (connected over Database Gateway) Self-managed database connected over Cloud Enterprise Network (CEN) Self-managed database hosted on ECS Self-managed database connected over Express Connect, VPN Gateway, or Smart Access Gateway	Self-managed database hosted on ECS Self-managed database connected over Express Connect, VPN Gateway, or Smart Access Gateway Self-managed database without a public IP address or port number (connected over Database Gateway) Self-managed database connected over CEN Note Data synchronization uses internal network transmission, which ensures lower network latency.
Feature differences	Supports object name mapping for databases, tables, and columns. Supports filtering data to be migrated. Supports selecting the SQL operation types to synchronize. For example, you can synchronize only INSERT operations. Supports reading data from a virtual private cloud (VPC) that belongs to another Alibaba Cloud account. This feature lets you migrate a self-managed database in a VPC across Alibaba Cloud accounts.	Supports object name mapping for databases, tables, and columns. Supports filtering data to be synchronized. Supports modifying synchronization objects online. Supports two-way synchronization between data sources such as MySQL. Supports selecting the SQL operation types to synchronize. For example, you can synchronize only INSERT operations.
Billing methods	Only pay-as-you-go is supported.	Supports pay-as-you-go and subscription.
Billing rules	You are billed only when incremental data migration is running, including when it is paused. Schema migration and full data migration are not billed.	For pay-as-you-go instances, billing starts after the instance is configured and started. You are also billed when the task is paused. For subscription instances, a one-time fee is charged at the time of purchase based on the selected configuration and duration.

Note

For some databases that are not supported by data synchronization, such as MongoDB instances, you can use incremental data migration to synchronize data.