Create DTS tasks using Alibaba Cloud CLI-Data Transmission Service(DTS)-阿里云帮助中心

Overview

To create a DTS task using Alibaba Cloud CLI, complete the following steps:

Purchase a DTS instance: Call the CreateDtsInstance operation to purchase a DTS instance of a specified type and get its instance ID.
Configure the DTS task: Call the ConfigureDtsJob operation to configure the connection information for the source and destination databases, the objects to migrate or synchronize, and other settings.
Start the DTS task: Call the StartDtsJob operation to start the task, and then call the DescribeDtsJobDetail operation to query its status.

The examples in this topic transfer data from an RDS for MySQL source database to an RDS for MySQL destination database. They cover data migration, synchronization, subscription, and validation tasks. You can replace the source and destination database parameters as needed.

Prerequisites

Alibaba Cloud CLI is installed and configured. For more information, see Install or update Alibaba Cloud CLI and Configure and manage credentials.
The RAM user associated with your AccessKey has the required permissions to manage DTS. For more information, see Grant permissions to a RAM user to manage DTS by using system policies.
The source and destination database instances are created, and the necessary preparations for DTS are complete. For more information, see Preparations.

Notes

The SourceRegion and DestinationRegion parameters in the CreateDtsInstance operation must match the regions of your source and destination databases.
The Type parameter in the CreateDtsInstance operation specifies the task type. Valid values: MIGRATION (data migration), SYNC (data synchronization), and SUBSCRIBE (data subscription).
The DbList parameter in the ConfigureDtsJob operation is in JSON format and specifies the objects to migrate or synchronize. For format details, see Objects to migrate, synchronize, or subscribe to.
Data migration tasks support only the pay-as-you-go billing method. Data synchronization and data subscription tasks support both pay-as-you-go and subscription billing methods.
Pass database password parameters, such as SourceEndpointPassword and DestinationEndpointPassword, as environment variables to avoid exposing them in plaintext on the command line.

Create a data migration task

The following example creates a data migration task from one RDS for MySQL database to another.

Step 1: Purchase a migration instance

Run the following command to call the CreateDtsInstance operation to purchase a data migration instance.

aliyun dts CreateDtsInstance \
  --Type MIGRATION \
  --SourceRegion cn-hangzhou \
  --DestinationRegion cn-shanghai \
  --SourceEndpointEngineName MySQL \
  --DestinationEndpointEngineName MySQL \
  --PayType PostPaid \
  --InstanceClass small

Sample output:

{
  "InstanceId": "dtsse7s11v26j****",
  "JobId": "se7s11v26j****",
  "RequestId": "E52C5BD7-AF7D-4F5D-A8F5-D3B5F0****",
  "Success": true
}

Record the InstanceId value. This is the DTS instance ID, used as the DtsInstanceId parameter in subsequent steps.

Step 2: Configure the migration task

Run the following command to call the ConfigureDtsJob operation to configure the source and destination databases.

aliyun dts ConfigureDtsJob \
  --DtsInstanceId dtsse7s11v26j**** \
  --DtsJobName "rds_mysql_migration_demo" \
  --JobType MIGRATION \
  --SourceEndpointInstanceType RDS \
  --SourceEndpointInstanceID rm-bp1imrtn6fq****  \
  --SourceEndpointEngineName MYSQL \
  --SourceEndpointRegion cn-hangzhou \
  --SourceEndpointDatabaseName source_db \
  --SourceEndpointUserName dts_user \
  --SourceEndpointPassword '${SOURCE_DB_PASSWORD}' \
  --DestinationEndpointInstanceType RDS \
  --DestinationEndpointInstanceID rm-bp1k5v0rm3d****  \
  --DestinationEndpointEngineName MYSQL \
  --DestinationEndpointRegion cn-shanghai \
  --DestinationEndpointDataBaseName target_db \
  --DestinationEndpointUserName dts_user \
  --DestinationEndpointPassword '${DEST_DB_PASSWORD}' \
  --StructureInitialization true \
  --DataInitialization true \
  --DataSynchronization true \
  --DbList '{"source_db":{"name":"source_db","all":true}}'

Sample output:

{
  "RequestId": "1D234B10-5C84-4F26-B06A-3E48D2****",
  "DtsJobId": "se7s11v26j****",
  "ErrCode": "",
  "Success": "true",
  "HttpStatusCode": 200
}

Step 3: Start the migration task

Run the following command to call the StartDtsJob operation to start the migration task.

aliyun dts StartDtsJob \
  --DtsJobId se7s11v26j****

Sample output:

{
  "RequestId": "A7B3E61D-2C2F-4B3B-A5D8-F7E49D****",
  "ErrCode": "",
  "Success": "true",
  "HttpStatusCode": 200
}

Step 4: Query the task status

Run the following command to call the DescribeDtsJobDetail operation to query the status of the migration task.

aliyun dts DescribeDtsJobDetail \
  --DtsJobId se7s11v26j****

Confirm that the Status in the returned result is Migrating, which indicates that the migration task is running.

Create a data synchronization task

The following example creates a data synchronization task from one RDS for MySQL database to another.

Step 1: Purchase a synchronization instance

Run the following command to call the CreateDtsInstance operation to purchase a data synchronization instance.

aliyun dts CreateDtsInstance \
  --Type SYNC \
  --SourceRegion cn-hangzhou \
  --DestinationRegion cn-hangzhou \
  --SourceEndpointEngineName MySQL \
  --DestinationEndpointEngineName MySQL \
  --PayType PostPaid \
  --InstanceClass small \
  --SyncArchitecture oneway

Sample output:

{
  "InstanceId": "dtsbi6e22ay24****",
  "JobId": "bi6e22ay24****",
  "RequestId": "6C2FE2A1-B9D3-4E5C-8A1B-C3F28E****",
  "Success": true
}

Step 2: Configure the synchronization task

Run the following command to call the ConfigureDtsJob operation to configure the source and destination databases for the synchronization task.

aliyun dts ConfigureDtsJob \
  --DtsInstanceId dtsbi6e22ay24**** \
  --DtsJobName "rds_mysql_sync_demo" \
  --JobType SYNC \
  --SourceEndpointInstanceType RDS \
  --SourceEndpointInstanceID rm-bp1imrtn6fq****  \
  --SourceEndpointEngineName MYSQL \
  --SourceEndpointRegion cn-hangzhou \
  --SourceEndpointDatabaseName source_db \
  --SourceEndpointUserName dts_user \
  --SourceEndpointPassword '${SOURCE_DB_PASSWORD}' \
  --DestinationEndpointInstanceType RDS \
  --DestinationEndpointInstanceID rm-bp1k5v0rm3d**** \
  --DestinationEndpointEngineName MYSQL \
  --DestinationEndpointRegion cn-hangzhou \
  --DestinationEndpointDataBaseName target_db \
  --DestinationEndpointUserName dts_user \
  --DestinationEndpointPassword '${DEST_DB_PASSWORD}' \
  --StructureInitialization true \
  --DataInitialization true \
  --DataSynchronization true \
  --DbList '{"source_db":{"name":"source_db","all":true}}'

Sample output:

{
  "RequestId": "8E2D3A9C-4B5F-4C6A-9D7E-A1B2C3****",
  "DtsJobId": "bi6e22ay24****",
  "ErrCode": "",
  "Success": "true",
  "HttpStatusCode": 200
}

Step 3: Start the synchronization task

Run the following command to call the StartDtsJob operation to start the synchronization task.

aliyun dts StartDtsJob \
  --DtsJobId bi6e22ay24****

Step 4: Query the task status

Run the following command to call the DescribeDtsJobDetail operation to query the status of the synchronization task.

aliyun dts DescribeDtsJobDetail \
  --DtsJobId bi6e22ay24****

Confirm that the Status in the response is Synchronizing, which indicates that the synchronization task is in progress.

aliyun dts CreateDtsInstance \
  --Type SUBSCRIBE \
  --SourceRegion cn-hangzhou \
  --DestinationRegion cn-hangzhou \
  --SourceEndpointEngineName MySQL \
  --PayType PostPaid \
  --InstanceClass small

{
  "InstanceId": "dtsk3m1zb527a****",
  "JobId": "k3m1zb527a****",
  "RequestId": "3F5A8B2C-6D7E-4A9B-8C1D-E2F3A4****",
  "Success": true
}

aliyun dts ConfigureSubscription \
  --RegionId cn-hangzhou \
  --DtsInstanceId dtsk3m1zb527a**** \
  --DtsJobName "rds_mysql_subscribe_demo" \
  --SourceEndpointInstanceType RDS \
  --SourceEndpointInstanceID rm-bp1imrtn6fq**** \
  --SourceEndpointEngineName MYSQL \
  --SourceEndpointRegion cn-hangzhou \
  --SourceEndpointDatabaseName source_db \
  --SourceEndpointUserName dts_user \
  --SourceEndpointPassword '${SOURCE_DB_PASSWORD}' \
  --DbList '{"source_db":{"name":"source_db","all":true}}' \
  --SubscriptionDataTypeDDL true \
  --SubscriptionDataTypeDML true \
  --SubscriptionInstanceNetworkType vpc \
  --SubscriptionInstanceVPCId vpc-bp1opxu1zkhn00g****  \
  --SubscriptionInstanceVSwitchId vsw-bp1s5fnvk4gn2tw****

{
  "RequestId": "B4C5D6E7-8F9A-4B1C-2D3E-F4A5B6****",
  "DtsJobId": "k3m1zb527a****",
  "ErrCode": "",
  "Success": "true",
  "HttpStatusCode": 200
}

aliyun dts StartDtsJob \
  --DtsJobId k3m1zb527a****

aliyun dts DescribeDtsJobDetail \
  --DtsJobId k3m1zb527a****

Verify that the Status in the response is Normal, which indicates that the subscription task is running.

Create a data validation task

A data validation task verifies data consistency between the source and destination databases after migration or synchronization. The following example creates a validation task for an existing data migration task.

Step 1: Purchase a validation instance

Run the following command to call the CreateDtsInstance operation to purchase a data validation instance.

aliyun dts CreateDtsInstance \
  --Type CHECK \
  --SourceRegion cn-hangzhou \
  --DestinationRegion cn-shanghai \
  --SourceEndpointEngineName MySQL \
  --DestinationEndpointEngineName MySQL \
  --PayType PostPaid \
  --InstanceClass small

Sample output:

{
  "InstanceId": "dtsp2w8rx6315****",
  "JobId": "p2w8rx6315****",
  "RequestId": "D5E6F7A8-9B0C-4D1E-2F3A-B4C5D6****",
  "Success": true
}

Step 2: Configure the validation task

Run the following command to call the ConfigureDtsJob operation to configure the source and destination databases for the validation task.

aliyun dts ConfigureDtsJob \
  --DtsInstanceId dtsp2w8rx6315**** \
  --DtsJobName "rds_mysql_check_demo" \
  --JobType CHECK \
  --SourceEndpointInstanceType RDS \
  --SourceEndpointInstanceID rm-bp1imrtn6fq**** \
  --SourceEndpointEngineName MYSQL \
  --SourceEndpointRegion cn-hangzhou \
  --SourceEndpointDatabaseName source_db \
  --SourceEndpointUserName dts_user \
  --SourceEndpointPassword '${SOURCE_DB_PASSWORD}' \
  --DestinationEndpointInstanceType RDS \
  --DestinationEndpointInstanceID rm-bp1k5v0rm3d**** \
  --DestinationEndpointEngineName MYSQL \
  --DestinationEndpointRegion cn-shanghai \
  --DestinationEndpointDataBaseName target_db \
  --DestinationEndpointUserName dts_user \
  --DestinationEndpointPassword '${DEST_DB_PASSWORD}' \
  --StructureInitialization false \
  --DataInitialization true \
  --DataSynchronization false \
  --DbList '{"source_db":{"name":"source_db","all":true}}'

Sample output:

{
  "RequestId": "E6F7A8B9-0C1D-4E2F-3A4B-C5D6E7****",
  "DtsJobId": "p2w8rx6315****",
  "ErrCode": "",
  "Success": "true",
  "HttpStatusCode": 200
}

Step 3: Start the validation task

Run the following command to call the StartDtsJob operation to start the validation task.

aliyun dts StartDtsJob \
  --DtsJobId p2w8rx6315****

Step 4: Query the validation results

Run the following command to call the DescribeDtsJobDetail operation to query the results of the validation task.

aliyun dts DescribeDtsJobDetail \
  --DtsJobId p2w8rx6315****

Confirm that Status is Finished, which indicates that the validation is complete. Review the details in the response to determine whether the data is consistent.

API	Description
CreateDtsInstance	Purchases a DTS instance.
ConfigureDtsJob	Configures a data migration, data synchronization, or data validation task.
ConfigureSubscription	Configures a data subscription task.
StartDtsJob	Starts a DTS task.
DescribeDtsJobDetail	Queries the details of a DTS task.
DescribeDtsJobs	Queries a list of DTS tasks and the details of each task.

Overview

Prerequisites

Notes

Create a data migration task

Step 1: Purchase a migration instance

Step 2: Configure the migration task

Step 3: Start the migration task

Step 4: Query the task status

Create a data synchronization task

Step 1: Purchase a synchronization instance

Step 2: Configure the synchronization task

Step 3: Start the synchronization task

Step 4: Query the task status

Create a data subscription task

Step 1: Purchase a subscription instance

Step 2: Configure the subscription task

Step 3: Start the subscription task

Step 4: Query the task status

Create a data validation task

Step 1: Purchase a validation instance

Step 2: Configure the validation task

Step 3: Start the validation task

Step 4: Query the validation results

Related APIs