Migrate data between ApsaraDB RDS instances

Prerequisites

The source and destination ApsaraDB RDS instances must run one of the following database engines:

Precautions

• Data migration does not affect the data in the source database. During data migration, DTS reads data from the source database and replicates it to the destination database without deleting any data from the source. For more information about how data migration works, see How data migration works.

• During a full data migration, DTS consumes read and write resources on the source and destination databases, increasing their load. If your databases have poor performance, low specifications, or high workloads (for example, if the source database has many slow SQL queries or tables without primary keys, or if deadlocks occur in the destination database), the increased load can strain your databases or even cause service interruptions. Perform the data migration during off-peak hours, such as when the CPU utilization of both databases is below 30%.

• If a source table lacks a primary key or a unique constraint and contains non-unique data, duplicate data may be created in the destination database.

• When you perform only a full data migration, do not write new data to the source ApsaraDB RDS instance during the migration to ensure data consistency.

• For a failed migration task, DTS automatically attempts to resume it. When you cut over to the destination instance, stop or release the migration task first. This prevents the task from being automatically resumed, which could overwrite data in the destination instance with data from the source database.

• DTS automatically creates databases in the destination ApsaraDB RDS instance. If a database name does not comply with the naming conventions for ApsaraDB RDS instances, you must create the database in the destination ApsaraDB RDS instance before configuring the migration task.

  Note

  For information about the naming conventions and how to create a database, see Create a database and an account.

• If you are migrating data between ApsaraDB RDS for PostgreSQL instances, be aware that after the cutover to the destination instance, new sequences will not start incrementing from the maximum value of the sequences in the source database. Before the cutover, you must query the maximum value of the relevant sequences in the source database and then set these values as the starting values for the corresponding sequences in the destination database. Run the following command to query the sequence values in the source database:

  do language plpgsql $$
  declare
    nsp name;
    rel name;
    val int8;
  begin
    for nsp,rel in select nspname,relname from pg_class t2 , pg_namespace t3 where t2.relnamespace=t3.oid and t2.relkind='S'
    loop
      execute format($_$select last_value from %I.%I$_$, nsp, rel) into val;
      raise notice '%',
      format($_$select setval('%I.%I'::regclass, %s);$_$, nsp, rel, val+1);
    end loop;
  end;
  $$;

Billing

Migration types

• Schema migration

  DTS migrates the schemas of the migration objects to the destination instance.

• Full data migration

  DTS migrates all existing data of the specified objects from the source ApsaraDB RDS instance to the destination ApsaraDB RDS instance.

• Incremental data migration

  DTS synchronizes incremental data updates from the source ApsaraDB RDS instance to the destination ApsaraDB RDS instance. Incremental data migration allows you to migrate your data smoothly with minimal downtime.

SQL operations for incremental migration

Database account permissions

Procedure

1. Log on to the Data Transmission Service (DTS) console.

   Note

   If you are automatically redirected to the Data Management (DMS) console, you can click the icon in the lower-right corner and then click to return to the classic DTS console.

2. In the navigation pane on the left, click Data Migration.

3. At the top of the Migration Tasks page, select the region where the destination instance is located.

4. In the upper-right corner of the page, click Create Data Migration Task.

5. Configure the source and destination databases.

   Section	Parameter	Description
   N/A	Task Name	DTS automatically generates a task name. We recommend changing this to a descriptive name for easier identification. The task name does not need to be unique.
   Source Database	Instance Type	Select RDS Instance.
   	Instance Region	Select the region where the source ApsaraDB RDS instance is located.
   	RDS Instance ID	Select the ID of the source ApsaraDB RDS instance. Note The source and destination ApsaraDB RDS instances can be the same or different. You can use DTS to migrate data within a single ApsaraDB RDS instance or between two different ApsaraDB RDS instances.
   	Database Name	Enter the name of the database to migrate from the ApsaraDB RDS for PostgreSQL instance. Note This parameter is displayed and required only when the ApsaraDB RDS instance runs the PostgreSQL engine.
   	Database Account	Enter the database account for the source ApsaraDB RDS instance. For more information about the required permissions, see Permissions required for database accounts.
   	Database Password	Enter the password for the database account. Note After you enter the source database information, you can click Test Connectivity next to Database Password to verify that the information is correct. If the information is correct, the message Passed is displayed. If the message Failed is displayed, click Diagnose next to the Failed message and adjust the source database information based on the prompts.
   	Connection Method	Select Non-encrypted or SSL-encrypted. If you select SSL-encrypted, you must first enable SSL encryption for the ApsaraDB RDS instance. For more information, see Enable SSL encryption. Note This parameter is displayed and required only when the ApsaraDB RDS instance runs the MySQL engine.
   Destination Database	Instance Type	Select RDS Instance.
   	Instance Region	Select the region where the destination ApsaraDB RDS instance is located.
   	RDS Instance ID	Select the ID of the destination ApsaraDB RDS instance. Note The source and destination ApsaraDB RDS instances can be the same or different. You can use DTS to migrate data within a single ApsaraDB RDS instance or between two different ApsaraDB RDS instances.
   	Database Name	Enter the name of the destination database in the ApsaraDB RDS for PostgreSQL instance. The name can be different from the source database name. Note This parameter is displayed and required only when the ApsaraDB RDS instance runs the PostgreSQL engine.
   	Database Account	Enter the database account for the destination ApsaraDB RDS instance. For more information about the required permissions, see Permissions required for database accounts.
   	Database Password	Enter the password for the database account. Note After you enter the destination database information, you can click Test Connectivity after Database Password to verify that the entered information is correct. If the information is correct, a Passed message is displayed. If a Failed message is displayed, click Diagnose after Failed and adjust the destination database information based on the prompts.
   	Connection Method	Select Non-encrypted or SSL-encrypted. If you select SSL-encrypted, you must first enable SSL encryption for the ApsaraDB RDS instance. For more information, see Enable SSL encryption. Note This parameter is displayed and required only when the ApsaraDB RDS instance runs the MySQL engine.

6. After you complete the configuration, click Set Whitelist and Next.

   DTS automatically adds the IP addresses of DTS servers in the corresponding region to the whitelist of your ApsaraDB RDS instance. For more information about the IP address blocks of DTS servers, see IP address blocks of DTS servers.

   Warning

   Adding the public CIDR blocks of DTS servers, whether automatically or manually, may introduce security risks. By using this product, you acknowledge and accept these potential risks. You are responsible for implementing basic security measures, including but not limited to using strong passwords, restricting open ports, using authentication for internal API calls, regularly reviewing and restricting unnecessary network segments, or connecting through private networks such as Express Connect, VPN Gateway, or Smart Access Gateway.

7. Select the migration types and objects.

   Parameter	Description
   Migration Type	Select migration types based on your business requirements and the features supported by the database engine. To perform only a full data migration, select both Schema Migration and Full Data Migration. To perform a migration with minimal downtime, select Schema Migration, Full Data Migration, and Incremental Data Migration. Note If you do not select Incremental Data Migration, do not write new data to the source ApsaraDB RDS instance during the migration to ensure data consistency.
   Migration Objects	In the Migration Objects box, click the objects that you want to migrate, and then click the icon to move them to the Selected Objects box. Note You can select objects at the database, table, or column level. If you select only tables or columns, other objects such as views, triggers, and stored procedures are not migrated. By default, object names in the destination database are the same as in the source database. To rename an object in the destination database, use the object name mapping feature. For more information, see Object name mapping. Using the object name mapping feature can cause the migration of dependent objects to fail.
   Map or Rename	To rename migration objects in the destination database, use the object name mapping feature. For more information, see Object name mapping.
   Retry Duration for Unstable Connections	When the connection to the source or destination database is unstable, DTS retries the connection for 720 minutes (12 hours) by default. You can customize this duration. If DTS successfully reconnects within the specified time, the migration task automatically resumes. Otherwise, the task fails. Note You are charged for the DTS instance during the connection retry period. We recommend that you customize the retry duration based on your business needs or release the DTS instance as soon as possible after the source and destination instances are released.

8. After you complete these settings, click Pre-check and Start in the lower-right corner of the page.

   Note

   • DTS performs a precheck before the task starts. The task can start only after it passes the precheck.

   • If the precheck fails, click the icon next to the failed item to view details.

     • Fix the issues based on the details, and then run the precheck again.

     • For warnings that do not require a fix, you can select Ignore or Ignore and Rerun Precheck to run the precheck again.

9. After the task passes the precheck, click Next.

10. On the Confirm Order page, select a Instance Class and select the Data Transmission Service (Pay-As-You-Go) Terms of Service checkbox.

11. Click Buy and Start. The migration task starts.

    • Full data migration

      Do not manually stop the task. Wait for it to complete automatically to avoid data loss.

    • Incremental data migration

      The task does not stop automatically. Stop it manually.

      Note

      Stop the task during off-peak hours or when you are ready to switch to the destination instance.

      1. Wait until the task status changes to Incremental Data Migration and shows Undelayed. Then, stop writing data to the source database for a few minutes. The status of Incremental Data Migration may show a latency.

      2. Wait until the status of Incremental Data Migration changes back to Undelayed. Then, stop the migration task manually. The progress of schema migration and full data migration both show 100%. To pause the migration task, select the task and click Pause in the batch operations bar at the bottom.