Database migration

Updated at:

The database migration feature provides a complete solution for database migration, validation, and cleanup. This topic describes how to create a ticket to migrate a database.

Prerequisites

  • The following database types are supported: RDS for MySQL, PolarDB for MySQL, and self-managed MySQL databases on ECS instances.

    Note

    The source and destination databases must be of the same type.

  • The source and destination databases must be managed in Security Collaboration mode. For more information, see View the control mode.

  • Write operations to the source database are stopped.

Background information

As your business grows, you may need to upgrade, downgrade, or migrate your databases. Data Management (DMS) provides a database migration feature for these scenarios. It supports homogeneous migration of schemas and data for various databases. The feature also enables you to change destination database names, run pre-migration checks, verify data consistency after migration, and clean up the source database. This ensures a complete lifecycle for database migration and provides convenient multi-tenant management for Software as a Service (SaaS) vendors.

Advantages

Many data migration tools exist, but most are for operations and maintenance (O&M) and lack permission management. The DMS database migration solution offers these advantages:

  • It integrates the security control capabilities of DMS, including permission management and process control.

  • It integrates the data migration feature of Data Transmission Service (DTS) to provide high-performance, concurrent services. For more information, see High performance.

  • It provides a complete solution for migration, validation, and cleanup. You can also update tenant routing configurations and link services to create a complete SaaS migration path for your applications.

Limits

  • You can migrate all tables in a database, but you cannot migrate specific tables.

  • This feature supports full data migration but does not support incremental data migration.

  • Only table objects can be migrated.

Billing

The database migration feature is currently free to use.

Procedure

  1. Log on to the DMS console V5.0.
  2. In the top navigation bar, choose Solution > Migrate Database.

    Note

    If you use the DMS console in simple mode, move the pointer over the 2023-01-28_15-57-17.png icon in the upper-left corner of the DMS console and choose All Features > Solution > Migrate Database.

  3. On the Create Database Migration Ticket page, configure the database migration ticket.

    Parameter

    Description

    Task Name

    Enter a clear and specific task name to reduce communication costs.

    Business Background

    Briefly describe the business background of the task.

    Stakeholders

    Select the ticket collaborators.

    Source Instance

    Select the instance where the source database is located.

    Destination Instance

    Select the destination instance for the source database.

    Note

    The source and destination instances must be of the same database type.

    Source Database

    Select the database to migrate.

    Note
    • If you select multiple source databases, they must have the same owner.

    • Have the export and change permissions for the source database. To apply for permissions, see Manage access control permissions.

    Destination Database

    The destination database name is the same as the source database name by default. Click Edit next to the destination database to change its name.

    Migration Object

    Select the objects to migrate. You can select multiple objects.

    • Schema Migration: Migrates the table schemas from the source database.

    • Full Data Migration: Migrates the table data from the source database.

    Data Consistency Verification

    After migration, compare the data to ensure the destination data is identical to the source data.

    • Do not verify: Does not verify if the data is the same after migration.

    • Automatically verify after migration: Automatically starts data consistency verification after the migration is complete.

      After you select Automatically verify after migration, in the Parameter Settings dialog box, set the maximum number of different rows per table and the data validation limit for tables without primary keys. Then, click Confirm.

    • Manually trigger as needed: After migration, you can choose to skip the verification or start it immediately.

    Cleanup and Release

    After migration, clean up the migrated database from the source instance.

    • Do not clean up: Does not clean up the migrated database from the source instance.

    • Manually trigger as needed: After data consistency verification is complete, you can choose to clean up the source data immediately or with a delay.

    • Automatically clean up after migration (not supported): Cleaning up source data is a high-risk operation and cannot be triggered automatically.

  4. Click Submit application. The system automatically runs a precheck.

    You can view the precheck results in the Precheck section. If the precheck fails, follow the instructions in the error message.

    • Source database permission check: The user who submits the ticket must have export and change permissions for the source database. If not, you are prompted to apply for the required permissions.

    • Destination database permission check:

      • If the destination database exists, the user who submits the ticket must have change permissions for the destination database. If not, you are prompted to apply for the required permissions.

      • If the destination database does not exist, the user who submits the ticket must be an administrator, a DBA, or an instance owner. Otherwise, an error message is returned.

    • Resource Access Management (RAM) permission check: The system checks whether the Alibaba Cloud account has the AliyunDMSDefaultRole role for the migration. If not, you are prompted to grant the role.

    • Destination instance remaining space check:

      • If the destination instance is an RDS for MySQL or PolarDB for MySQL instance, the system checks whether the remaining storage capacity is sufficient for the migration.

      • If the destination instance is a self-managed database on an ECS instance, the precheck fails and prompts you to manually check the remaining space on the destination instance.

    • Check for objects with the same name:

      • If you perform a schema migration, the destination instance cannot contain database-level objects with the same name.

      • If you perform only a full data migration, the source and destination databases must contain objects with the same name and the same Data Definition Language (DDL) statements. An error is reported if the DDL statements are different. A warning is reported if the tables with the same name contain historical data.

    • Whitelist check:

      • If the destination instance is an RDS for MySQL or PolarDB for MySQL instance, the system checks whether the whitelists of the source and destination instances contain the required IP addresses for DTS. If not, the system returns the missing IP addresses and prompts you to add them.

      • If the destination instance is a self-managed database on an ECS instance, the system returns all required IP addresses for the DTS whitelist.

  5. After the precheck is passed, click Submit for Approval.

  6. After the ticket is approved, click Start Now or Schedule.

    • Start Now: Executes the migration task immediately.

    • Schedule: In the Schedule Settings dialog box, select the date and time to start the migration task.

    During task execution, you can view the migration progress and logs in the Database Migration wizard of the Execute section on the ticket details page.

  7. Optional: After the database migration is complete, click Next: Data Consistency Verification to verify data consistency.

    • Skip verification: Skips the data consistency verification step.

    • Start now: In the Parameter Settings dialog box, set the maximum number of different rows per table and the data validation limit for tables without primary keys. Then, click Confirm.

    • Schedule: In the Schedule Settings dialog box, set the maximum number of different rows per table and the data validation limit for tables without primary keys, select a start time for the verification, and then click Confirm.

    Note

    If data inconsistencies exist, click View Details to view an overview. In the overview dialog box, click Details to view the details of the inconsistent data:

    • If the data volume is small, you can sync the data manually,

    • If the volume of inconsistent data is large, identify the cause, clean up the destination database, and then migrate the database again.

  8. Optional: After the data consistency verification is complete, click Next: Data Cleanup to clean up the source data.

    • Complete: Completes the database migration ticket.

    • Clean up now: This operation immediately deletes the migrated tables. Proceed with caution.

    • Delayed cleanup: In the Delayed Cleanup dialog box, specify a cleanup delay from 1 to 30 days. Within this delay period, you can use the one-click rollback feature to restore the data in the source database if needed.