View details of a data migration task

Schema migration

This phase migrates data object definitions, such as tables, indexes, constraints, comments, and views, from the source database to the destination database. Temporary tables are automatically filtered out. If the source database is not an OceanBase database, Data Transmission automatically converts data types and SQL syntax to be compatible with the destination OceanBase tenant type and then replicates them to the destination database. If an object with the same name already exists in the destination database, Data Transmission skips it by default. You must ensure that the table schemas are consistent between the source and destination databases.

When the task enters the forward switchover step during data migration, Data Transmission automatically deletes hidden columns and unique indexes based on the migration task type. For more information, see Hidden column mechanism in Data Migration Service.

On the Schema Migration page, you can view the status, start time, end time, total duration, and migration progress for databases, tables, and views.

You can also perform the following actions on destination objects:

• View creation syntax: On the Database or Table tab, click View next to the desired object to view its creation syntax.

  If the table creation syntax is fully compatible, the DDL syntax that was executed on the OBServer is displayed. If it is not fully compatible, the converted creation syntax is displayed.

• Modify creation syntax and retry: For failed DDL statements, you can review and modify the DDL conversion result, and then retry the migration to the destination.

• Retry or retry all failed objects: For failed schema migration tasks, you can retry them individually or click Retry All Failed Objects in the upper-right corner of the tab to retry all of them.

• Skip or batch skip: For failed schema migration tasks, you can skip them individually. You can also select multiple objects to skip and click Batch Skip in the upper-right corner.

• Remove or batch remove: For failed schema migration tasks, you can remove them individually. You can also select multiple objects to remove and click Batch Remove in the upper-right corner.

• View database return code: For failed schema migration tasks, you can view the DDL statement and the error message returned from the OBServer execution.

Full migration

This phase migrates historical data from the tables in the source database to the corresponding tables in the destination database. On the Full Migration page, you can filter by source or destination database, or select View Problematic Objects to identify objects that are blocking the overall migration progress. You can also view Table Objects, Table Indexes, and Full Migration Performance. The status of the full migration is displayed as completed only after both the table objects and table indexes are migrated.

• On the Table Objects tab, you can view the object name, source database, destination database, estimated data volume, completed data volume, and status.

• On the Table Indexes tab, you can view the table object, source database, destination database, creation time, end time, duration, and status. You can also view the index creation syntax, retry individual or all failed objects, and skip or remove individual or multiple failed indexes.

• On the Full Migration Performance tab, you can view graphical performance metrics including Source RPS, Destination RPS, Source Migration Traffic, Destination Migration Traffic, Source Average Read Time, Source Average Slice Time, and Destination Average Write Time, along with performance baselines to help you identify performance issues.

  

  Full migration combined with incremental synchronization ensures eventual consistency between the destination and source databases. If any objects fail during full migration, the specific failure reason is displayed.

  Important

  If you did not configure Schema Migration when you selected the Migration Type, Data Transmission migrates data based on matching fields between the source and destination without checking for table schema consistency.

Incremental synchronization

After an incremental synchronization task starts, it synchronizes data changes (inserts, updates, and deletes) from the source database to the corresponding tables in the destination database. If the source database is continuously receiving write operations, Data Transmission starts the incremental pull module before the full data migration begins. This module pulls, parses, encapsulates, and stores incremental update data from the source instance.

After full data migration completes, Data Transmission starts the incremental replay module to retrieve incremental data from the pull module. After filtering, mapping, and transformation, the data is synchronized to the destination instance. If a DDL operation on the source causes a JDBCWriter exception and the migration task fails, the page displays the problematic DDL statement and a skip button. Click Skip in the pop-up dialog box and confirm the operation.

When you create a data migration task, enable monitoring and alerting to stay informed about the task status. For more information, see Set alerts. If monitoring and alerting is not enabled, the status of the incremental synchronization step changes from Running to Monitoring when the synchronization latency is 1 minute or less.

With monitoring and alerting enabled:

• If the incremental synchronization latency exceeds the configured alert threshold, the step remains in the Running state and no alert is triggered.

• If the incremental synchronization latency is less than or equal to the configured alert threshold, the status of the step changes from Running to Monitoring. Once the status is Monitoring, it does not revert to Running even if the latency later exceeds the threshold.

For data migration tasks in the Running state, you can view the latency, current offset time, and incremental synchronization performance in the incremental synchronization section. The latency is displayed in the format: X seconds (updated Y seconds ago). An update interval (Y) of less than 20 seconds is considered normal.

For data migration tasks in the Paused or Failed state, you can enable DDL/DML statistics to track database operations after enabling the feature. You can also view the details of incremental synchronization objects and performance metrics.

• The Sync Object Statistics tab displays table-level DML counts. The totals for Changes, Delete, Insert, and Update at the top of the tab represent the total number of DML operations for the current migration task, which is the sum of the statistics from the Incremental Sync Objects tab.

  

  If you disable DDL/DML statistics after you have enabled it, Data Transmission retains the historical statistics.

• The Incremental Sync Performance tab displays:

  • Latency: The lag time in seconds for syncing incremental changes from the source to the destination.

  • Migration Traffic: The throughput in KB/s for syncing incremental change data from the source to the destination.

  • Average Execution Time: The mean execution time per SQL statement, in milliseconds.

  • Average Commit Time: The mean transaction commit time, in milliseconds.

  • RPS: The number of records processed per second.

Full verification

After the full data migration is complete and the incremental data has caught up with the source, Data Transmission automatically starts a full data verification task for the configured source and destination tables.

During incremental synchronization, you can also initiate custom data verification using the provided interfaces.

On the Full Verification page, you can view the overall status, start time, end time, total duration, estimated total rows, verified rows, RPS, and real-time traffic.

The Full Verification page includes the Verification Objects and Full Verification Performance tabs.

• On the Verification Objects tab, you can view the verification progress and the object list.

  

  • For all migration objects, you can view the object name, source and destination databases, full verification progress, result, and result summary.

  • You can filter by source database, destination database, or verification result.

  • Select View Completed Objects Only to view basic information, such as object names, for objects that have completed schema migration.

  • To re-verify all migration objects, click Re-verify > Restart Full Verification.

  • If tables show inconsistent verification results:

    To obtain the original inconsistent data for verification analysis, click Download Inconsistent Results to save the data locally.

    Important

    You can download inconsistent results only when the Full Verification step is in the Completed state and some results are Inconsistent.

    To re-verify all data in a table, click Re-verify > Re-verify Abnormal Tables.

    To re-verify only inconsistent records in a table, click Re-verify > Verify Only Inconsistent Table Records.

    Important

    Correction operations are not supported if the source is empty.

• On the Full Verification Performance tab, you can view graphical performance metrics including Source RPS, Destination RPS, Source Migration Traffic, and Destination Migration Traffic, along with performance baselines to help you identify performance issues.

Data Transmission Service lets you skip full verification for tasks that are being verified or have failed verification. On the Full Verification page, click Skip Full Verification, and in the dialog box that appears, click OK.

After full verification is complete, click Proceed to Next Phase to start the forward switchover. After you enter the switchover process, you cannot re-run the current verification task for data comparison or correction.

Forward switchover

Forward switchover, which is an abstracted and standardized version of a traditional system cutover, does not switch application connections. It is a required workflow that coordinates the data migration task with the application switch. You must complete the entire forward switchover process before you switch application connections to the destination.

Forward switchover is an essential step in data migration. It ensures that Data Transmission completes all forward migration work and lets you start the reverse incremental component based on your business needs. Key tasks include the following:

1. Confirm that the data migration is complete and wait for the forward synchronization latency to catch up.

2. Data Transmission automatically adds check constraints, foreign key constraints, and other objects that were ignored during schema migration when the destination is Oracle or OceanBase in Oracle compatibility mode.

3. Data Transmission automatically deletes auxiliary hidden columns and unique indexes that were added for migration dependency.

   This applies only to migration tasks between Oracle and OceanBase, or between OceanBase databases. For more information, see Hidden column mechanism in Data Migration Service.

4. Manually migrate unsupported database objects, such as triggers, functions, and stored procedures, from the source to the destination.

5. If the migration task includes reverse incremental synchronization, disable triggers and delete foreign key constraints on the source.

Forward switchover steps:

1. Start Forward Switchover

   This step confirms the start of forward operations but performs no actual backend actions. After you confirm that the data migration is complete and you are ready for the business cutover, click Start Forward Switchover to begin the process.

   Important

   Before you start the forward switchover, ensure that the source is in a no-write state.

2. Switchover Precheck

   This pre-switchover check verifies the following:

   • The latency between the source and destination. The check passes if the latency is within 15 seconds.

   • The write permissions of the source account. If reverse incremental synchronization is involved, this check verifies that the configured source account has write permissions to ensure that the reverse synchronization works.

   • The incremental read permissions of the destination account. If reverse incremental synchronization is involved, this check verifies that the configured destination account has read permissions to ensure that the reverse synchronization works.

   • The incremental log configuration of the destination. If reverse incremental synchronization is involved, this check verifies that the destination incremental logs meet the extraction requirements.

   If the switchover precheck passes, Data Transmission Service automatically executes the next operation. If the switchover precheck fails, Data Transmission Service provides the Retry and Skip operations.

   Important

   Skipping this check may cause data loss or reverse incremental synchronization failure. Proceed with caution.

3. Start Destination Store

   Note

   This step appears only for tasks with reverse incremental synchronization.

   If the precheck passes, Data Transmission automatically starts the incremental log pull service on the destination to capture, parse, and store DML/DDL operations in preparation for reverse incremental synchronization. This takes about 3 to 5 minutes.

4. Confirm Source Write Stop

   This step confirms that there are no ongoing write operations to the source. If this is confirmed, click Confirm to proceed.

5. Confirm Sync Catches Up to Write-Stop Offset

   This step checks whether the destination sync offset has caught up to the source write-stop offset. If the task is stuck in the Running or Failed state for a long time, click Skip.

   Important

   Skipping this step may cause data inconsistency between the source and destination. Proceed with caution.

6. Stop Forward Sync

   This step stops the forward synchronization service. After the service is stopped, any database changes that are generated at the source for this task are no longer synchronized to the destination. If the service fails to stop, Data Transmission Service provides the Retry or Skip operations.

   Important

   Use the skip option only if you can confirm that the forward synchronization service has already completed processing in the background. Otherwise, unexpected source data may be written to the destination.

7. Process Database Objects

   This step handles objects that were ignored during migration or are unsupported by Data Transmission to ensure a smooth business cutover to the destination.

   • Migrate database objects to the destination: You must manually migrate unsupported objects, such as triggers, functions, and stored procedures, from the source to the destination. After you complete this, click Mark as Complete.

   • Disable source triggers and foreign key constraints: This is required only for tasks with reverse incremental synchronization to prevent synchronization failures. You must perform this step manually and then click Mark as Complete.

   • Add ignored objects to the destination: This applies only when the destination is Oracle or OceanBase in Oracle compatibility mode. This step automatically adds check constraints, foreign key constraints, and other objects that were ignored during schema migration. Other destinations migrate these objects by default, requiring no extra action from you.

   • Delete auxiliary hidden columns and unique indexes: This applies only to migration between Oracle and OceanBase, or between OceanBase databases. This step automatically deletes hidden columns and unique indexes that were added by Data Transmission for consistency. The execution time depends on the data volume of the destination. A Skip option is provided during execution, but if you use it, you must delete them manually afterward. Proceed with caution. For more information, see Hidden column mechanism in Data Migration Service.

8. Start Reverse Incremental

   Note

   This step appears only for tasks with reverse incremental synchronization.

   This step starts incremental synchronization on the destination to synchronize DML/DDL changes from the destination to the source in real time. The configuration matches the initial task setup. For more information about incremental DDL, see the Incremental DDL documentation for your specific database.

Reverse incremental

For data migration tasks in the Running state, you can view the latency, current offset time, and reverse incremental performance in the reverse incremental section. The latency is displayed as 'X seconds (updated Y seconds ago)'. A Y value of less than 20 seconds is normal.

For data migration tasks in the Paused or Failed state, you can enable DDL/DML statistics to track database operations after enabling the feature. You can also view the details of reverse incremental synchronization objects and performance metrics.

• The Sync Object Statistics tab shows table-level DML counts. The totals for Changes, Delete, Insert, and Update at the top represent the sum of all DML operations for the current migration task and are the sum of the individual statistics on the Sync Object Statistics tab.

• The Reverse Incremental Performance tab displays:

  • Latency: The lag time in seconds for syncing incremental changes from the destination to the source.

  • Migration Traffic: The throughput in KB/s for syncing incremental change data from the destination to the source.

  • Average Execution Time: The mean execution time per SQL statement, in milliseconds.

  • Average Commit Time: The mean transaction commit time, in milliseconds.

  • RPS: The number of records processed per second.