DocsICP FilingConsole
官方文档
首页

Migrate data from an Oracle database to an OceanBase Oracle tenant

更新时间：
复制 MD 格式
产品详情
我的收藏

This topic describes how to use Data Transmission Service (DTS) to migrate data from an Oracle database to an OceanBase Oracle tenant.

Important

If a data migration task remains inactive for an extended period (with a status of Failed, Paused, or Completed), it may not be resumable due to factors such as the retention period of incremental logs. To reclaim resources, Data Transmission Service releases data migration tasks that have been inactive for more than three days. We recommend that you configure alerts for your tasks and handle any exceptions promptly.

Prerequisites

  • Data Transmission Service (DTS) has the required permissions to access cloud resources. For more information, see Authorize DTS to perform data migration.

  • You have created dedicated database users for data migration in the source Oracle database and the destination OceanBase Oracle tenant and granted the required permissions to them. For more information, see Create database users.

  • You have created the corresponding database in the destination OceanBase Oracle tenant. The data migration feature of OceanBase supports migrating tables and columns, but not databases. Therefore, you must create the database in the destination tenant in advance.

  • Before incremental data replication, the Oracle source instance must have Archive Log enabled and a LogFile switch must have occurred.

  • The Oracle source instance must have the LogMiner tool installed and enabled.

    The LogMiner tool lets you obtain the content of Oracle archived redo log files.

  • Confirm that database-level or table-level supplemental logging is enabled for the Oracle instance.

  • Enabling database-level supplemental logging for primary keys (PKs) and unique keys (UKs) can add pressure to the LogMiner Reader and the Oracle database. This occurs because tables that do not require synchronization generate many unnecessary logs. For this reason, DTS supports enabling only table-level supplemental logging for PKs and UKs. However, if you configure extract, transform, and load (ETL) filtering on non-PK or non-UK columns, you must enable supplemental logging for those columns or for all columns.

  • You must synchronize the clocks between the Oracle machine and the data transmission machine, for example, by configuring an NTP service. Otherwise, your data may be at risk. If you use Oracle RAC, you must also synchronize the clocks among the multiple Oracle instances.

Limits

  • Source database operation limits

    Do not perform DDL operations that alter the database or table structure during schema migration and full migration. This can interrupt the data migration task.

  • DTS supports Oracle Database versions 10G, 11G, 12C, 18C, and 19C. Versions 12C and later include Container Database (CDB) and Pluggable Database (PDB).

  • DTS supports migrating only standard tables and views.

  • DTS supports migrating only objects with database, table, and column names that use ASCII characters and do not contain special characters. Special characters include line breaks, spaces, and characters such as .|"'`()=;/&\.

  • DTS does not support triggers at the destination. The presence of triggers at the destination may cause the data migration to fail.

  • DTS does not support migrating index-organized tables (IOTs) from Oracle databases. Doing so will interrupt the data migration task.

  • Data type limits

    • Incremental data synchronization is not supported for tables where all columns are of LOB types such as BLOB, CLOB, or NCLOB.

    • For tables without a primary key that contain LOB-type fields, reverse incremental synchronization will cause data quality issues.

  • Data source identifiers and user accounts must be globally unique within the Data Transmission Service system.

  • The incremental log parsing for Oracle databases supports a maximum of 5 TB per day.

  • Oracle Database 11G and earlier versions do not support creating database objects with names longer than 30 bytes. During the reverse incremental step, do not create database objects that exceed this limit in the OceanBase Oracle tenant.

  • DTS does not support migrating database objects, including schemas, tables, and columns, with names longer than 30 bytes from Oracle Database 12C and later versions. To migrate objects with names that exceed 30 bytes, contact the helpdesk.

  • DTS does not support certain UPDATE commands on the source Oracle database. The following is an example of an unsupported UPDATE command.

     UPDATE TABLE_NAME SET KEY=KEY+1;

    In the preceding example, TABLE_NAME is the table name, and KEY is a column of the NUMERIC type defined as the primary key.

  • The VPC where the self-managed database is located must be in the same region as the OceanBase database.

Notes

  • When you perform incremental synchronization for an Oracle database, ensure that each archived object is smaller than 2 GB.

  • Retain Oracle archived objects for at least two days. To recover from a sudden spike in archived data, the required objects may no longer be available, which prevents recovery.

  • If the source Oracle database contains DML statements that swap primary keys, DTS will parse the logs abnormally, leading to data loss when migrating to the destination. An example of a DML statement that swaps primary keys is as follows:

    update test set c1=(case when c1=1 then 2 when c1=2 then 1 end) where c1 in (1,2);

  • The character set of the Oracle instance can be AL32UTF8, AL16UTF16, ZHS16GBK, or GB18030.

    If the source character set is UTF-8, use a compatible character set for the destination, such as UTF-8 or UTF-16. This helps prevent issues such as garbled text at the destination due to character set incompatibility.

  • When migrating data from an Oracle database to an OceanBase Oracle tenant, do not perform any operations that change the ROWID on any table. Such operations include import, export, ALTER TABLE, FLASHBACK TABLE, and partition splitting or merging.

    When operations that affect ROWID behavior, such as updating partition keys or merging partitions, occur in the source Oracle database, the destination OceanBase Oracle tenant adds a hidden column that depends on the ROWID. This can lead to a risk of data loss.

  • Clock skew between nodes, or between a client terminal and a server, can cause inaccurate latency times for incremental synchronization or reverse incremental synchronization.

    For example, if a clock is ahead of the standard time, the latency may appear as a negative value. If a clock is behind the standard time, the latency may appear longer than it is.

  • Due to the historical use of daylight saving time (DST) in China, a one-hour time difference may exist between the source and destination for the TIMESTAMP(6) WITH TIME ZONE data type during incremental synchronization from an Oracle database to an OceanBase Oracle tenant. This can occur on the start and end dates of DST from 1986 to 1991, and from April 10 to April 17, 1988.

  • If DDL synchronization is not enabled and you change a unique index on the destination, you must restart the data migration task. Otherwise, data inconsistency may occur.

  • If forward switchover is not enabled for the data migration task, you must delete the corresponding unique indexes and pseudo-columns from the destination database. If you do not delete them, data cannot be written to the destination, and new pseudo-columns will be generated when you import data downstream. This causes conflicts with the pseudo-columns in the source database.

    If forward switchover is enabled for the data migration task, DTS automatically deletes hidden columns and unique indexes based on the task type. For more information, see Description of the hidden column mechanism in Data Transmission Service.

  • For incremental synchronization from an Oracle database to an OceanBase Oracle tenant, if a new table without a primary key is added for migration, DTS does not automatically delete the hidden columns and unique indexes added to the destination. You must delete them manually before performing reverse migration.

    You can check the logs/msg/manual_table.log file to identify the tables without primary keys that were added during the incremental synchronization phase.

  • If the source and destination have different character encoding settings, schema migration provides a policy to expand the field length definition. For example, the field length can be increased by 1.5 times, and the length unit can be changed from BYTE to CHAR.

    After conversion, data from different character sets in the source can be successfully migrated to the destination. However, after the cutover, reverse incremental synchronization may fail if data exceeds the length limit and cannot be written back to the source.

  • If the source contains data types with time zone information, such as TIMESTAMP WITH TIME ZONE, ensure that the destination database supports the corresponding time zone from the source. Otherwise, data inconsistency may occur during data migration.

  • In a database and table aggregation scenario:

    • You must use matching rules to map the relationship between the source and the destination.

    • You must create the table schema at the destination. If you use DTS to create the schema, you must skip any failed objects during the schema migration step.

  • Check the objects in the Oracle database recycle bin. If the number of objects is greater than 100, a timeout may occur when querying internal tables. You must clean up the objects in the recycle bin.

    • You can check whether the recycle bin is enabled.

      SELECT Value FROM V$parameter WHERE Name = 'recyclebin';

    • Check the number of objects in the recycle bin.

      SELECT COUNT(*) FROM RECYCLEBIN;

  • If you configure only Incremental Synchronization when you create the data migration task, DTS requires that the archived logs of the source database be retained for more than 48 hours.

    If you configure Full Migration and Incremental Synchronization, DTS requires that the archived logs of the source database be retained for at least seven days. Otherwise, DTS may fail to obtain incremental logs, which causes the data migration task to fail or even leads to data inconsistency between the source and destination.

  • If the source or destination contains table objects whose names differ only in case, the data migration result may not be as expected. This can happen if the source or destination is case-insensitive.

  • When the destination is an OceanBase database MySQL tenant of V4.1.0 or later, migration of the Latin1 character set is supported.

  • For incremental synchronization tasks that use an Oracle database as the source (excluding tasks that obtain incremental data through Kafka), if a single transaction spans multiple archived logs, LogMiner may not return complete data. This can lead to data loss. We recommend that you configure full data validation and data correction to ensure data consistency.

Supported source and destination instance types

In the following table, an Oracle tenant of an OceanBase database is abbreviated as OB_Oracle.

Source

Destination

Oracle (self-managed database in a VPC)

OB_Oracle (OceanBase cluster instance)

Oracle (self-managed database with a public IP address)

OB_Oracle (OceanBase cluster instance)

Data type mapping

Important

  • CLOB and BLOB data must be smaller than 48 MB.

  • Migration of ROWID, BFILE, XMLType, UROWID, UNDEFINED, and UDT data types is not supported.

  • Incremental synchronization is not supported for tables with LONG or LONG RAW data types.

Oracle database

OceanBase Oracle tenant

CHAR(n CHAR)

CHAR(n CHAR)

CHAR(n BYTE)

CHAR(n BYTE)

NCHAR(n)

NCHAR(n)

VARCHAR2(n)

VARCHAR2(n)

NVARCHAR2(n)

NVARCHAR2(n)

NUMBER(n)

NUMBER(n)

NUMBER (p, s)

NUMBER(p,s)

RAW

RAW

CLOB

CLOB

NCLOB

NVARCHAR2

Note

In an OceanBase Oracle tenant, NVARCHAR2 fields do not support null values. If a null value exists in the source, it is represented as the string "NULL".

BLOB

BLOB

REAL

FLOAT

FLOAT(n)

FLOAT

BINARY_FLOAT

BINARY_FLOAT

BINARY_DOUBLE

BINARY_DOUBLE

DATE

DATE

TIMESTAMP

TIMESTAMP

TIMESTAMP WITH TIME ZONE

TIMESTAMP WITH TIME ZONE

TIMESTAMP WITH LOCAL TIME ZONE

TIMESTAMP WITH LOCAL TIME ZONE

INTERVAL YEAR(p) TO MONTH

INTERVAL YEAR(p) TO MONTH

INTERVAL DAY(p) TO SECOND

INTERVAL DAY(p) TO SECOND

LONG

CLOB

Important

This type does not support incremental synchronization.

LONG RAW

BLOB

Important

This type does not support incremental synchronization.

SDO_GEOMETRY

SDO_GEOMETRY

Check and modify the system configuration of the Oracle instance

You must perform the following operations:

  1. Enable ARCHIVELOG mode on the source Oracle database

  2. Enable supplemental logging on the source Oracle database

  3. (Optional) Set system parameters for the Oracle database

Enable ARCHIVELOG mode on the source Oracle database

SELECT log_mode FROM v$database;

The value of the log_mode field must be archivelog. If it is not, modify it as follows:

  1. You can execute the following commands to enable ARCHIVELOG mode.

    SHUTDOWN IMMEDIATE;
STARTUP MOUNT;
ALTER DATABASE ARCHIVELOG;
ALTER DATABASE OPEN;

  2. You can execute the following command to view the path and quota of the archived logs.

    Check the path and quota of the recovery file. Set db_recovery_file_dest_size to a large value. After you enable archiving, you must periodically clean up archived logs using tools such as Recovery Manager (RMAN).

    SHOW PARAMETER db_recovery_file_dest;

  3. You can change the quota for archived logs as needed.

    ALTER SYSTEM SET db_recovery_file_dest_size =50G SCOPE = BOTH;

Enable supplemental logging on the source Oracle database

The LogMiner Reader supports Oracle system configurations where only table-level supplemental logging is enabled. If a new table is created in the source Oracle database during migration and needs to be migrated, you must enable supplemental logging for its PK and UK before you perform DML operations. Otherwise, DTS reports an incomplete log error.

To prevent issues such as inconsistent indexes between the source and destination, unexpected ETL behavior, and reduced migration performance for partitioned tables, add the following supplemental logs:

  • Add database-level or table-level supplemental_log_data_pk andsupplemental_log_data_ui.

  • Add specific columns to the supplemental log

    • Add all columns that are part of PKs and UKs at both the source and destination to resolve inconsistent index issues.

    • If you use ETL, add the ETL columns. This resolves issues with unexpected ETL behavior.

    • If the destination is a partitioned table, add the partition key columns. This resolves the issue where partition pruning cannot be performed, which reduces write performance.

    You can execute the following statement to check the result.

    SELECT log_group_type FROM all_log_groups WHERE OWNER = '<schema_name>' AND table_name = '<table_name>';

    The check passes if the query result includes ALL COLUMN LOGGING. If it does not, check whether the ALL_LOG_GROUP_COLUMNS table contains the union of all the preceding columns.

    The following is an example of how to add specific columns to the supplemental log:

    ALTER TABLE <table_name> ADD SUPPLEMENTAL LOG GROUP <table_name_group> (c1, c2) ALWAYS;

The following table describes the risks and solutions for DDL operations during a data migration task.

Operation

Threat

Solution

CREATE TABLE (Requires synchronization)

If the destination table is a partitioned table, the source and destination indexes are inconsistent, or ETL is required, data migration performance may be affected and ETL behavior may be unexpected.

You must enable database-level supplemental logging for PKs and UKs. Manually add the involved columns to the supplemental log.

Add, delete, or modify a PK, UK, or partition key column, or modify an ETL column

The rules for adding supplemental logs at startup will not be met. This may cause data inconsistency or reduced data migration performance.

Add supplemental logs according to the preceding rules.

The LogMiner Reader performs checks in the following two ways. If the LogMiner Reader detects that supplemental logging is not enabled, it exits.

  • Enable database-level supplemental_log_data_pk and supplemental_log_data_ui.

    You can execute the following command to check whether supplemental logging is enabled. If both query results are YES, supplemental logging is enabled.

    SELECT supplemental_log_data_pk, supplemental_log_data_ui FROM v$database;

    If it is not enabled, you must perform the following operations:

    1. You can execute the following statement to enable supplemental logging.

      ALTER DATABASE ADD supplemental log DATA(PRIMARY KEY, UNIQUE) columns;

    2. After this is enabled, switch the archived logs twice and wait for more than 5 minutes before starting the task. If you are using Oracle RAC, switch between the instances.

      ALTER SYSTEM SWITCH LOGFILE;

      In an Oracle RAC environment, if you switch one instance multiple times before you switch to another instance instead of alternating between instances, the instance that is switched later may locate a log from before supplemental logging was enabled when it determines the starting pull file.

  • Enable table-level supplemental_log_data_pk and supplemental_log_data_ui.

    1. You can execute the following statement to confirm that database-level supplemental_log_data_min is enabled.

      SELECT supplemental_log_data_min FROM v$database;

      A result of YES or IMPLICIT indicates that supplemental logging is enabled.

    2. You can execute the following statement to confirm that table-level supplemental logging is enabled for the tables to be synchronized.

      SELECT log_group_type FROM all_log_groups WHERE OWNER = '<schema_name>' AND table_name = '<table_name>';

      Each type of supplemental log returns one row. The result must include ALL COLUMN LOGGING, or both PRIMARY KEY LOGGING and UNIQUE KEY LOGGING.

      If table-level supplemental logging is not enabled, you can execute the following statement.

      ALTER TABLE table_name ADD SUPPLEMENTAL LOG DATA (PRIMARY KEY, UNIQUE) COLUMNS;

    3. After you enable it, switch the archived logs twice and wait for more than 5 minutes before you start the task. For an Oracle RAC, alternate the switches between the instances.

      ALTER SYSTEM SWITCH LOGFILE;

Set system parameters for the Oracle database (Optional)

Set the _log_parallelism_max system parameter of the Oracle database to 1. The default value is usually 2.

  1. Query the value of _log_parallelism_max. You can query the value in one of the following two ways:

    • Method 1

      SELECT NAM.KSPPINM,VAL.KSPPSTVL,NAM.KSPPDESC FROM SYS.X$KSPPI NAM,SYS.X$KSPPSV VAL WHERE NAM.INDX= VAL.INDX AND NAM.KSPPINM LIKE '_%' AND UPPER(NAM.KSPPINM) LIKE '%LOG_PARALLEL%';

    • Method 2

      SELECT VALUE FROM v$parameter WHERE name = '_log_parallelism_max';

  2. Modify the value of _log_parallelism_max. The modification method varies based on your setup:

    • Modifying Oracle RAC

      ALTER SYSTEM SET "_log_parallelism_max" = 1 SID = '*' SCOPE = spfile;

    • Modifications for non-Oracle RAC

      ALTER SYSTEM SET "_log_parallelism_max" = 1 SCOPE = spfile;

    If you receive the write to SPFILE requested but no SPFILE specified at startup error when you modify the _log_parallelism_max parameter in Oracle Database 10G, you must perform the following operations:

    CREATE SPFILE FROM PFILE;
SHUTDOWN IMMEDIATE;
STARTUP;
SHOW PARAMETER SPFILE;

  3. After you modify the _log_parallelism_max system parameter, you must restart the instance, switch the archived log twice, and wait for more than five minutes before you start the task.

Procedure

  1. Log on to the OceanBase Management Console and purchase a data migration task.

    For more information, see Purchase a data migration task.

  2. On the Data Transmission > Data Migration page, click Configure for your new data migration task.

    image.png

    To reference an existing task configuration, you can click Reference Configuration. For more information, see Reference a data migration task configuration.

  3. On the Select Source and Destination page, configure the parameters.

    Parameter

    Description

    Migration Task Name

    Use a combination of Chinese characters, digits, and letters. The name cannot contain spaces and must be no more than 64 characters long.

    Source

    If you have already created an Oracle data source, select it from the drop-down list. If not, click Create Data Source in the drop-down list to create one in the dialog box that appears. For more information about the parameters, see Create an Oracle data source.

    Destination

    If you have already created an OceanBase Oracle tenant data source, select it from the drop-down list. If not, click Create Data Source in the drop-down list to create one in the dialog box that appears. For more information about the parameters, see Create an OceanBase data source.

    Tags (Optional)

    Click the text box and select the target tag from the drop-down list. You can also click Manage Tags to create, modify, or delete tags. For more information, see Manage data migration tasks using tags.

  4. Click Next. On the Select Migration Type page, set Synchronization Topology to One-way Synchronization.

    Data Transmission Service supports One-way Synchronization and Two-way Synchronization. This topic describes the operations for a one-way synchronization task. For more information about two-way synchronization, see Configure a two-way synchronization task.

  5. Select the migration types for the data migration task.

    A one-way synchronization topology supports the following migration types: Schema Migration, Full Migration, Incremental Synchronization, Full Validation, and Reverse Incremental Synchronization.

    image

    Migration Type

    Description

    Schema Migration

    After the schema migration task starts, DTS migrates the definitions of data objects (such as tables, indexes, constraints, comments, and views) from the source database to the destination database. It automatically filters out temporary tables.

    Full Migration

    After the full migration task starts, DTS migrates the existing data from the source database tables to the corresponding tables in the destination database. If you select Full Migration, collect statistics information for the Oracle database using the GATHER_SCHEMA_STATS or GATHER_TABLE_STATS statement before migrating data.

    Incremental Synchronization

    After the incremental synchronization task starts, DTS synchronizes data changes (additions, modifications, or deletions) from the source database to the corresponding tables in the destination database.

    Incremental Synchronization includes DML Sync and DDL Sync, which you can configure as needed. For more information, see Customize DDL and DML synchronization. The following limits apply to Incremental Synchronization:

    • DDL synchronization is not supported in scenarios where multiple tables are aggregated into a single table.

    • If you select DDL Sync, the data migration task may be interrupted if a DDL operation that DTS does not support occurs in the source database.

    • If the DDL operation is to add a new column, set the column's attribute to Null. Otherwise, the data migration task may be interrupted.

    Full Validation

    After full migration is complete and the incremental data is synchronized to the destination and is nearly in sync with the source, DTS automatically starts a full data validation task for the configured data tables in the source and destination databases.

    • If you select Full Validation, collect statistics information for both the Oracle database and the OceanBase Oracle tenant before the validation starts.

    • If you select Incremental Synchronization but do not select all DML options under DML Sync, DTS does not support full data validation for this scenario.

    Reverse Incremental

    After the reverse incremental synchronization task starts, it can synchronize data changes from the destination database back to the source database in real time after a service cutover.

    Reverse incremental synchronization typically reuses the configuration of incremental synchronization, but you can also customize it as needed.

  6. Click Next. On the Select Migration Objects page, select the objects for the data migration task.

    You can select migration objects using two methods: Specify Objects and Matching Rules. This topic describes how to select migration objects using the Specify Objects method. For more information about configuring matching rules, see Configure and modify matching rules.

    Important

    • The names of the tables to be migrated and the columns in these tables cannot contain Chinese characters.

    • If a database name or table name contains the "$$" characters, the creation of the data migration task may fail.

    • If you select DDL Synchronization in the Select Migration Type step, we recommend using matching rules to select migration objects. This ensures that all new objects that match the rules are synchronized. If you select migration objects individually, new or renamed objects will not be synchronized.

    image.png

    1. In the Select Migration Objects section, select Specify Objects.

    2. In the Select Migration Scope area, in the Source Objects list, select the objects to migrate. You can select tables and views from one or more databases to migrate.

    3. Click > to add them to the Destination Objects list.

    Data Transmission Service lets you import objects from text files. It also lets you rename destination objects, set row filters, view column information, and remove a single object or all objects.

    Note

    When you select migration objects using the Matching Rules method, the matching rule syntax overrides the renaming capability, and the Actions section is limited to setting filter conditions. For more information, see Configure and modify matching rules.

    Operation

    Description

    Import Object

    1. In the list on the right of the selection area, click Import Object in the upper-right corner.

    2. In the dialog box that appears, click OK.

      Important

      The import operation overwrites previous selections. Proceed with caution.

    3. In the Import Migration Objects dialog box, import the objects to be migrated.

      You can import a CSV file to rename database tables, set row filtering conditions, and perform other operations. For more information, see Download and import migration object configurations.

    4. Click Check Validity.

      After you import the migration objects, check their validity first. Column field mapping is not currently supported.

    5. After the check is passed, click OK.

    Rename

    Data Transmission Service lets you rename migration objects. For more information, see Rename database and table objects.

    Settings

    Data Transmission Service supports row filtering using WHERE conditions. For more information, see Filter data with SQL conditions.

    You can also view the column information of migration objects in the View Columns area.

    Remove/Remove All

    Data Transmission Service lets you remove one or more objects that have been temporarily selected for the destination during data mapping.

    • Remove a single migration object

      In the list on the right of the selection area, hover over the target object and click the displayed Remove button to remove the migration object.

    • Remove all migration objects

      In the list on the right of the selection area, click Remove All in the upper-right corner. In the dialog box that appears, click OK to remove all migration objects.

  7. Click Next. Configure the parameters on the Migration Options page.

    • Full Migration

      On the Select Migration Type page, the following parameters appear only when you select One-way Synchronization > Full Migration.

      image

      Parameter

      Description

      Read Concurrency

      This parameter specifies the number of concurrent threads for reading data from the source during full migration. The maximum value is 512. A high degree of concurrency may put excessive pressure on the source and affect your services.

      Write Concurrency

      This parameter specifies the number of concurrent threads for writing data to the destination during full migration. The maximum value is 512. A high degree of concurrency may put excessive pressure on the destination and affect your services.

      Full Migration Rate Limit

      You can decide whether to enable the full migration rate limit as needed. If you enable it, set RPS (the maximum number of data rows that can be migrated to the destination per second during full migration) and BPS (the maximum volume of data that can be migrated to the destination per second during full migration).

      Note

      The RPS and BPS settings here only serve as a rate-limiting capability. The actual performance of full migration is affected by factors such as the source, destination, and instance specifications.

      Processing Policy for Existing Records in Destination Table Objects

      The processing policy includes Ignore and Stop Migration:

      • Select Ignore: If a destination table already contains data and a conflict occurs between the data being written and the existing data, Data Transmission Service logs the conflicting data and keeps the existing data unchanged.

        Important

        If you select Ignore, full data verification uses the IN pattern to pull data. This process cannot verify scenarios where data exists in the destination but not in the source, and verification performance is degraded to a certain degree.

      • If you select the default value Stop Migration, the full migration fails with an error if the destination table contains data. You must process the data in the destination table before you can restart the migration.

        Important

        If an error occurs and you click Resume, Data Transmission Service will ignore this configuration option and continue to migrate table data. Proceed with caution.

      Is deferred indexing supported?

      You can set whether to allow index creation after full data migration is complete. The postponed index creation feature can shorten the time required for full migration. For notes on selecting postponed index creation, see the description below the table.

      Important

      • On the Select Migration Type page, this parameter is displayed only if you select both Schema Migration and Full Migration.

      • Only non-unique indexes support postponed creation.

      • When creating an index, if the destination OceanBase database encounters the following errors, Data Transmission Service will ignore them and assume the index was created successfully, without attempting to create it again.

        • OceanBase database in MySQL-compatible mode reports a Duplicate key name error.

        • OceanBase database in Oracle-compatible mode reports a name is already used by an existing object error.

      If the destination is an OceanBase database and you select Allow, configure the following settings:

      • Concurrency for a single index DDL statement: A higher degree of parallelism consumes more resources and results in a faster migration speed.

      • Maximum number of concurrent index DDL statements: The maximum number of postponed index DDL statements that the system can call at the same time.

      If you allow postponed index creation, we recommend that you adjust the following business tenant parameters using a command-line tool according to the hardware of the OceanBase database and the current service traffic.

      // File memory buffer limit
alter system set _temporary_file_io_area_size = '10' tenant = 'xxx'; 
// V4.x Disable throttling
alter system set sys_bkgd_net_percentage = 100;

    • Incremental Synchronization

      The following parameters appear only when you select One-way Synchronization > Incremental Synchronization on the Select Migration Type page.

      image

      Parameter

      Description

      Write Concurrency

      This parameter specifies the number of concurrent threads for writing data to the destination during incremental synchronization. The maximum value is 512. A high degree of concurrency may put excessive pressure on the destination and affect your services.

      Incremental Synchronization Rate Limit

      You can decide whether to enable the incremental synchronization rate limit as needed. If you enable it, set RPS (the maximum number of data rows that can be synchronized to the destination per second during incremental synchronization) and BPS (the maximum volume of data that can be synchronized to the destination per second during incremental synchronization).

      Note

      The RPS and BPS settings here only serve as a rate-limiting capability. The actual performance of incremental synchronization is affected by factors such as the source, destination, and instance specifications.

      Incremental Synchronization Start Offset

      • If you select Full Migration, this parameter is not displayed.

      • If you select Incremental Synchronization but not Full Migration, specify a point in time from which to start migrating data. The default value is the current system time. For more information, see Configure an incremental synchronization offset.

    • Reverse Incremental

      On the Select Migration Type page, the parameters for this area are displayed only if you select One-way Synchronization > Reverse Incremental. The configuration parameters for Reverse Incremental default to Reuse Incremental Synchronization Configuration.

      image

      You can also choose not to reuse the incremental synchronization configuration and configure it manually.

      Parameter

      Description

      Write Concurrency

      This parameter specifies the number of concurrent threads for writing data to the source during reverse incremental synchronization. The maximum value is 512. A high degree of concurrency may put excessive pressure on the source and affect your services.

      Reverse Incremental Rate Limit

      You can decide whether to enable the reverse incremental rate limit as needed. If you enable it, set RPS (the maximum number of data rows that can be synchronized to the source per second during reverse incremental synchronization) and BPS (the maximum volume of data that can be synchronized to the source per second during reverse incremental synchronization).

      Note

      The RPS and BPS settings here only serve as a rate-limiting capability. The actual performance of reverse incremental synchronization is affected by factors such as the source, destination, and instance specifications.

      Incremental Synchronization Start Offset

      • If you select Full Migration, this parameter is not displayed.

      • If you do not select Full Migration but select Incremental Synchronization, the setting defaults to the forward switch (if any) and cannot be modified.

    • Advanced Options

      image

      Parameter

      Description

      Character Encoding and Length Definition Options

      This parameter is displayed only if you select Schema Migration on the Select Migration Type page and the source and destination character sets do not match.

      Note

      If the source and destination character sets are inconsistent (for example, GBK for the source and UTF-8 for the destination), field truncation and data inconsistency may occur.

      When you select Automatically expand destination field length, i.e., N BYTE → 1.5N BYTE, if the expanded length exceeds the maximum length, the field is set to the maximum length.

      Add Hidden Column for Tables Without Non-null Unique Keys

      When you migrate data from an Oracle database to an OceanBase Oracle tenant, you must configure whether to add a hidden column for tables that do not have a non-null unique key. For more information, see Description of the hidden column mechanism in Data Transmission Service.

      Storage Class for Destination Table Objects

      This area is displayed only if the destination OceanBase Oracle tenant is V4.3.0 or later, and you select either Schema Migration or Incremental Synchronization > DDL Sync on the Select Migration Type page.

      The storage classes for destination table objects include Default, Row Store, Column Store, and Mixed Row and Column Store. This setting determines the storage class of these objects during schema migration or incremental synchronization. For more information, see default_table_store_format.

      Note

      The behavior of the Default option is determined by the destination parameter configuration. Table objects from a schema migration or new table objects from incremental DDL are written with a structure that corresponds to the set storage class.

  8. Click Precheck to precheck the data migration task.

    In the Precheck step, Data Transmission Service checks whether items meet the requirements, such as the read and write permissions of the database user and the network connectivity of the database. You can start the data migration task only after all check items pass. If the precheck fails:

    • You can identify and troubleshoot the problem, and then run the precheck again until it succeeds.

    • You can also click Skip in the Actions column for a failed precheck item. A dialog box appears that describes the impact of skipping this operation. To proceed, click OK.

  9. After the precheck passes, click Start Task.

    If you do not want to start the task immediately, click Save. You can then start the task manually from the Data Migration Task List page or using batch operations. For more information about batch operations, see Batch Operations on Data Migration Tasks.

    Data Transmission Service lets you modify migration objects and their row filtering conditions while a data migration task is running. For more information, see View and modify migration objects and their filter conditions. After the data migration task starts, it performs the migration steps sequentially based on the selected migration types. For more information, see View migration details.

References

Previous：Migrate data from a MySQL database to an OceanBase database in MySQL-compatible modeNext：Migrate data from an OceanBase database in MySQL-compatible mode to a MySQL database