Data Transmission Service is a feature of OceanBase Database that enables data interaction between OceanBase Database and homogeneous or heterogeneous data sources. It supports online migration of historical data and real-time synchronization of incremental data.
Features
Data Transmission Service provides a visual, centralized management platform. You can configure it easily to migrate data in real time. This service helps you migrate or synchronize data from homogeneous or heterogeneous databases to OceanBase Database with low risk, low cost, and high efficiency.
Data migration: Data migration is a one-time task. Release task resources after the migration completes. Use this feature to migrate data between homogeneous or heterogeneous data sources. It applies to scenarios such as database upgrades, cross-instance data migration, database sharding, and scale-out.
A data migration task is the basic unit of the data migration feature. When creating a data migration task, you can specify a migration scope as broad as the database level or as narrow as the table level. For more information, see Data migration.
NoteData migration tasks currently support only the pay-as-you-go billing model. The service is free during a limited-time trial period. Billing will begin at a later date, which will be announced separately. For more information, see Data Transmission Service billing.
Data synchronization: Data synchronization is a continuous action. After you create a sync task, it continuously synchronizes data to maintain data consistency between the source and destination and enable real-time data flow for critical business operations. Use this feature to achieve real-time data synchronization between data sources. It applies to scenarios such as active geo-redundancy, geo-disaster recovery, data aggregation, and real-time data warehouses. For more information, see Data synchronization.
NoteData synchronization tasks currently support only the subscription billing model (annual or monthly). For more information, see Data Transmission Service billing.
Supported migration types
Abbreviation guide
Instance type | Abbreviation |
RDS instance | RDS |
PolarDB MySQL instance | PolarDB |
Self-managed database in VPC | VPC |
Self-managed database with public IP | Public network |
OceanBase Database in MySQL-compatible mode | OB_MySQL |
OceanBase Database in Oracle-compatible mode | OB_Oracle |
OceanBase Database Serverless instance in MySQL-compatible mode | OB_MySQL Serverless instance |
OceanBase Database Serverless instance in Oracle-compatible mode | OB_Oracle Serverless instance |
Data migration
Migrating data from an Oracle database to OceanBase Database in MySQL-compatible mode, or from a MySQL database to OceanBase Database in Oracle-compatible mode, is not supported.
For the supported scope of incremental DDL operations, see Supported DDL operations and limits.
Data migration task | Schema migration | Full migration | Incremental DML | Incremental DDL | Full validation | Reverse incremental | Table without primary key |
MySQL (RDS/PolarDB/VPC/public network) → OB_MySQL (cluster instance/Serverless instance/tenant instance) | Supported | Support | Supported | Supported | Supported | Supported | Supported |
OB_MySQL (cluster instance/Serverless instance) → MySQL (RDS/PolarDB/VPC/public network) | Supported | Supported | Supported | Supported | Supported | Supported | Supported |
OB_MySQL (cluster instance/VPC/Serverless instance) → OB_MySQL (cluster instance/Serverless instance) | Supported | Supported | Supported | Supported | Supported | Supported | Supported |
OB_MySQL (cluster instance/Serverless instance) → OB_MySQL (VPC) | Supported | Supported | Supported | Supported | Supported | Support | Supported |
Oracle (public network/VPC) → OB_Oracle (cluster instance) | Supported | Supported | Supported | Supported | Supported | Supported | Supported |
OB_Oracle (cluster instance) → Oracle (VPC) | Supported | Supported | Supported | Supported | Supported | Supported | Supported |
OB_Oracle (cluster instance/VPC) → OB_Oracle (cluster instance) | Supported | Supported | Supported | Supported | Supported | Supported | Supported |
OB_Oracle (cluster instance) → OB_Oracle (VPC) | Supported | Supported | Supported | Supported | Supported | Supported | Supported |
OB_Oracle (cluster instance) → MySQL (RDS/PolarDB/VPC/public network) | Not supported | Not supported | Supported | Not supported | Not supported | Not supported | Supported |
TiDB → OB_MySQL (cluster instance/Serverless instance) | Supported | Supported | Supported | Not supported | Supported | Supported | Supported |
PolarDB-X 1.0 → OB_MySQL (cluster instance/Serverless instance) | Not supported | Supported | Supported | Not supported | Supported | Not supported | Supported |
PolarDB-X 2.0 → OB_MySQL (cluster instance/Serverless instance) | Supported | Supported | Supported | Not supported | Support | Supported | Supported |
PostgreSQL (RDS instance) → OB_Oracle (cluster instance/VPC) | Supported | Supported | Supported | Not supported | Supported | Supported | Supported |
Migration type descriptions
Migration type | Description |
Schema migration | Migrates definitions of database objects (such as tables, indexes, constraints, comments, and views) from the source database to the destination database. It automatically filters out temporary tables. If the source database is not OceanBase Database, Data Transmission Service automatically converts data types and SQL syntax according to the syntax standards of the target OceanBase tenant type, then replicates them to the destination database. |
Full migration | Migrates historical data from source tables to corresponding destination tables. On the Full migration page, filter objects by source and destination databases, or check Show problematic objects to identify objects blocking overall migration progress. You can also view Table objects, Table indexes, and Full migration performance. Full migration completes only after both table objects and table indexes finish migrating. |
Incremental synchronization | After an incremental sync task starts, it synchronizes changed data (inserts, updates, or deletes) from the source database to corresponding destination tables. If the source database continuously receives write operations, Data Transmission Service starts the incremental pull module before full data migration begins. This module pulls incremental updates from the source instance, parses and encapsulates them, and stores them in Data Transmission Service. After full data migration completes, Data Transmission Service starts the incremental replay module to retrieve incremental data from the pull module. The data undergoes filtering, mapping, and transformation before syncing to the destination instance. |
Full validation | After full data migration completes and incremental data nearly catches up with the source, Data Transmission Service automatically starts a full data validation task for configured source and destination tables. You can also start custom data validation during incremental sync. On the Full validation page, view overall status, start time, end time, total duration, estimated total rows, migrated rows, real-time traffic, and records per second (RPS). |
Forward switchover | Forward switchover (an abstracted and standardized version of traditional system cutover) does not switch application connections. It is a required workflow that coordinates with your application switchover. Complete all forward switchover steps before switching your application connections to the destination. Forward switchover is essential for data migration. It ensures Data Transmission Service completes all forward migration tasks and lets you start reverse incremental components based on business needs. Key steps include the following:
|
Reverse incremental | After migration completes, for business cutover scenarios, start a reverse incremental sync task (from destination to source) in Data Transmission Service before switching your business database. This task streams changes made to the destination database after switchover back to the original source database in real time. |
Data synchronization
For the supported scope of incremental DDL operations, see the documentation for creating data synchronization tasks.
Data sync task | Schema sync | Full sync | Incremental DML | Incremental DDL | Table without primary key |
OB_MySQL (cluster instance) → OB_MySQL (cluster instance/VPC/Serverless instance) | Supported | Supported | Supported | Supported | Supported |
OB_MySQL (VPC) → OB_MySQL (cluster instance/Serverless instance) | Supported | Supported | Supported | Supported | Supported |
OB_MySQL (cluster instance/Serverless instance) → ADB (Alibaba Cloud instance) | Supported | Supported | Supported | Supported | Not supported |
OB_Oracle (cluster instance/VPC) → OB_Oracle (Serverless instance) | Supported | Supported | Supported | Supported | Supported |
OB_MySQL (cluster instance/VPC) → OB_Oracle (cluster instance/VPC) | Supported | Supported | Supported | Supported | Supported |
OB_Oracle (cluster instance/VPC) → OB_MySQL (cluster instance/VPC) | Supported | Supported | Supported | Supported | Supported |
PostgreSQL (RDS) → OB_Oracle (cluster instance) | Supported | Supported | Supported | Not supported | Supported |
OB_MySQL (cluster instance/Serverless instance) → DataHub (public network/Alibaba Cloud instance) | Supported | Supported | Supported | Supported | Supported |
OB_Oracle (cluster instance) → DataHub (public network/Alibaba Cloud instance) | Supported | Supported | Supported | Supported | Supported |
OB_MySQL (cluster instance/Serverless instance) → Kafka (public network/VPC/Alibaba Cloud instance) | Supported | Supported | Supported | Supported | Supported |
OB_Oracle (cluster instance) → Kafka (public network/VPC/Alibaba Cloud instance) | Not supported | Supported | Supported | Supported | Supported |
OB_MySQL (cluster instance/Serverless instance) → RocketMQ (Alibaba Cloud instance) | Not supported | Supported | Supported | Supported | Supported |
OB_Oracle (cluster instance) → RocketMQ (Alibaba Cloud instance) | Not supported | Supported | Supported | Not supported | Supported |
Supported database versions
Transmission feature | OceanBase Database version | Other endpoint versions |
Data migration | V1.4.79, V2.2.30, V2.2.52, V2.2.76, V2.2.77, V3.1.x, V3.2.x, V4.0.x, V4.1.x, V4.2.1.10, V4.2.2.1, V4.2.3.0, V4.2.4.0, V4.2.5.0, V4.3.0.1, V4.3.1.0, V4.3.2.1, V4.3.3.1, V4.3.4.1, V4.3.5.0 |
|
Data synchronization | V1.4.79, V2.2.30, V2.2.52, V2.2.76, V2.2.77, V3.1.x, V3.2.x, V4.0.x, V4.1.x, V4.2.1.10, V4.2.2.1, V4.2.3.0, V4.2.4.0, V4.2.5.2, V4.3.0.1, V4.3.1.0, V4.3.2.1, V4.3.3.1, V4.3.4.1, V4.3.5.0 |
|