ApsaraDB for OceanBase

Batch read

• ApsaraDB for OceanBase supports Oracle and MySQL tenant modes. When you configure the where clause for data filtering or function columns in the column parameter, ensure that the syntax complies with the SQL constraints of the corresponding tenant mode. Otherwise, the SQL statement may fail.

• You can read data from a view.

• During a batch read, do not modify the data being synchronized. This prevents data quality issues, such as data duplication or loss.

• If you configure the data source for Read by Partition, the account used to access the data source requires system permissions.

Batch write

• We recommend using the batch method to write data. This method sends a write request only when the number of accumulated rows reaches a predefined threshold.

• ApsaraDB for OceanBase supports Oracle and MySQL tenant modes. When you configure the preSql and postSql parameters, ensure that the syntax complies with the SQL constraints of the corresponding tenant mode. Otherwise, the SQL statement may fail.

Real-time read

• This feature supports only the OceanBase MySQL tenant mode.

• To synchronize real-time data, you must enable the binlog feature. For more information, see Binlog-related operations (Alibaba Cloud instances), Binlog-related operations (OB Cloud instances), or .

• Real-time full database synchronization tasks do not support data sources in connection string mode.

• For real-time full database synchronization tasks, the database version must be V3.0 or later.

• ApsaraDB for OceanBase is a distributed relational database that can integrate data from multiple physically distributed databases into a single logical database. However, when you synchronize data in real time from ApsaraDB for OceanBase to AnalyticDB for MySQL, you can only synchronize data from a single physical database. Data synchronization from a logical database is not supported.

Configure a whitelist

Add the CIDR block of the VPC where your serverless resource group or exclusive resource group for Data Integration resides to the whitelist of your OceanBase instance. For more information, see Configure an IP address whitelist.

Create an account and grant permissions

Create a database account with the required operation permissions on the OceanBase database. For more information, see Create an account.

Single-table batch synchronization

• Supported data sources: All data source types supported by Data Integration.

• Configuration guide: Configure a batch synchronization task

Single-table real-time synchronization

• Supported data source: Kafka

• Configuration guide: Configure a real-time full database synchronization task

Full-database real-time synchronization

• Supported data source: MySQL

• Configuration guide: Configure a real-time full database synchronization task

Configure a batch synchronization task by using the code editor

If you want to configure a batch synchronization task by using the code editor, you must configure the related parameters in the script based on the unified script format requirements. For more information, see Use the Code Editor. The following information describes the parameters that you must configure for data sources when you configure a batch synchronization task by using the code editor.

Reader script example

{
    "type": "job",
    "steps": [
        {
            "stepType": "apsaradb_for_OceanBase", // The plug-in name.
            "parameter": {
                "datasource": "", // The name of the data source.
                "where": "",
                "column": [ // The columns.
                    "id",
                    "name"
                ],
                "splitPk": ""
            },
            "name": "Reader",
            "category": "reader"
        },
        {
            "stepType": "stream",
            "parameter": {
                "print": false,
                "fieldDelimiter": ","
            },
            "name": "Writer",
            "category": "writer"
        }
    ],
    "version": "2.0",
    "order": {
        "hops": [
            {
                "from": "Reader",
                "to": "Writer"
            }
        ]
    },
    "setting": {
        "errorLimit": {
            "record": "0" // The number of dirty data records allowed.
        },
        "speed": {
            "throttle": true, // If set to false, the mbps parameter does not take effect and no throttling is applied. If set to true, throttling is enabled.
            "concurrent": 1, // Concurrency.
            "mbps":"12"// Throttling rate. 1 mbps = 1 MB/s.
        }
    }
}

Reader script parameters

Writer script example

{
    "type":"job",
    "version":"2.0",// Version number.
    "steps":[
        {
            "stepType":"stream",
            "parameter":{},
            "name":"Reader",
            "category":"reader"
        },
        {
            "stepType":"apsaradb_for_OceanBase",// The plug-in name.
            "parameter":{
                "datasource": "The data source name",
                "column": [// The columns.
                    "id",
                    "name"
                ],
                "table": "apsaradb_for_OceanBase_table",// The table name.
                "preSql": [ // The SQL statement to be executed before the data synchronization task runs.
                    "delete from @table where db_id = -1"
                ],
                "postSql": [// The SQL statement to be executed after the data synchronization task runs.
                    "update @table set db_modify_time = now() where db_id = 1"
                ],
                "obWriteMode": "insert",
            },
            "name":"Writer",
            "category":"writer"
        }
    ],
    "setting":{
        "errorLimit":{
            "record":"0"// The number of dirty data records allowed.
        },
        "speed":{
            "throttle":true,// If set to false, the mbps parameter does not take effect and no throttling is applied. If set to true, throttling is enabled.
            "concurrent":1, // Concurrency.
            "mbps":"12"// Throttling rate. 1 mbps = 1 MB/s.
        }
    },
    "order":{
        "hops":[
            {
                "from":"Reader",
                "to":"Writer"
            }
        ]
    }
}

Writer script parameters