Create an AnalyticDB for MySQL synchronization

Preparations

1.Prepare an AnalyticDB for MySQL instance and define a table schema. When you synchronize data from DataHub to an AnalyticDB for MySQL cluster, DataHub fields are mapped to the corresponding AnalyticDB for MySQL data types. The following table lists the mappings.

DataHub	MySQL
TINYINT	TINYINT
SMALLINT	SMALLINT
INTEGER	INT
BIGINT	BIGINT
STRING	VARCHAR
BOOLEAN	BOOLEAN / TINYINT
FLOAT	FLOAT
DOUBLE	DOUBLE
TIMESTAMP	TIMESTAMP / BIGINT
DECIMAL	DECIMAL

The TINYINT, SMALLINT, INTEGER, and FLOAT types in DataHub are supported in DataHub SDK for Java V2.16.1-public and later. 2.Synchronization description: (1) DataHub synchronizes only TUPLE-type data to AnalyticDB for MySQL. (2) ReplaceInto and IgnoreInto modes: In ReplaceInto mode, the REPLACE INTO statement overwrites existing data when a primary key conflict occurs. In IgnoreInto mode, the INSERT IGNORE INTO statement ignores conflicts and writes only non-duplicate data. (3) Data synchronization latency is typically in seconds. After data is written to DataHub, it is synchronized to AnalyticDB for MySQL within seconds. Network exceptions may cause duplicate data updates, but at-least-once delivery is guaranteed. (4) The write performance of AnalyticDB for MySQL affects synchronization performance. Poor instance performance may slow down synchronization and cause data to accumulate. In extreme cases, data may be lost if the synchronization latency exceeds the data lifecycle.

Create a DataConnector

In the left-side navigation pane of the DataHub console, click Project Manager. On the Project List page, find a project and click View in the Actions column. On the details page of the project, find a topic and click View in the Actions column.

The following parameters are required to create a DataConnector. For more information about synchronization configurations, see the DataHub SDK documentation.

1. Host: the endpoint of AnalyticDB for MySQL. To ensure that the service can be connected, you must enter the internal endpoint.

2. Import Fields: the fields to be synchronized to AnalyticDB for MySQL. You can synchronize all the fields of the DataHub topic or part of them based on your business requirements.

3. Write Mode: the write mode. Valid values:

   • IGNORE: This mode ignores duplicate data. The INSERT IGNORE INTO statement is executed to write data.

   • OVERWRITE: This mode updates duplicate data. The REPLACE INTO statement is executed to write data.

Example

1. Create an AnalyticDB for MySQL instance and a table with the defined table schema in the AnalyticDB for MySQL console.

2. Create a topic in the DataHub console. In this example, the created topic is of the TUPLE type.

The topic schema contains three fields:

• field1: STRING type, NOT NULL

• field2: BIGINT type, NOT NULL

• field3: DOUBLE type, NOT NULL

3. Create a DataConnector. Select IGNORE from the Write Mode drop-down list and all fields from the Import Fields drop-down list.

4. Write data of the TUPLE type to the created topic. The following table shows the four records that are written to the topic.

Sequence	field1	field2	field3
47502	f_1	1	1
47503	f_2	2	2
47504	f_1	1	1
47505	f_2	2	2

mysql> select * from  wurao_tuple_test_001;
+--------+--------+--------+
| field1 | field2 | field3 |
+--------+--------+--------+
| f_1    |      1 |   1.00 |
| f_2    |      2 |   2.00 |
| f_1    |      1 |   1.00 |
| f_2    |      2 |   2.00 |
+--------+--------+--------+
4 rows in set (0.00 sec)

mysql>

Synchronize data to AnalyticDB for MySQL V3.0

For more information, see the "Synchronize data to AnalyticDB for MySQL V3.0" section of the Synchronize data to ApsaraDB RDS, ApsaraDB RDS for MySQL, and AnalyticDB for MySQL V3.0 topic.