Offline Integration Input Supports Sharding for MySQL Data Sources

Scenario

Dataphin supports sharding to improve performance and integrates sharded data into Dataphin.

Solutions and Features

1. In the data integration component, you can read tables from multiple databases using enumeration, regex-like patterns, or a combination of both.

   • Enumeration:

     table_001, table_002, and so on: read the tables table_001 and table_002.

   • Regex-like patterns:

     • table_abc*: read all tables that start with the prefix table_abc.

     • table_[001-100]: read the 100 tables from table_001 to table_100.

   • Mixed mode: table_001; table_002; table_abc*; table_[001-100].

     

2. Click Exact Match. In the Confirm Match Details dialog box, preview the data source name, database name, and table name.

   

3. Each time you run the sync, it identifies the latest matching tables based on your lookup rules.