Configure the MySQL input component

Prerequisites

• A MySQL data source has been created. For more information, see Create a MySQL data source.

• Your account must have read permission on the data source. If you lack this permission, apply for it. For more information, see Apply for data source permissions.

Procedure

1. From the top menu bar of the Dataphin homepage, choose Develop > Data Integration.

2. From the top menu bar of the Data Integration page, select a Project. If you are in Dev-Prod mode, select an environment.

3. In the left-side navigation pane, click Batch Integration. In the Batch Integration list, click the desired batch pipeline to open its configuration page.

4. In the upper-right corner of the page, click Component Library to open the Component Library panel.

5. In the left-side navigation pane of the Component Library panel, select Input. In the input component list on the right, find the MySQL component and drag it onto the canvas.

6. Click the icon on the MySQL input component card to open the MySQL Input Configuration dialog box.

7. In the MySQL Input Configuration dialog box, configure the parameters.

   Parameter	Description
   Step Name	Specifies the name of the MySQL input component. Dataphin automatically generates a step name, but you can change it to suit your business needs. Follow these naming rules: The name can contain only Chinese characters, letters, underscores (_), and digits. The name cannot exceed 64 characters in length.
   Source Table Mode	Select a source table mode. The options are Single table and Multiple tables. Single table: Use this mode to synchronize data from one source table to one target table. Multiple tables: Use this mode to synchronize data from multiple source tables to a single target table. When data from multiple tables is written to the same target table, Dataphin uses the UNION algorithm.
   Data Source	The drop-down list displays all MySQL data sources in Dataphin, regardless of whether you have read permission. If you set Source Table Mode to Single table, you can select only one data source. Click the icon to copy the current data source name. If you set Source Table Mode to Multiple tables, you can select multiple data sources that share the same time zone. For data sources where you lack read permission, you can click Apply next to the data source to request the permission. For more information, see Apply for data source permissions. If you do not have a MySQL data source, click Create Data Source in the drop-down list to create one. For more information, see Create a MySQL data source.
   Time Zone	The job processes time-formatted data based on the specified time zone. By default, it uses the time zone configured for the selected data source. You cannot change it. Note For tasks created before V5.1.2, you can select Data Source Default or Channel Time Zone. The default option is Channel Time Zone. Data Source Default: The time zone of the selected data source. Channel Time Zone: The time zone configured for the current integration task under Properties > Channel Configuration.
   Database (Optional)	Select the database that contains the table. If you leave this field empty, Dataphin uses the database specified when the data source was registered. If you set Source Table Mode to Multiple tables, you can select multiple databases. Click the icon to view all selected databases in the Database List dialog box.
   Table Matching Method	You can select Wildcard Rule or Regular Expression. Note This setting can be configured only when Source Table Mode is set to Multiple tables.
   Table	Select the source table. If you set Source Table Mode to Single table, you can enter a keyword to search for the table, or enter the exact table name and click Exact Match. After you select a table, the system automatically checks its status. Click the icon to copy the name of the selected table. If you set Source Table Mode to Multiple tables, you can use different expressions to add tables based on the selected table matching method. Wildcard Rule: Enter a table expression in the input box to filter tables with the same structure. The system supports enumeration, regular expression-like, and mixed formats. For example, table_[001-100];table_102;table_abc*. The * wildcard character is valid only at the beginning or end of an expression. If you select Regular Expression, enter a regular expression supported by the current database. The system matches tables in the target database based on this expression. At runtime, the task dynamically matches and synchronizes the latest set of tables that match the regular expression. After you enter the expression, you can click Exact Match to view the list of matched tables in the Confirm Matching Details dialog box.
   Split Key (Optional)	The system uses the configured split key to partition data for parallel reading. This feature can be used in conjunction with the concurrency setting. You can use a column from the source table as the split key. For optimal performance, we recommend using a primary key or an indexed column as the split key. Important When you select a column of a date or time type, the system identifies the minimum and maximum values and performs a brute-force split based on the total time range and concurrency level. Even distribution is not guaranteed.
   Filter (Optional)	Specifies the filter conditions for data extraction. The following options are available: Set Fixed Value: Extracts the corresponding data. For example, ds=20211111. Variable Parameter Configuration: Extracts a subset of data. For example, ds=${bizdate}.
   Output Fields	This section displays all fields from the selected tables that match the filter criteria. You can perform the following operations: Manage fields: Delete fields that you do not need to output to downstream components. To delete a single field: Click the icon in the Actions column to delete the field. To delete multiple fields in bulk: Click Field Management. In the Field Management dialog box, select the fields you want to remove, click the left-arrow icon to move them to the unselected list, and then click OK. Bulk Add: Click Bulk Add to configure fields in bulk by using the JSON, TEXT, or DDL format. Note After you add fields in bulk and click OK, the new configuration overwrites the existing field configuration. To configure fields in bulk by using the JSON format, for example: // Example: [{ "index": 1, "name": "id", "type": "int(10)", "mapType": "Long", "comment": "comment1" }, { "index": 2, "name": "user_name", "type": "varchar(255)", "mapType": "String", "comment": "comment2" }] Note index specifies the column number, name specifies the name for the new field, and type specifies the data type for the new field. For example, "index":3,"name":"user_id","type":"String" specifies that the fourth column of the file is imported (column numbers are 0-based), the new field is named user_id, and its data type is String. To configure fields in bulk by using the TEXT format, for example: // Example: 1,id,int(10),Long,comment1 2,user_name,varchar(255),Long,comment2 The row delimiter separates each field definition. The default is a newline character (\n). Semicolons (;) and periods (.) are also supported. The column delimiter is used to separate field names from field types, and the default is a half-width comma (,). You can use','. The field type is optional, and the default is','. To configure fields in bulk by using the DDL format, for example: CREATE TABLE tablename ( user_id serial, username VARCHAR(50), password VARCHAR(50), email VARCHAR (255), created_on TIMESTAMP, ); Add a new output field: Click + Add Output Field, enter the Field, Type, and Description, and select a Mapping Type. After you configure the current row, click the icon to save.

8. Click OK to save the configuration for the MySQL input component.