Data nodes

Create a data node

1. Log on to the Gold Security console. In the navigation pane on the left, click Data Nodes.

2. On the data node list page, click Create Data Node.

3. On the Create Data Node page, enter the following basic information:

   • Data Node Name: The name of the node. The name must be unique within the tenant. The name must be 1 to 100 characters long and can contain letters, digits, and underscores (_).

   • Creation Class: The class of the node. Supported classes are File, DB, and Offline. This setting cannot be changed after the node is created.

   • Description: Optional. A description of the node. The description can be 1 to 500 characters long.

4. Configure the node based on the Creation Class that you selected in the previous step.

   • File/DB

     • Data Source: Select a data source configuration. You can search for a configuration by entering a keyword from its name. If you have not created a data source configuration, click Create Data Source Configuration on the right. For more information, see Create a data source configuration.

     • Reconciliation Fields: The fields to use for reconciliation. For a File, these are fields in the file properties. For a DB, these are fields returned by an SQL query.

       Note

       If there are multiple reconciliation fields, separate them with commas (,).

     • Sort Field: The field used to sort the data. You can select one of the reconciliation fields that you entered.

     • Data Fetching Policy: You can select Fetch 1 Row at a Time or Fetch by Fixed Field.

       • Fetch 1 Row at a Time: During reconciliation, one row of data is retrieved from each side of the sorted data and then compared.

       • Fetch by Fixed Field: Fetches all data that has the same value in the fixed field at once.

         Important

         The fixed field must be the same as the sort field.

     • Hash Field: The field used for hash calculation during data sharding. If the data volume for reconciliation is large, you can select one of the reconciliation fields and hash its value based on specific digits. The retrieved data is then distributed to different shards based on the hash value. This method supports concurrent sorting and reconciliation, which improves efficiency.

       Note

       The hash field can be the same as the sort field.

     • Hash Digits: The digit positions to use for hashing. For example, an entry of 4-6 means that the fourth and fifth digits of the value are used.

     • Advanced Configuration: This option is disabled by default. If you enable it, you must enter a filter expression.

       Note

       The Filter Expression processes data row by row. It supports Groovy syntax.

     • Aggregate for Reconciliation: This option is disabled by default. If you enable it, configure the related parameters as shown in the following figure. To add multiple properties, click Add Property.

   • Offline

     • Offline Data Source Configuration: Select an offline data source configuration. You can search for a configuration by entering a keyword from its name. If you have not created an offline data source configuration, click Create Data Source Configuration on the right. For more information, see Create a data source configuration.

     • Offline Table: Select the ODPS table to use for reconciliation.

       Note

       ODPS tables support fuzzy matching. If no match is found, confirm that the project name and table name are correct.

5. After you complete the configuration, click Submit.

   The new node appears in the data node list.

View data node details

Click a data node name in the list to open the Data Node Details page. This page contains the following two sections:

• Basic Information (① in the figure)

• Configuration Information (② in the figure)

Modify a data node

1. In the Actions column of the data node list, click Modify to open the Modify Data Node page.

2. Modify the node information as needed.

3. When you are finished, click Submit to save the changes. The changes take effect for all associated reconciliation points.