Data processing configurations

Scale out data processing resources

1. In the left-side navigation pane of the Instance Details page, choose Configuration Center > Advanced Configurations. On the page that appears, click Data Processing Configuration. On the Data Processing Configuration tab, select the data source that you want to manage and click Manage.

2. On the page that appears, find the data source that you want to edit.

3. Click Edit in the Actions column to modify the data processing configuration.

   By default, a data processing configuration has the following versions:

   • api_v1: the first published version of the data processing configuration. If the configuration has been pushed and indexes are rebuilt, the status of the version is In Use. If the configuration has not been pushed and no indexes are rebuilt, the status of the version is Unused.

   • api_edit: the version of the data processing configuration that is being modified. The status of the version is Modifying.

   Version names are suffixed with _v{n}. {n} is an integer that increases by 1 each time a version is generated. For example, the second version is named api_v2, the third version is named api_v3, and so on. Enter remarks for each version to distinguish among different versions.

   Modify the process_partition_count parameter.

   Note

   By default, a free quota of two resources is provided. Each resource consists of 4 vCPUs and 8 GB of memory. You are charged for the resources that exceed the free quota. Valid values of the process_partition_count parameter: an integer from 1 to 50.

4. After you modify the parameter, click OK. In the message that appears, you can click the link to view how the resources that exceed the free quota are billed.

5. After you modify the data processing configuration, click Publish in the Actions column. A new version is generated.

6. After you modify the data processing configuration, go to the O&M Management page of the instance and click Update Configurations. In the Instance Configuration Update panel, push the offline configurations of the instance based on the specified data source to make the offline configurations take effect.

   In the Instance Configuration Update panel, set Configuration Type to Offline Configuration. In Index Schema Version, select the index tables that you want to update. Confirm the Dictionary Configuration Version and Data Processing Configuration Version. Select all Target Clusters. In Trigger Index Rebuild, select Specify Index Version to Take Effect and specify the Full Index Version.

   Important

   To make the offline configurations take effect on the version that is in use, you must specify an index schema version.

Usage notes

After a new version of the data processing configuration is pushed, the volume of incremental data may fluctuate for a short amount of time. The fluctuations stop after new servers work as expected.