By default, each data source is provided with a free quota of two resources for data processing. If you want to update a large amount of data within a tight schedule, you can modify the process_partition_count parameter to scale out data processing resources.
Scale out data processing resources
In the left-side navigation pane of the Instance Details page, choose Configuration Center > Advanced Configurations. On the page that appears, click the Data Processing Configuration tab. On the Data Processing Configuration tab, select the data source that you want to manage and click Manage in the Actions column.
On the page that appears, find the version of data processing configuration that you want to modify.
Click Modify in the Actions column to modify the data processing configuration.
By default, a data processing configuration has the following versions:
api_v1: the first published version of the data processing configuration. If the configuration has been pushed and indexes are rebuilt, the version is in the In Use state. If the configuration has not been pushed and no indexes are rebuilt, the version is in the Unused state.
api_edit: the version of the data processing configuration that is being modified. The version is in the Modifying state.
The version names are suffixed with _v{n}. n indicates an integer that increases by 1 each time a version is generated. For example, the second version of data processing configuration is named api_v2, and the third version is named api_v3. You must enter a description for each version to differentiate the versions.
Modify the process_partition_count parameter.
Note:
By default, a free quota of two resources is provided. Each resource consists of 4 CPU cores and 8 GB of memory. You are charged for the resources that exceed the free quota. Valid values of the process_partition_count parameter: an integer from 1 to 50.
After you modify the parameter, click Submit. In the message that appears, you can click the link to view how you are charged for the resources that exceed the free quota.
After you modify the data processing configuration, click Publish in the Actions column. A new version of data processing configuration is generated.
After you modify the data processing configuration, choose O&M Center > O&M Management in the left-side navigation pane. On the page that appears, click the Update Configurations tab. In the Instance Configuration Update panel, push the offline configurations of the instance based on the specified data source to make the offline configurations take effect.
To make the offline configurations take effect on the version that is in use, you must specify an index schema version.
Usage notes:
After a new version of the data processing configuration is pushed, the volume of incremental data may fluctuate for a short period of time. The fluctuations stop after the new servers work as expected.