Scenario-based configuration, scenario-based template, dynamic cluster configuration, index template configuration, index lifecycle configuration-Elasticsearch(ES)-阿里云帮助中心

Scenario-based configuration templates package best-practice cluster and index settings for common workloads such as general-purpose, data analytics, database acceleration, and search. Select a matching template and the system applies the recommended settings automatically, reducing cluster and performance issues caused by improper configuration.

Scenario-based configuration templates include three types of configurations:

Dynamic cluster configuration: Dynamically modifies cluster-level settings. This is equivalent to running the PUT /_cluster/settings command. For more information, see Cluster update settings.
Index template configuration: Defines a template that is automatically applied when you create a new index and does not affect existing ones. This is equivalent to running the PUT _template/aliyun_default_index_template command. For more information, see Index templates.
Index lifecycle configuration: Automates the transition of an index through the hot, warm, cold, and delete phases. This is equivalent to running the PUT _ilm/policy/aliyun_default_ilm_policy command. This feature is available only for instances of version 6.7.0 or later that have cold data nodes. For more information, see Setting up a new policy.

Applicability

This feature is available only for v2 architecture clusters. To identify your cluster architecture:

Log on to the Alibaba Cloud Elasticsearch console. On the Basic Information page of the instance, view the Control Architecture Type: Cloud-native Control Architecture (v3) or Basic Control Architecture (v2).

Usage notes

Available templates vary by instance version and type. Check the console for the most current options. Instances with the Indexing Service enabled support only the log scenario.
Modifying a scenario-based template is equivalent to running the PUT _template/aliyun_default_index_template command. This operation does not restart the instance or affect existing indexes or the cluster status.
The recommended index template is named aliyun_default_index_template with an order priority of Integer.MIN_VALUE+1, which is lower than custom templates. Do not change this priority so that your custom templates always take precedence.
The recommended index lifecycle policy is named aliyun_default_ilm_policy and is applied to the aliyun_default_index_template template by default.
Enabling index lifecycle configuration does not prevent you from creating other index lifecycle policies on the cluster. The close action is not supported in index lifecycle policies.
When you purchase a new instance, you can select a scenario on the purchase page. The general-purpose scenario is selected by default for Commercial Edition instances, and the log scenario for Enhanced Edition instances. After the purchase, the corresponding template is automatically applied. If Scenario is set to None, follow the steps in this topic to enable it. You must then manually submit the configuration to apply a template.
The available options in the console may vary depending on the management version of your Elasticsearch instance. If scenario-based configuration is not available in the console, use the Elasticsearch API (PUT _template/aliyun_default_index_template) to configure the settings.
If your cluster does not have cold data nodes or runs a version earlier than 6.7.0, the index lifecycle configuration option is disabled by default. To use this feature, see Elasticsearch ILM for instructions on how to configure it manually.

Modify a scenario-based configuration template

Log on to the Alibaba Cloud Elasticsearch console.
In the left-side navigation pane, click Elasticsearch Clusters.
Navigate to the target instance.
1. In the top menu bar, select a resource group and region.
2. On the Elasticsearch Clusters page, click the ID of the target instance.
In the left-side navigation pane, click Cluster Configuration.
In the Scenario-based Configuration area, click Scenario next to Modify.
In the Select Scenario dialog box, select a Scenario and click OK. This is a dynamic operation and does not require a cluster restart.
Modify the scenario configuration template.

A scenario-based configuration template includes three types of configurations:
- Dynamic cluster configuration: Dynamically modifies cluster settings. This is equivalent to running the PUT /_cluster/settings command. For more information, see Cluster update settings.
- Index template configuration: Defines a template that is automatically applied when you create a new index. Changes to the template do not affect existing indexes. This is equivalent to running the PUT _template/aliyun_default_index_template command. For more information, see Index templates.
- Index lifecycle configuration: For instances of version 6.7.0 or later, you can enable an index lifecycle template. This is equivalent to running the PUT _ilm/policy/aliyun_default_ilm_policy command. For more information, see Setting up a new policy.
The following steps show how to modify the Index Template Configuration. The procedure is similar for the other configuration types.
1. Click Index Template Configuration.
2. On the Index Template Configuration page, click Apply.
  
  The functions are described as follows:
  - Apply: Applies the recommended template from the right panel to the current configuration in the left panel. You can then modify the current configuration.
  - Compare: Displays the differences between the current configuration and the recommended template. This button is available only after you modify the current configuration. Editing is disabled in comparison mode.
  - Reset: Resets the current configuration to the one currently active on the cluster.
3. In the Current Configuration section on the left, modify the configuration of the current template.
4. After you make changes, click Compare to view the differences between your modified configuration and the recommended template.
  
  The comparison page displays the current configuration in the left pane and the recommended configuration for the general-purpose scenario in the right pane. Lines with differences are highlighted. For example, the current thresholds for all levels of search.slowlog.threshold.fetch are 0ms, while the recommended values are 200ms for warn, 50ms for trace, 80ms for debug, and 100ms for info. From the comparison page, you can click apply to apply the recommended configuration, click cancel to return to the edit page, or click reset to restore the default values.
5. Click Cancel.
Click Submit.

After submission, the modified configuration is applied to the cluster.

Dynamic cluster parameters

After you enable a scenario-based template, you can dynamically change the cluster settings. The following table describes the dynamic cluster configuration parameters.

Parameter	Description
`cluster.routing.allocation.balance.index`	The weight factor for balancing the number of shards of each index across nodes. The default value is 0.55f. A higher value distributes the shards for each index more evenly across all nodes in the cluster. For search scenarios, increasing this value to 0.8f can improve query performance by making shard distribution more uniform.
`cluster.routing.allocation.balance.shard`	The weight factor for the total number of shards allocated on a node. The default value is 0.45f. A higher value increases the tendency to balance the total number of shards across all nodes in the cluster.
`search.max_buckets`	The maximum number of buckets allowed in a single response. Supported in Elasticsearch 6.2 and later. The default value is -1 (no limit), though a deprecation warning is logged when results exceed 10,000 buckets. In Elasticsearch 7.x, the default value is 10,000.

Index template parameters

After you enable a scenario-based template, you can dynamically change the cluster index template. The following table describes the index template parameters.

Important

The recommended index template is named aliyun_default_index_template with an order priority of Integer.MIN_VALUE+1. Do not change this priority to avoid conflicts with custom templates. This template provides recommended index settings for your use case without affecting custom configurations.

Sharding and performance parameters

Parameter	Description
`index_patterns`	The index pattern for template matching. This pattern supports wildcards and defaults to `*`. We do not recommend modifying this setting.
`index.number_of_shards`	The number of primary shards for an index. Before version 7.x, the default was 5. In version 7.0 and later, the default is 1. Setting this to 1 effectively controls the number of shards in the cluster and prevents performance issues caused by an excessive number of shards.
`index.refresh_interval`	The frequency of the refresh operation. The default is 1s. If real-time visibility is not critical, increase this value (for example, to 10s) to reduce refresh overhead and improve cluster performance.
`index.unassigned.node_left.delayed_timeout`	The timeout for delaying replica shard allocation after a node leaves the cluster. The default value is 1min. Increasing this value (for example, to 5min) can speed up cluster recovery.

Segment merge parameters

Parameter	Description
`index.merge.policy.segments_per_tier`	The number of segments per tier. A smaller value results in fewer segments but more merge operations, which can decrease indexing performance. The default value is 10. This value should not be less than `index.merge.policy.max_merge_at_once`.
`index.merge.policy.max_merged_segment`	The approximate maximum size of a single segment during indexing. The default value is 5 GB.

Translog parameters

Parameter

Description

index.translog.durability

Controls whether the translog is synchronously persisted to disk after each data update. Valid values:

request: Persists the translog to disk synchronously for each request, ensuring no data loss on node failure.
async: Persists the translog to disk asynchronously at regular intervals. This improves write performance but risks data loss on node failure before persistence completes.

Slow log parameters

Parameter	Description
`index.search.slowlog.level`	The log level for slow queries.
`index.search.slowlog.threshold.query.warn`	The threshold for slow query logs at the warn level.
`index.search.slowlog.threshold.query.info`	The threshold for slow query logs at the info level.
`index.search.slowlog.threshold.query.debug`	The threshold for slow query logs at the debug level.
`index.search.slowlog.threshold.query.trace`	The threshold for slow query logs at the trace level.
`index.search.slowlog.threshold.fetch.warn`	The threshold for slow fetch logs at the warn level.
`index.search.slowlog.threshold.fetch.info`	The threshold for slow fetch logs at the info level.
`index.search.slowlog.threshold.fetch.debug`	The threshold for slow fetch logs at the debug level.
`index.search.slowlog.threshold.fetch.trace`	The threshold for slow fetch logs at the trace level.
`index.indexing.slowlog.level`	The log level for slow indexing operations.
`index.indexing.slowlog.source`	The number of `_source` characters to log.
`index.indexing.slowlog.threshold.index.warn`	The threshold for slow indexing logs at the warn level.
`index.indexing.slowlog.threshold.index.info`	The threshold for slow indexing logs at the info level.
`index.indexing.slowlog.threshold.index.debug`	The threshold for slow indexing logs at the debug level.
`index.indexing.slowlog.threshold.index.trace`	The threshold for slow indexing logs at the trace level.

Other parameters

Parameter	Description
`index.lifecycle.name`	The name of the index lifecycle policy.
`mappings._default_._all.enabled`	Controls whether the `_all` field is enabled. In version 5.x, the default value is `true` (we recommend setting it to `false`). In version 6.x, the default value is `false`. This parameter is deprecated in version 7.x.

Index lifecycle parameters

An index lifecycle policy defines how an index transitions through different phases. When enabled, the policy is named aliyun_default_ilm_policy and applied to the aliyun_default_index_template template by default. You can view the policy by using the Elasticsearch APIs: Getting templates and Get lifecycle policy API.

Important

Enabling scenario-based index lifecycle configuration does not prevent you from creating other index lifecycle policies on the cluster.
This option is disabled by default on clusters that lack cold data nodes or run a version earlier than 6.7.0. To use this feature, see Elasticsearch ILM for instructions on how to configure it manually.
The close action is not supported in index lifecycle policies.

Parameter	Description
`phases.hot.min_age`	The minimum age of an index before it enters the hot phase.
`phases.hot.actions.set_priority.priority`	The priority of an index in the hot phase.
`phases.warm.min_age`	The minimum age of an index before it enters the warm phase.
`phases.warm.actions.allocate.number_of_replicas`	The number of replica shards for an index in the warm phase.
`phases.warm.actions.allocate.require.box_type`	The node attribute required for shard allocation in the warm phase. For example, you can require that shards are allocated only to `warm` nodes.
`phases.warm.actions.set_priority.priority`	The priority of an index in the warm phase.
`phases.cold.min_age`	The minimum age of an index before it enters the cold phase.
`phases.cold.actions.set_priority.priority`	The priority of an index in the cold phase.
`phases.delete.min_age`	The minimum age of an index before it enters the delete phase.