Index Management Center

How it works

Access the index management center

The index management feature is available in the Elasticsearch console only for version 7.10 instances with Indexing Service or OpenStore enabled.

1. Log on to the Alibaba Cloud Elasticsearch console.

2. In the left-side navigation pane, click Elasticsearch Clusters.

3. Navigate to your target instance.

   1. In the top menu bar, select a resource group and a region.

   2. In the Elasticsearch Clusters list, click the ID of your target instance.

4. In the left-side navigation pane, choose Configuration and Management > Index Management Center.

The Index Management Center contains the following tabs: Data Stream Management, Index Management, Index Template Management, and Content Template Management.

Manage component templates

If multiple index templates share settings, mappings, or alias configurations, we recommend creating a component template and referencing it in your index templates to avoid redundant definitions. If you do not need to reuse configurations across templates, you can skip this step and define all configurations directly in the index template.

Create an index template

Before you create a data stream, you must first create a matching index template to define the structure and lifecycle policy for its backing indexes. This process has two steps: configuring the index lifecycle policy (optional) and configuring the index template.

1. Click the Index Template Management tab, and then click Create Index Template.

2. Optional: In the Create Index Template panel, configure an index lifecycle policy.

   If you do not need to manage the lifecycle of the data stream's backing indexes, click Skip. We recommend configuring a lifecycle policy to automatically manage the rollover and data migration of backing indexes with ILM, which helps optimize cloud hosting space and reduce storage costs.

   Parameter	Description
   Index Lifecycle Policy	New Index Lifecycle Policy: Create a new ILM policy. Existing Index Lifecycle Policy: Choose from existing lifecycle policies in the cluster.
   Policy Name	If you choose to create a new policy, you must enter a custom name. If you choose to select an existing policy, select one from the drop-down list. The configurations of the selected policy are automatically applied to the current index template, and you can modify them if needed.
   Hot Phase	Enabled by default and cannot be disabled.
   Rolling Update	We recommend enabling this option. By default, ILM checks for indexes that meet the policy criteria every 10 minutes. An index might not roll over immediately after meeting the rollover conditions. You can modify the check interval by using the indices.lifecycle.poll_interval parameter. If no rollover policy is configured, all index data is stored in Indexing Service, which increases your cloud hosting costs.
   Duration After Which Hosting Is Canceled	The default is 3 days. After this period, the index automatically exits write hosting in Indexing Service to reduce billing overhead. If the specified duration exceeds the start time of the cold or delete phase, the index is automatically unhosted when it enters that phase and becomes read-only. Important Long-term data hosting incurs index hosting storage fees. We recommend setting an unfollow time to unhost an index after it no longer receives writes. The cloud service has limits on the duration of write hosting and the number of hosted shards. For more information, see OpenStore storage engine.
   Cold Phase	OpenStore Custom Hot/Cold Storage edition: Supports enabling and configuring the cold phase. OpenStore Smart Hot/Cold Storage edition: Does not have a cold phase.
   OpenStore	This can be configured only after the Cold Phase is enabled. It is suitable for scenarios with high query complexity and high costs for mass storage. For more information, see OpenStore storage-compute separation (high-performance retrieval) engine. Notes: Only version 7.10 kernel-enhanced edition Indexing Service instances with OpenStore Custom Hot/Cold Storage enabled at the time of purchase support OpenStore configuration. After the cold phase and OpenStore storage are enabled, the system automatically migrates data to cold data nodes and OpenStore storage when the index reaches its migration time.
   Delete Phase	Set the deletion time for the index. For OpenStore Smart Hot/Cold Storage, Elasticsearch intelligently tiers data based on the index's read/write frequency before the index is deleted.

3. Click Save and Next. In the Index Template Configuration wizard, configure the index template information.

   Every document written to a data stream must contain a @timestamp field mapped as a date or date_nanos type. Otherwise, the write operation will fail. If the source data does not have a @timestamp field, you can use an ingest pipeline to automatically add it by retrieving the metadata value from _ingest.timestamp. We recommend specifying the mapping for the @timestamp field in the index template's mappings. If not specified, the field is automatically mapped to the date or date_nanos type.

   Note

   The kernel-enhanced edition instance provides a low-priority aliyun_default_index_template template for the * index pattern to reduce costs for logging scenarios. This template enables field presorting (index.sort.field) to improve search performance for time-series data. Presorting is not supported for nested type fields. We recommend disabling it in your template if you use nested fields.

   Parameter	Description
   Index Template Name	Enter a custom name for the index template.
   Index Pattern	Define index patterns to match data stream and index names. Wildcard characters (*) are supported. Spaces and the following special characters are not allowed: \, /, ?, ", <, >, and |.
   Create Data Stream	Enable data stream mode. If this option is not enabled, this template cannot be used to create a data stream. For more information, see Data streams.
   Priority	Define the template priority. A higher number indicates a higher priority.
   Index Lifecycle Policy	If you created an index lifecycle policy in the previous step, the system automatically references that policy, and it cannot be changed. If you did not create a policy, you can leave this blank or enter the name of an existing policy in the cluster (only one name is supported). For more information about lifecycle policies, see ILM.
   Configuration	Configure index settings, mappings, and aliases. The configuration format must strictly follow the official Elasticsearch specifications. You can provide settings (example: { "index.number_of_replicas": "1", "index.number_of_shards": "5" }), mappings (example: { "properties": { "like": { "type": "keyword" }, "name": { "type": "text" }, "age": { "type": "integer" }, "email": { "type": "keyword" } } }), aliases (example: { "aliases": { "demo": {} } }), or compose the template from component templates. Examples of component templates include component-openstore-index-template for OpenStore Custom Hot/Cold Storage and component-hybrid-openstore-index-template for OpenStore Smart Hot/Cold Storage.

4. Click OK.

   After the template is created, a Created. dialog box appears. You can click Go to Create Data Stream to navigate directly to the data stream creation page, or close the dialog box to return to the index template list.

   • The created index template appears in the index template list. When you first visit the Index Template Management page, you can see the default index templates created by Alibaba Cloud Elasticsearch. We recommend that you do not modify the configuration of these default templates.

   • The Content Template column of the index template lists the component templates it references. Referencing component templates allows you to create composable templates. Click a component template name to go to the Content Template Management page to manage that template.

   • In the index template list, you can modify an index template by clicking Modify in the Actions column for the target template. You can also view and modify an index template by clicking its Index Template Name or Index Lifecycle Policy.

Manage data streams

Before creating a data stream, you must create a matching index template with data stream mode enabled.

Manage indexes

After a data stream is created, its backing indexes are automatically generated and hosted in Indexing Service. You can use the index management feature to view and manage these cloud-hosted indexes, including filtering indexes, viewing lifecycle phases, and controlling their write hosting status.

Click the Index Management tab to view and manage the indexes in your cluster.

By default, all indexes are displayed. You can also filter or search for specific indexes based on your needs:

• For the OpenStore Custom Hot/Cold Storage edition, you can filter indexes by using Hosted Indexes and OpenStore Indexes.

• For the OpenStore Smart Hot/Cold Storage edition, you can only filter indexes by using Hosted Indexes. The top of the Index Management page also displays the Total OpenStore Index Size and the Indexes.

API reference

• Cancel index hosting.

  POST /<yourIndexName>/_cube/unfollow

  Forcefully canceling index hosting: During cancellation, an index must complete segment merging before hosting ends. The time required for merging depends on factors like the number of segments, data write activity, and data volume. If this process takes too long, you can use the following command to forcefully cancel index hosting on kernel-enhanced edition instances with kernel version 1.6 or later.

  POST /<yourIndexName>/_cube/unfollow?force=true

  Replace <yourIndexName> with your actual index name.

  Important

  The APIs for canceling and forcefully canceling index hosting do not support batch operations. Frequent cancellations can affect the stability of the hosting service.

  Forceful cancellation of index hosting is supported only on kernel-enhanced edition instances with kernel version 1.6 or later. For instances with kernel versions earlier than 1.6, you must first upgrade the kernel version. For more information, see Upgrade versions.

• Get hosting status information.

  GET /<yourIndexName>/_cube/stats

  Replace <yourIndexName> with your actual index name.

• Get the overall hosting status of the cluster in Indexing Service.

  GET /_cube/stats

• Restore a snapshot to an Indexing Service instance.

  For example, to restore the index_1 index from the snapshot_1 snapshot in the my_backup repository to an Indexing Service instance, use the following command. Replace the placeholder information with your actual values.

  POST /_snapshot/my_backup/snapshot_1/_restore
  {
    "indices": "index_1",
    "ignore_index_settings": [
      "index.apack.cube.following_index"
    ]
  }

• Manually trigger a data stream rollover.

  POST <yourDatastreamName>/_rollover

  Replace <yourDatastreamName> with your actual data stream name.