Alibaba Cloud Elasticsearch cluster FAQ

FAQ

• Purchase and cancellation

  • What should I do if I select an incorrect configuration when purchasing an Elasticsearch cluster?

  • What specific versions do the version options on the Elasticsearch purchase page correspond to?

  • What should I do if no VPCs are available when I purchase an Elasticsearch cluster?

  • What should I do if no vSwitches are available for my VPC when purchasing a cluster, causing a vSwitch: may not be empty error?

  • Does the access endpoint change if I cancel an Elasticsearch cluster and purchase a new one?

  • How do I release or unsubscribe from an Elasticsearch cluster?

  • How long after stopping an Elasticsearch cluster is it released?

  • Can I purchase a single-node Elasticsearch cluster?

  • What should I do if resources are unavailable when purchasing a cluster?

  • Why is it recommended that users with legacy 1-vCPU and 2-GiB clusters upgrade their specifications as soon as possible?

  • Can I get a free trial of an Alibaba Cloud Elasticsearch cluster?

  • What should I do if my Elasticsearch cluster remains in the "Creating" state after purchase?

  • Is a separate Kibana purchase required after creating an Elasticsearch cluster?

  • Why can't I find the cluster that I created?

  • When do I need to purchase dedicated master nodes and client nodes for an Elasticsearch cluster?

  • When I purchase an Elasticsearch cluster, what is the default account associated with the password I enter?

• Product features

  • Does Alibaba Cloud Elasticsearch support version upgrades or downgrades?

  • Can I log on to an Elasticsearch cluster over SSH to modify the configuration?

  • Is Logstash 6.7 compatible with Elasticsearch 6.3?

  • Does Quick BI support Elasticsearch as a data source?

  • Does Elasticsearch support scoring plug-ins?

  • Does Elasticsearch support the LDAP feature?

  • Does Elasticsearch provide a Java SDK?

  • How can I view the kernel version of an Elasticsearch cluster?

  • When should I use force restart on an Elasticsearch cluster, and what are its impacts?

  • How can I verify that the Log4j2 vulnerability is fixed for my Elasticsearch cluster?

  • How do I enable private network communication between Elasticsearch clusters in different regions?

  • How do I migrate data between Elasticsearch clusters?

  • Do I need to purchase client nodes to enable the HTTPS protocol?

  • What is the maximum number of shards supported by a single node?

  • How are indices that start with .monitoring-es generated, and what are their purposes?

  • When cloud disk encryption is enabled for Elasticsearch data disks, what encryption algorithm is used?

  • Does Alibaba Cloud Elasticsearch support port 9300?

  • How do I archive cold data from an Elasticsearch cluster to OSS?

• Data migration and synchronization

  • How do I synchronize data from MongoDB to Alibaba Cloud Elasticsearch?

• Cluster restart

  • How long does it take to restart an Elasticsearch cluster or a node?

  • Does enabling or disabling public access trigger a cluster restart?

  • Does changing the cluster access password trigger a cluster restart?

  • If an index has no replicas, will it affect a cluster restart?

  • How do I restart a node of a specific role, such as a Kibana node, or an individual node?

  • What should I do if a cluster restart gets stuck?

  • Can I schedule periodic node restarts for an Elasticsearch cluster?

• Cluster load and status

  • What should I do when CPU utilization and load are unevenly distributed across cluster nodes?

  • What should I do if my Elasticsearch cluster status is YELLOW?

  • What should I do if my Elasticsearch cluster status is RED due to a high load?

  • What should I do if monitoring data or an alert indicates high CPU usage?

  • What should I do if the disk usage of my Elasticsearch cluster is too high?

  • What should I do if monitoring data or an alert indicates high memory usage?

  • Shards are unevenly distributed. How can I rebalance them?

  • The cluster load is high, and the main log reports a java.lang.StackOverflowError for the entire cluster. What should I do?

  • What causes high fielddata usage?

  • How do I check the actual amount of memory allocated in the JVM configuration?

• Cluster query and write

  • What should I do if a query to my Elasticsearch cluster is slow or returns no results?

  • A Data too large... which is larger than the limit of error is reported when writing to the cluster. What should I do?

  • Does Elasticsearch support deleting indexes in bulk?

  • When I create a new index, an index uuid conflicted error occasionally occurs and prevents documents from being written to the index. What should I do?

  • How do I modify the index.max_result_window parameter, which specifies the maximum number of documents that can be returned in a paginated query?

  • When updating data in Elasticsearch, a Rejecting mapping update to [] as the final mapping would have more than 1 type error is reported. What should I do?

  • How do I query the detailed content of documents in an index?

  • How do I adjust the queue size?

  • How do I query or export data within a specific time range?

  • Is there a limit on the number of documents in a bulk insert operation in Elasticsearch?

  • What should I do about the time lag between query results and actual data?

• Cluster configuration and changes

  • How should I plan the resources, specifications, and shard count for an Elasticsearch cluster?

  • How do I view the configuration parameters of an Elasticsearch cluster?

  • Will changing the cluster configuration affect the Elasticsearch service?

  • After the number of nodes is changed, does the cluster automatically rebalance the shards?

  • Does Elasticsearch support changing the cloud disk type?

  • Can I convert other node types to cold nodes?

  • After upgrading the cluster specifications, can I downgrade the configuration? If so, how?

  • If traffic temporarily surges, how can I change the cluster configuration to ensure business continuity?

  • When I upgrade a cluster, the message UpgradeVersionMustFromConsole is returned. What should I do?

  • How long does it take to upgrade an Elasticsearch version?

  • Will upgrading the Elasticsearch version affect cluster services?

  • What should I do if a cluster upgrade fails or times out?

  • I cannot change the cluster configuration. What should I do?

  • Does Elasticsearch support modifying JVM parameters?

  • Can I adjust the http.max_content_length and discovery.zen.ping_timeout values in the cluster's YML configuration file?

  • Can I switch the VPC for an Elasticsearch cluster?

  • If I did not enable data nodes when I purchased a Log-Enhanced Edition cluster, can I enable them when I upgrade the cluster?

  • Will changing the cloud disk type for an Elasticsearch cluster cause data loss?

  • When I upgrade a cluster, a message indicates that the cluster is unhealthy, but the cluster status is Green. What should I do?

  • Can I upgrade only the CPU of an Elasticsearch cluster to avoid data migration?

  • Why can't I downgrade cold node configurations?

  • When I scale in the data nodes of a cluster, the error "This operation will cause insufficient cluster resources (Disk/CPU/Memory) or abnormal shard allocation" is reported. What should I do?

  • When I scale in the data nodes of a cluster, the error "The cluster is in an abnormal state or has unfinished tasks" is reported. What should I do?

  • When I scale in the data nodes of a cluster, the error "The number of reserved nodes must be greater than 2" is reported. What should I do?

  • When I scale in the data nodes of a cluster, the error "This operation is not supported by the current Elasticsearch cluster configuration" is reported. What should I do?

  • An index that uses auto_expand_replicas causes a data migration or node scale-in to fail. What should I do?

  • How do I clear the cache for an Elasticsearch cluster?

  • How do I change the zone of an Elasticsearch cluster?

• Plug-ins, tokenizers, and synonyms

  • How do I customize and extend the dictionary for the IK analyzer?

  • When I use the IK analysis plug-in, an ik startOffset error is reported. What should I do?

  • If my local IK dictionary files are lost, can I retrieve them from the cluster management page?

  • After I update the IK dictionary, how can I apply the new dictionary to existing data?

  • Is there a standard value for full GC?

  • Can I uninstall unused plug-ins?

  • Are the dictionaries of the Alibaba Cloud Elasticsearch IK analysis plug-in identical to those of the open source IK plug-in?

  • Can custom plug-ins access external networks, for example, to read dictionary files from GitHub?

  • Do custom plug-ins support hot updates?

  • How is the analysis-aliws tokenizer configured, and what is its file format?

  • What are the differences among Elasticsearch synonyms, the IK tokenizer, and the AliNLP tokenizer?

  • Where do I configure the Elasticsearch IK analysis mode?

  • What built-in Chinese tokenizers does Alibaba Cloud Elasticsearch support?

  • If I use OSS to hot-update dictionary files, does Elasticsearch automatically update when the dictionary file content in OSS changes?

  • Does the IK analyzer used by Alibaba Cloud Elasticsearch support remote dictionaries?

  • How do I install the vector search plug-in (aliyun-knn) for an Alibaba Cloud Elasticsearch 7.10 cluster?

  • Will restarting a cluster to install a plug-in affect cluster services?

• Logs

  • Can I set a retention period for .security logs in Elasticsearch?

  • How do I save Elasticsearch logs to a local machine?

  • I cannot view the query and update logs of Elasticsearch. What should I do?

  • How do I configure and view the slow logs of an Elasticsearch cluster?

  • How do I programmatically and periodically pull the slow logs of an Elasticsearch cluster?

  • How do I find out which clients are using my Elasticsearch cluster?

• Data backup and restoration

  • Can a snapshot from an Elasticsearch cluster be restored to a cluster of a different version?

  • When I back up Elasticsearch data, a message indicates that the cluster is unhealthy. What should I do?

  • I enabled automatic backups but have not configured an OSS bucket. Does this mean the backups failed?

  • When I migrate or restore data from a snapshot, the destination cluster reports a shard allocation failure. The POST /_cluster/reroute?retry_failed=true command does not fix the issue, and the corresponding index remains in an abnormal state. What should I do?

  • Can data from Elasticsearch be exported to a local machine?

  • How do I restore a snapshot across clusters?

  • What data backup options does Alibaba Cloud Elasticsearch provide?

• Cluster monitoring and alerting

  • How do I configure X-Pack Watcher alerts?

  • What should I do if I receive an alert about a GC memory allocation failure?

  • Does Elasticsearch support monitoring with Grafana?

  • What do the cluster status metric values mean?

  • How do I view the disk usage of an Elasticsearch cluster?

  • A promotion failed error is reported during CMS GC. What should I do?

• Cluster access

  • How does connecting to an Alibaba Cloud Elasticsearch cluster differ from connecting to an open-source one?

  • When accessing an Elasticsearch cluster with a client, can I disable Basic Authentication?

  • If an Alibaba Cloud ECS instance and an Elasticsearch cluster are in the same VPC but different zones, can the ECS instance access the Elasticsearch cluster over the private network?

  • How do I configure the IP address whitelist for public or private access to an Elasticsearch cluster?

  • How do I connect to an Elasticsearch cluster over the Internet?

  • I cannot access an Elasticsearch cluster and receive the error Failed to establish a new connection: [Errno 61] Connection refused. What should I do?

  • Will resetting the password affect access to an Elasticsearch cluster?

  • The Elasticsearch-Head plug-in V5.0.0 cannot access any version of Alibaba Cloud Elasticsearch clusters. What should I do?

• Indexing

  • How do I close an index?

Incorrect cluster configuration

For configuration items not listed in this table, see the Upgrade cluster configuration and Downgrade a cluster documentation to determine if they can be modified.

Mapping buy page versions to Elasticsearch versions

When purchasing a cluster, choose a version close to your existing self-managed cluster. If you have no existing cluster, select the latest version.

No VPCs available when purchasing Elasticsearch

See View RAM user details to check if the RAM user has permission to list VPCs. If the user lacks this permission, see Create a custom policy to grant it.

Error: "vSwitch: may not be empty"

This error means no vSwitches exist in your selected zone. Check which zone you selected on the cluster configuration page, then verify in the VPC console whether a vSwitch exists in that zone. If not, create one. Create a VPC with an IPv4 CIDR block.

Endpoint change on instance replacement

Yes. Update your client code with the new endpoint before unsubscribing from the original instance to avoid service disruption.

Release or unsubscribe from an Elasticsearch cluster

• To release a pay-as-you-go cluster or an expired subscription cluster, see Release a cluster.

• To unsubscribe from an unexpired subscription cluster:

In the top navigation bar of the console, chooseExpenses > Orders, go to theUser Centerto unsubscribe. For more information, seeUnsubscription methods.

Retention period for expired instances

The instance is released 7 days after its service ends. Once released, its data is permanently deleted and cannot be recovered. Expiration and overdue payments.

Single-node Elasticsearch clusters

No. A minimum of two data nodes is required. Configure nodes for an Alibaba Cloud Elasticsearch cluster.

Unavailable resources during instance creation

If resources remain unavailable, try again later. Resource inventory is dynamic and Alibaba Cloud continually replenishes resources.

Upgrade 1 vCPU, 2 GiB data nodes

Alibaba Cloud Elasticsearch stopped selling 1 vCPU, 2 GiB data nodes in May 2021 due to performance stability concerns. Existing nodes are unaffected but suitable for learning only, not production, and are not covered by the SLA. Upgrade to higher specifications as soon as possible. Upgrade cluster configuration.

Free trial of ES

New users can try an Alibaba Cloud ES instance with the free trial. To learn the basic features of ES, see the Use Alibaba Cloud Elasticsearch to perform a basic search tutorial.

Elasticsearch cluster stuck initializing

Initialization time depends on cluster specifications, data structure, and size. This process can take several hours.

A newly purchased cluster does not display its internal endpoint while the cluster is in the Activating state. The internal endpoint becomes available after the cluster is created.

If the change involves restarting the Elasticsearch cluster, the restart duration depends on the index data, the cluster size, and the restart method. It typically takes several hours. For more information, see Restart a cluster or node.

You can monitor the change progress in the task list to determine whether cluster creation or restart is proceeding normally. If the change progress remains unchanged for a long period, see View instance task progress to troubleshoot the issue.

Do I need to purchase Kibana separately?

Cannot find a created instance

Make sure you have selected the correct region in the region selector at the top of the Elasticsearch console. If the region is correct but the cluster is still not visible, try the following methods:

1. Clear your browser cache and log on to your Alibaba Cloud account again. If your logon session has expired, you must log on again to restore it.

2. Try a different network.

3. Go to the Order Management page in the Expenses and Costs console and filter the orders by Alibaba Cloud Elasticsearch. Check the order status to confirm whether the cluster was released because it was not renewed. A cluster is released 7 days after its service ends. Once released, its data is permanently deleted and cannot be recovered by renewal. Create a new cluster.

When to purchase dedicated master and coordinating nodes

Coordinating nodes handle incoming query and write requests, forward the requests to the appropriate data nodes, and then merge the results. If your cluster frequently runs resource-intensive aggregation queries, purchase dedicated coordinating nodes. Generally, maintain a 1:5 ratio of coordinating nodes to data nodes (1 coordinating node for every 5 data nodes), with a minimum of two. The specifications of the coordinating nodes should match those of the data nodes. For a more detailed guide, see specification and capacity evaluation.

Default username

The default username is elastic. You can also create custom users. Manage access control with Elasticsearch X-Pack roles.

Version upgrades and downgrades

In-place upgrades are available only for specific version paths: 5.5 to 5.6, 5.6 to 6.3, and 6.3 to 6.7.

For other upgrade paths or for downgrades, you must create a new cluster and manually migrate your data. This involves purchasing a new cluster of the target version, migrating the data from your original cluster, and then unsubscribing from or releasing the original cluster.

• For in-place version upgrades, see Upgrade cluster versions.

• To create a new cluster, see Create an Alibaba Cloud Elasticsearch cluster.

• For data migration between Alibaba Cloud Elasticsearch clusters, see Migration between Alibaba Cloud Elasticsearch clusters.

• To unsubscribe from an unexpired subscription cluster, see Unsubscription and refund policies. To release a pay-as-you-go cluster, see Release a cluster.

Take note of the following limits when you downgrade a version:

• Object Storage Service (OSS) snapshots cannot be restored from a later Elasticsearch version to an earlier Elasticsearch version.

• If you use Logstash to migrate data, make sure that the Logstash version is compatible with the Elasticsearch version of the destination cluster. Alibaba Cloud Logstash supports versions 8.11, 7.4, and 6.7.

• The Kibana version matches the Elasticsearch version of the cluster and cannot be downgraded separately.

Modifying cluster configuration via SSH

No. Elasticsearch does not support SSH access. To modify your cluster configuration, see Overview of cluster configuration.

Logstash 6.7 and Elasticsearch 6.3 compatibility

Yes, they are compatible. See product compatibility.

Support for Elasticsearch as a data source

You can connect Quick BI to Elasticsearch over the public network by adding the Quick BI IP addresses to the public whitelist of your Elasticsearch cluster. Alternatively, you can use Kibana or DataV for data analysis and visualization.

Scoring plugin support

Elasticsearch supports creating tokenizers for indexes and sorting by score. Quick start: From instance creation to data retrieval.

LDAP support

Yes. Deploy and test an on-premises Elasticsearch cluster of the same version first. Once tests pass, configure the LDAP template in the console. Best practices for integrating LDAP authentication with X-Pack.

Alibaba Cloud Elasticsearch Java SDK

Yes. Each version has its own SDK. Java API.

Kernel version

Forced restart of an Elasticsearch cluster

If your Elasticsearch cluster is unhealthy (in a yellow or red state), you cannot perform a standard restart. Instead, you must perform a forced restart. This is a high-risk action that may cause service instability, data loss, or read and write failures. Proceed with caution.

Verify the Log4j 2 fix

A successful cluster restart confirms that the vulnerability is fixed. [Vulnerability Alert] Apache Log4j 2 Remote Code Execution Vulnerability.

Log4j2 vulnerability and Elasticsearch upgrade

No. Simply follow the Procedure.

Private network communication for cross-region Elasticsearch instances

Migrate data between Elasticsearch clusters

Migration approaches vary by scenario (between Alibaba Cloud clusters, or from self-managed/third-party clusters). migration method selection.

Coordinating nodes for HTTPS

You do not need to purchase coordinating nodes to enable the HTTPS protocol for the following Alibaba Cloud Elasticsearch versions. If your cluster already has coordinating nodes, you can disable them.

• Version 7.16 and later.

• Version 7.10 in some regions. For specific availability, check the console.

For all other versions, enabling the HTTPS protocol requires coordinating nodes. Once purchased, these nodes cannot be disabled or removed.

Alibaba Cloud Elasticsearch regularly updates certificates after HTTPS is enabled. Versions 7.10 and below do not support rolling certificate updates on data nodes, so certificates are deployed on coordinating nodes to minimize service impact during updates. If your cluster lacks coordinating nodes, the system prompts you to purchase them when enabling HTTPS. Use the HTTPS protocol.

Maximum shards per node

In Elasticsearch 7.x, a single node supports a maximum of 1,000 shards. Other versions have no limit. Configure shards per node based on your cluster specifications. Evaluate specifications and storage capacity and Size your shards.

PUT /_cluster/settings
{
   "transient": {
      "cluster": {
        "max_shards_per_node":10000
      }
   }
}

The .monitoring-es indexes

By default, the X-Pack monitoring component collects cluster data every 10 seconds and stores it in indexes prefixed with .monitoring-*. For example, in a version 6.x instance, two primary index types are created and rolled over daily: .monitoring-es-6-* and .monitoring-kibana-6-*, with data stored in an index prefixed with .monitoring-es-6- and suffixed with the current date.

The .monitoring-es-6-* indexes consume significant disk space, primarily storing cluster status, cluster statistics, node statistics, and index statistics. Configure monitoring for an Elasticsearch cluster.

Encryption algorithm for data disks

Disk encryption for Elasticsearch uses AES-256 and Key Management Service (KMS) to encrypt cloud disks. Disk Encryption.

Port 9300 support

Archive ES cold data to OSS

Use the Log-Enhanced Edition with OpenStore storage. Store massive amounts of data by customizing hot and cold storage with OpenStore.

Synchronize MongoDB to Alibaba Cloud Elasticsearch

Use Monstache to synchronize data from ApsaraDB for MongoDB to Alibaba Cloud Elasticsearch in real time. Use Monstache to synchronize data from MongoDB to Elasticsearch in real time.

Cluster and node restart duration

Restart duration depends on cluster specifications, data structure, and data volume. It typically takes several hours. Restart a cluster or node.

Public network access and cluster restart

No. While this action briefly changes the cluster's status as it takes effect, it does not affect normal operations.

Password change and cluster restart

No. A password change triggers a cluster reload, not a restart. Reset an Elasticsearch cluster password.

Impact of no index replicas on cluster restart

Yes. Without replicas, service interruptions can occur during restart. With low load, replica shards, and off-peak timing, the service generally remains available. However, timeouts may occur during high concurrency, force restarts, heavy workloads, or when indexes lack replicas. Configure a retry mechanism on your client and restart during off-peak hours.

Restart a node role or a single node

• Restart a node role

  On the Basic Information page of the instance, click Restart, set the Operation Type to Node Role, and select the corresponding node role. Restart a cluster or node.

• Restart a single node

  You can restart a single node in one of the following ways:

  • On the Basic Information page of your instance, click Restart. In the dialog box that appears, set Operation Type to Node and select the node you want to restart. Restart a cluster or node.

  • On the Basic Information page, find the Node Visualization section, hover over the node you want to restart, and click Restart in the pop-up. View cluster status and node information.

Stuck instance restart

Periodic restart of Elasticsearch nodes

Periodic node restarts are not supported. As a workaround, call the RestartInstance API from a scheduled task that you configure yourself.

Unbalanced node load

Resolve a yellow Elasticsearch cluster status

• Cause

  An Elasticsearch cluster enters a yellow state if the configured number of replicas for an index exceeds the number of available nodes minus one. This state prevents Elasticsearch from allocating all replica shards.

• Solution

  Run the GET _cat/indices?v command to view the shard distribution, identify the yellow index, and set its number of replicas to 0. After the cluster status returns to green, restore the original number of replicas for the index.

  Warning

  Setting the number of replicas to 0 can cause data loss if a node goes offline, so proceed with caution. After the cluster status returns to green (in about one minute), restore the original number of replicas as soon as possible.

  PUT test/_settings
  {
    "index" : {
      "number_of_replicas":"0"
    }
  }                                

Troubleshoot a red cluster status

A red cluster status occurs when a node hosting a primary shard is unavailable, preventing the shard from being allocated. Run the GET /_cat/indices?v command to view the shard distribution and identify the index with a red status. Use the following table to troubleshoot the issue.

You can also troubleshoot using Kibana monitoring or the Advanced Monitoring and alerting feature.

High cluster CPU utilization

Handle high disk usage in an Elasticsearch cluster

Handling high memory usage in Elasticsearch

Rebalancing uneven shards

High cluster load and java.lang.StackOverflowError

This error indicates a Lucene stack overflow and is caused by queries using regular expressions or fuzzy matching. This issue is fixed in Elasticsearch 6.0 and later. To resolve this, upgrade the cluster version or optimize your query statement. java.lang.StackOverflowError for the entire cluster.

High fielddata usage

Use advanced monitoring to track fielddata cache usage. Investigate if a node's fielddata cache exceeds one gigabyte (GB). To troubleshoot, run the GET /_nodes/stats/indices/fielddata?fields=* command to view the memory usage of fielddata on each node. You can also run the GET _cat/fielddata command to identify which fields have high fielddata usage, and then examine the slow log to determine which queries are running on those fields.

If your use case involves these scenarios, optimize your query statements and field types. As a short-term solution, run the POST _cache/clear?fielddata=true command to clear the cache and reduce memory usage. Troubleshoot and resolve high fielddata memory usage.

Check JVM heap memory

Run GET _nodes/stats/jvm?pretty. By default, the heap memory is half of the cluster memory. This setting cannot be changed.

Adjust queue size

Query or export data by time range

To query data for a specific time range, use a range query.

To export data for a specific time range, filter it with Logstash. Logstash configuration file.

Size limit for bulk requests

Yes. The default maximum payload for a single bulk request is 100 MB (HTTP settings). Total size depends on document size and complexity, not just count. If your documents are large, reduce the number per batch. Start with 5-15 MB batches and increase gradually. Using and Sizing Bulk Requests.

Handling time discrepancies in Elasticsearch query results

Slow or timed-out Elasticsearch queries

Resolving the "Data too large" error

• Cause

  This error occurs when a high write throughput triggers the circuit breaker, indicating the cluster has insufficient resources to handle the volume of write requests.

• Solution

  Important

  If you cannot perform these operations, stop all query and write requests and then force-restart the cluster. After the cluster returns to a normal state, follow the steps below.

  1. Run thePOST /<index_name>/_cache/clear?fielddata=true command to clear the index cache. If the issue persists, proceed to the next step.

  2. Run theGET /_cat/indices?v command to check whether shards are unevenly distributed. How do I rebalance unevenly distributed shards?. If the issue persists, proceed to the next step.

  3. Reduce concurrent writes, delete unnecessary indexes to release resources, and minimize the use of Kibana monitoring. We recommend using advanced monitoring instead.

     To disable Kibana monitoring, run the following command:

     PUT _cluster/settings
     {
     	"persistent": {
     		"xpack.monitoring.collection.enabled": false
     	}
     }

     If the issue persists, proceed to the next step.

  4. Upgrade the configuration of the cluster to increase its capacity.

Bulk-deleting indexes

"index uuid conflicted" error

This is a known issue. To resolve it, upgrade your instance's kernel version to 1.5.0 or later. See Upgrade cluster versions for instructions.

Modify index.max_result_window

In Elasticsearch, the default value of the index.max_result_window parameter is 10000. This parameter defines the maximum number of documents (from + size) that can be returned in a paged query. If you query for a number of documents that exceeds this value, the following error is returned: Result window is too large, from + size must be less than or equal to: [10000].

PUT /my_index/_settings  
{  
  "index": {  
    "max_result_window": 50000  
  }
}

Resolving the "more than 1 type" error

Query documents in an index

GET _search
{
   "query": {
   "match_all": {}
  }
}

Alternatively, use the Discover page in Kibana. To use this page, you must first create an index pattern. For detailed instructions, see the Kibana Guide.

After I insert a document, an immediate update fails with a 404 error indicating the record cannot be found. What should I do?

In Elasticsearch, a newly written document is not immediately visible for search and update operations. Each index has a refresh_interval setting that controls how often the index is refreshed to make recently written documents visible. If you insert a document and immediately try to update it, the update request can be rejected with a 404 error because the document has not yet been refreshed into the visible index segments.

PUT /<index_name>/_settings
{
  "index": {
    "refresh_interval": "5s"
  }
}

Plan cluster resources, specifications, and shards

Use the capacity planning tool on the purchase page to estimate the required specifications based on your business needs. specification and storage capacity evaluation.

Configuration parameters of an Elasticsearch instance

You can view the parameters on the instance's Basic Information page. View Basic Information of an Instance.

When using a Transport Client to access an Elasticsearch instance, set cluster.name to the instance ID. Transport Client (5.x).

Impact of changing cluster configuration

Changing the cluster configuration triggers a rolling restart. Service typically remains available if the cluster is healthy (green), each index has at least one replica, and resource utilization is moderate. Check metrics on the Cluster Monitoring page — service is likely uninterrupted when node CPU is around 80%, heap memory around 50%, and load_1m below the vCPU count. Perform changes during off-peak hours.

Automatic shard reallocation after node changes

Yes. When the number of data nodes changes, the cluster automatically reallocates shards. However, this automatic reallocation does not guarantee a perfectly even data distribution. Due to factors such as index size, the number of shards, and the number of nodes, data may remain unbalanced across shards after the reallocation. To troubleshoot and resolve uneven shard allocation, see Analysis and solutions for unbalanced cluster load.

Changing the cloud disk type

Yes. You can upgrade or downgrade the cloud disk type. The performance tiers, ordered from highest to lowest, are ESSD, Standard SSD, and Ultra Disk.

Convert Elasticsearch nodes to cold nodes

No. This operation can destabilize the cluster. "Hot-Warm" Architecture in Elasticsearch 5.x.

Downgrade an instance type

Yes. For instructions, see Scale in the data nodes of a cluster or Downgrade a cluster.

Handle temporary traffic surges

"UpgradeVersionMustFromConsole" error

This error indicates that the requested version upgrade is not supported. Alibaba Cloud Elasticsearch supports direct upgrades only along the following version paths: from 5.5.3 to 5.6.16, from 5.6.16 to 6.3.2, and from 6.3.2 to 6.7.0.

Elasticsearch upgrade duration

The upgrade duration depends on your cluster's data volume, data structure, and specifications. A typical upgrade takes about an hour.

Impact of Elasticsearch version upgrades

During an upgrade, the cluster remains readable and writable, but no other changes are allowed. Upgrade during off-peak hours. Upgrade cluster versions.

Configuration upgrade failure or timeout

The most common cause is an unhealthy cluster state. Stop all queries and writes, then follow What to do if an Elasticsearch cluster has a red status due to high load? to resolve it. After recovery, retry the upgrade. Alternatively, perform a forced update to bypass the health check — this can disrupt service, so proceed with caution.

If the configuration upgrade fails for other reasons, troubleshoot and resolve the issue based on the error message.

Cluster reconfiguration failure

Configure http.max_content_length and discovery.zen.ping_timeout

Switching VPC for an Elasticsearch cluster

No. To switch the VPC for an Elasticsearch cluster, you must purchase a new cluster in the target VPC, migrate your data, and then release the original cluster.

Add data nodes during an upgrade

No. You cannot directly add data nodes to a cluster that was created without them. Instead, you must create a new cluster and migrate your data to it. For specific migration solutions, see the Migration Solution Selection Guide.

Data loss when changing cloud disk type

No, existing data will not be lost. However, new data written during the upgrade may be lost. Perform the upgrade during off-peak hours or after stopping write operations. upgrade cluster configuration.

Unhealthy cluster with green status during upgrade

The cluster may contain closed indexes. Run the POST /<index_name>/_open command to open the affected indexes. Upgrade cluster configuration.

vCPU upgrades and data migration

No. Upgrading or downgrading vCPUs triggers a blue-green update, changing node IP addresses and migrating data from the original nodes to the new ones.

If I upgrade the performance level of an ESSD (for example, from PL1 to PL2), does it immediately resolve I/O bottlenecks, and how long does it take to take effect?

Upgrading the performance level (PL) of an ESSD increases the IOPS available to your cluster, but the change does not take effect immediately. The upgrade involves migrating the underlying data to storage that supports the new performance level.

There is no fixed duration for the change to take effect. The actual time depends on factors such as your data volume, the current instance state, and the change method used. Some paths, such as a blue-green update, take longer because nodes are recreated and data is migrated from the original nodes to new ones.

When Elasticsearch experiences an I/O bottleneck, should I scale out data nodes or upgrade the disk type?

Whichever option you choose, ensure that shards remain evenly distributed across data nodes. Uneven shard distribution can create hotspots that limit the actual performance improvement, even after scaling out or upgrading the disk type.

Cold data node downgrade restrictions

Downgrading a cluster has several restrictions. For example, the target specifications must have at least 50% of the current vCPUs and memory. You cannot downgrade to 1 vCPU/2 GiB, 2 vCPUs/2 GiB, 4 vCPUs/4 GiB, or 2 vCPUs/4 GiB. Downgrade a cluster.

If your cluster does not meet the downgrade requirements, you can create a new instance, migrate your data, and then release the original cluster. For data migration solutions, see Migration method selection.

Insufficient resources or abnormal shard allocation error

PUT _template/template_1
{
  "template": "*",
  "settings": {
    "number_of_replicas": 2
  }
}  

Error when scaling in data nodes

Use the Cluster Diagnosis feature to check the cluster's health, then resolve the error based on the diagnostic results. Perform a diagnostic on an Elasticsearch cluster.

"Reserved nodes must be greater than 2" error

To ensure cluster reliability, you must reserve at least two data nodes. For a multi-zone cluster, you must also keep at least two nodes in each zone and maintain an equal number of nodes across all zones. If you cannot meet these requirements, you must select different nodes to scale down or migrate, or upgrade the cluster configuration.

Cluster configuration error during data node scale-in

Run the GET _cluster/settings command to check the cluster configuration. Look for a setting that disables data distribution: "cluster.routing.allocation.enable" : "none". If this setting is present, temporarily change it to "cluster.routing.allocation.enable" : "all". If this change affects other operations, revert the setting after the scale-in.

Troubleshoot failures caused by auto_expand_replicas

• Cause

  Using the X-Pack access control feature can cause this issue. In earlier Elasticsearch versions, this feature applies the "index.auto_expand_replicas" : "0-all" setting to the .security index by default. This setting leads to failures in data migration or node scale-in operations.

• Solution

  1. Check the index settings.

     GET .security/_settings

     The command returns a result similar to the following:

     {
       ".security-6" : {
         "settings" : {
           "index" : {
             "number_of_shards" : "1",
             "auto_expand_replicas" : "0-all",
             "provided_name" : ".security-6",
             "format" : "6",
             "creation_date" : "1555142250367",
             "priority" : "1000",
             "number_of_replicas" : "9",
             "uuid" : "9t2hotc7S5OpPuKEIJ****",
             "version" : {
               "created" : "6070099"
             }
           }
         }
       }
     }

  2. Modify the configuration using one of the following methods.

     • Method 1

       PUT .security/_settings
       {
         "index" : {
           "auto_expand_replicas" : "0-1"
         }
       }

     • Method 2

       PUT .security/_settings
       {
         "index" : {
           "auto_expand_replicas" : "false",
           "number_of_replicas" : "1"
         }
       }

       Important

       The number_of_replicas parameter sets the number of replicas for each primary shard in an index. Configure this parameter based on your business requirements, ensuring the value is at least 1 and does not exceed the number of available data nodes.

Clear the Elasticsearch cache

Change Elasticsearch cluster zone

See Migrate Elasticsearch nodes in a zone or upgrade the deployment mode of an Elasticsearch cluster.

Independent disk scaling

JVM parameter modification

Alibaba Cloud Elasticsearch uses the JVM parameters recommended by Elasticsearch. These parameters cannot be modified. By default, JVM heap size is 50% of cluster memory, up to 32 GB. Heap size settings.

Customize IK analyzer dictionaries

You can add or delete dictionary content using the cold or hot update features of the IK analysis plug-in. Use the IK analysis plug-in (analysis-ik).

IK analysis plug-in: "ik startOffset" error

This error is caused by a bug in Elasticsearch 6.7 and requires a cluster restart. Restart a cluster or node.

Retrieve lost on-premises IK dictionary files

No. You cannot retrieve the file — only delete or update dictionary files on the cluster management page. Download the official main dictionary and stopword files, replace their content with your system dictionary terms, and upload them.

Apply IK dictionary changes to existing data

If an index uses IK tokens, dictionary changes from a cold or hot update apply only to new data. To apply changes to existing data, reindex. Configure YML parameters.

Standard value for full GC

To determine if a full GC (which cleans the entire heap memory) is problematic, analyze service latency and compare current performance against historical baselines. The CMS collector starts garbage collection when memory usage reaches 75% to leave headroom for traffic bursts.

Uninstalling unused built-in plugins

You can uninstall a built-in plugin if Uninstall appears in its Actions column. For instructions, see Install or uninstall a built-in plugin.

IK dictionary consistency

Yes. The Alibaba Cloud Elasticsearch IK analysis plug-in includes the same built-in dictionaries as the open-source version. IK Analysis for Elasticsearch.

Custom plug-in network access

No, custom plug-ins cannot access external networks. To access an external file, upload it to Object Storage Service (OSS) and read it from there.

Custom plugin hot updates

No. To implement hot updates, reference the IK analysis plugin method. IK Analysis for Elasticsearch.

Analysis-aliws configuration and file format

For detailed configuration steps, see Use the analysis-aliws plug-in.

Synonyms, IK tokens, and AliNLP tokens

IK analysis mode configuration

Alibaba Cloud Elasticsearch comes with the IK analyzer plugin pre-installed, which cannot be uninstalled. Apply cold and hot updates to the default dictionary and use it when creating index mappings. Use the IK analyzer plugin (analysis-ik).

Built-in Chinese tokenizers

Alibaba Cloud Elasticsearch supports the built-in Chinese tokenizers analysis-ik and analysis-aliws. You can use them after configuring their corresponding dictionaries.

Automatic updates for dictionary files in OSS

No. Alibaba Cloud Elasticsearch does not currently support hot updates for dictionary files from OSS. After a dictionary file in OSS is modified, you must manually upload and configure it to apply the changes. Furthermore, for an index configured with an IK analyzer, synonyms, or an AliNLP analyzer, the updated dictionary affects only new data. To apply the changes to all existing data, you must reindex.

IK analyzer and remote dictionaries

No. The Alibaba Cloud Elasticsearch IK analyzer supports uploading and updating dictionaries. Use the IK analyzer plugin (analysis-ik). Remote dictionaries and related configurations, such as IKAnalyzer.cfg.xml, are not supported. To use a remote dictionary, see Implement dynamic synonym updates in Alibaba Cloud Elasticsearch.

Install the vector search plugin (aliyun-knn) for Elasticsearch 7.10

Restart impact on services

Typically, if the cluster load is low and indexes have replicas, the cluster continues serving traffic during a restart. However, timeouts can occur during high-concurrency forced restarts, on heavily loaded clusters, without replica shards, or under heavy workloads. Implement a retry mechanism in your client and restart during off-peak hours.

Retention period for .security indexes

Store Elasticsearch logs on-premises

Use the ListSearchLog API to retrieve cluster logs and save them on-premises.

Viewing Elasticsearch logs

Configure slow logs and lower the time threshold. Query logs.

Configure and view slow logs

Retrieve slow logs programmatically

Call the ListSearchLog API to retrieve slow logs programmatically.

Find clients of an Elasticsearch cluster

For limitations, precautions, and how to enable these logs, see Query logs.

Snapshot restoration across different versions

With automatic backups, you can only restore a snapshot to the original cluster or use the cross-cluster snapshot restoration feature. Automatic Backup and Restoration and Cross-Cluster Backup and Restoration with an OSS Bucket.

With manual backups, you can restore a snapshot to other Elasticsearch clusters. Restore between clusters of the same version to avoid compatibility issues. Manual Backup and Restoration.

Unhealthy cluster state during backup for version upgrade

If the cluster state is unhealthy, you cannot create a snapshot backup. Restore the cluster state to green first.

Automatic backup without a configured OSS bucket

Alibaba Cloud Elasticsearch provides an OSS bucket for your cluster by default. To view the automatic snapshots, Log on to the Kibana console and run the GET _snapshot/aliyun_auto_snapshot/_all command.

Shard failure during a snapshot restore when the POST /_cluster/reroute?retry_failed=true command fails

When you restore data from a snapshot, you may encounter an error similar to the following:

POST /_snapshot/aliyun_snapshot_from_instanceId/es-cn-instanceId_datetime/_restore
{
    "indices": "myIndex",
    "settings": {
    "max_restore_bytes_per_sec" : "150mb" 
    }
}

Export data to a local machine

Back up data from your Elasticsearch cluster to OSS, then use the Download objects feature in the OSS console to download the backup locally.

Restore snapshots across clusters

For specific instructions, limitations, and notes on cross-cluster snapshot restoration, see Back up and restore data with a cross-cluster OSS repository. For cross-region data migration within the same account, you can use commands that are used to create manual snapshots for index data and restore data from the snapshots. For more migration methods, see Migration method selection.

Data backup options

Data backup covers available options, use cases, and limitations.

Configure X-Pack Watcher alerts

Handle GC memory allocation failures

Grafana monitoring

Available only for ES 6.7.0 instances with kernel version 1.2.0 or later. View monitoring metrics.

Cluster status metric values

Elasticsearch disk usage

View disk usage in the console or with X-Pack monitoring in Kibana. Metrics and exception handling and Configure monitoring indices.

CMS GC "promotion failed" error

You might encounter this error when using the CMS garbage collector in Alibaba Cloud Elasticsearch. It typically indicates that there is insufficient space in the old generation, causing object promotion to fail. To resolve this issue, consider the following:

• Monitoring and log analysis

  • Check the GC log for frequent CMS GC or Full GC operations to determine if the CMS GC promotion failed error is caused by insufficient space in the old generation.

  • Log in to the Log Query page in the Alibaba Cloud Elasticsearch console and search for logs containing promotion failed to find the root cause.

• Adjust heap memory and garbage collector settings

  • If your Elasticsearch version is 6.7.0 or later and each data node has 32 GB or more of memory, switch the garbage collector to G1 for better performance.

  • Based on your cluster's resource usage and business requirements, evaluate whether to increase the instance memory.

• Tuning suggestions

  If memory-related issues persist, evaluate your index data volume, query load, and cluster resource configuration. If necessary, contact Alibaba Cloud technical support for professional tuning guidance.

Client connection and open source comparison

To connect to an Alibaba Cloud Elasticsearch cluster, use its internal or public endpoint — equivalent to an open source cluster's address. Use a client to access an Alibaba Cloud Elasticsearch cluster.

Disabling basic authentication

No, you cannot disable basic authentication. It is a Kibana authentication mechanism provided by X-Pack, which is included in all Elasticsearch clusters.

Internal access: Same VPC, different AZs

Yes. An ECS instance can access an Elasticsearch cluster over the internal network as long as they are in the same VPC.

Configure an IP address whitelist

Connecting to an Elasticsearch cluster from the Internet

To connect to an Elasticsearch cluster from the Internet, use its public endpoint and configure an IP address whitelist. You must also configure the domain name, username, and password. Use a client to access an Alibaba Cloud Elasticsearch cluster.

Cannot access Elasticsearch: Connection refused

Password reset and Elasticsearch access

Resolve connection failures with Elasticsearch-Head 5.0.0

Index FAQ

Close an index

POST /<index_name>/_open # Open an index
POST /<index_name>/_close # Close an index