Openstore engine: Decoupled storage and compute for high-performance retrieval

Background information

In retrieval scenarios, high query performance and high availability usually require multiple replicas. Multiple replicas multiply write and storage overhead. When service traffic fluctuates, you cannot scale nodes quickly because shard data must be migrated. You must therefore reserve more compute resources than you need, which wastes resources.

The OpenStore storage engine (high-performance retrieval) is a key feature of the Alibaba Cloud Elasticsearch performance-enhanced edition and supports V7.10 and V8.17. You can create a performance-enhanced instance of the required version and enable the engine for high-performance retrieval scenarios.

Benefits

• Elastic scaling for large volumes of data — The OpenStore shared storage architecture lets you expand remote storage capacity quickly to meet growing business demands.

• Lower cost and higher performance:

  • OpenStore supports real-time data writes and updates. You do not need to configure complex index lifecycle policies. Data moves to remote shared storage automatically, which makes the engine easy to use.

  • In a single-replica scenario, storage costs are more than 40% lower than those of enterprise SSDs (ESSDs), and write performance improves by 40%.

  • In multi-replica scenarios, storage cost savings increase with the number of replicas.

Applicability

• Regions — The following regions are supported:

  • China (Hangzhou): Zones J and K

  • China (Hong Kong): Zones B, C, and D

  • China (Beijing): Zones K, F, and I

  • China (Shanghai): Zones E, G, and L

  • China (Shenzhen): Zones C, D, and E

  • US East 1 (Virginia): Zones A and B

• Instance version — Performance-enhanced instances of V7.10 and V8.17 are supported.

• Instance specifications — The minimum instance specification is 2 cores and 8 GB of memory. The minimum number of data nodes is 4.

• Instance storage capacity — The minimum local cache storage space for a single data node is 50 GB. The minimum total logical shared storage space is 160 GB.

• Billing method — Subscription and pay-as-you-go are supported.

• Number of zones — Single-zone deployments and dual-zone deployments with high availability (HA) are supported. Three-zone deployments are not currently supported.

• Data writes — By default, OpenStore sets the write threshold to 95% of shared storage usage. When the index data on a primary shard reaches this threshold, the system automatically blocks new writes to prevent cluster failures caused by storage overflow. You can call the following API operation to temporarily increase the write threshold to 98%, which gives you time to expand storage capacity:

PUT _cluster/settings
{
  "persistent": {
    "apack.openstore.quota.limit.ratio": "98%"
  }
}

Billing information

The OpenStore storage engine (high-performance retrieval) supports the subscription and pay-as-you-go billing methods. In addition to data node specification fees, storage fees consist of the following two parts:

• OpenStore shared storage space (DFS) — Remote shared storage, billed by capacity. The minimum total logical shared storage space is 160 GB.

• Local cache disk space — Local storage (an ESSD) that data nodes use to cache hot data, billed by capacity. The minimum capacity for each data node is 50 GB.

Unit prices are subject to the real-time prices displayed on the buy page. For more information about billing, see Billing rules.

Performance metrics