How to use the faster-bulk plug-in-Elasticsearch(ES)-阿里云帮助中心

The faster-bulk plug-in aggregates bulk requests by size and time interval to prevent small-batch writes from blocking the write queue. It is suited for high-throughput scenarios with numerous index shards. The plug-in is disabled by default and must be manually enabled. Because aggregation adds latency, it is not recommended for low-latency write scenarios.

Usage notes

You must install the plug-in before using it. For more information, see Install or uninstall a built-in plug-in.

Write performance

The following data shows faster-bulk plug-in performance in a specific test environment.

Test environment: Three 16-core, 64 GB data nodes and two 16-core, 64 GB independent client nodes, using the official esrally nyc-taxis dataset (650 bytes per document), with apack.fasterbulk.combine.interval set to 200 ms.

Translog status	Without plug-in	With plug-in	Performance improvement
Synchronous (default)	182,314/s	226,242/s	23%
Asynchronous	218,732/s	241,060/s	10%

Enable bulk aggregation

PUT _cluster/settings
{
   "transient" : {
      "apack.fasterbulk.combine.enabled":"true"
   }
}

Configure aggregation parameters

A data write is triggered when either the cumulative bulk request size or the aggregation time interval on a single data node reaches the configured threshold.

PUT _cluster/settings
{
   "transient" : {
      "apack.fasterbulk.combine.flush_threshold_size":"1mb",
      "apack.fasterbulk.combine.interval":"50"
   }
}

Parameter	Description	Default
apack.fasterbulk.combine.flush_threshold_size	The maximum cumulative size of aggregated bulk requests on a single data node.	1mb
apack.fasterbulk.combine.interval	The maximum time interval for aggregating bulk requests. Unit: ms.	50

In high-concurrency scenarios with large data volumes, increase the aggregation size or time interval within your cluster's capacity to prevent bulk requests from blocking the write queue.

Directed routing

When batch-writing documents without a routing value or primary key (_id), enable directed routing at the cluster or index level to improve write speed. Write requests that already specify a routing value or primary key (_id) are not affected.

To enable directed routing for a cluster:

PUT _cluster/settings
{
  "persistent" : {
    "index.direct_routing.global.enable" : "true"
  }
}

To enable directed routing for a specific index:

PUT <index_name>/_settings
{
  "index.direct_routing.enable" : "true"
}

Disable bulk aggregation

PUT _cluster/settings
{
   "transient" : {
      "apack.fasterbulk.combine.enabled":"false"
   }
}