Performance test report: 100M-scale vectors (1024-D / 1-bit quantization)

Updated at:

PolarDB-X's intelligent search (Search engine) provides vector search capabilities based on the OpenSearch protocol. When data volumes grow too large for full-precision vectors to fit entirely in memory, 1-bit quantization (32× compression) reduces hot data that must remain memory-resident by an order of magnitude. This document tests HNSW index query throughput, latency, and recall under 1-bit quantization across Lucene and Faiss vector engines using a 100M-scale high-dimensional vector dataset, and provides comparisons with FP32 full precision along with selection guidance, offering measured data reference for technology selection, capacity planning, and parameter tuning. Tests use 128 GiB memory nodes: at this specification, full-precision vectors cannot be fully cached, allowing the compression benefits of quantization on hot data to be accurately demonstrated.

Key findings

  • 1-bit quantization compresses hot data that must remain memory-resident (HNSW graph + quantized vectors) from over 100 GiB down to 13.5 GiB (Lucene) or 21.6 GiB (Faiss)—an order-of-magnitude reduction. SeeStorage and memory.

  • Faiss significantly outperforms Lucene in throughput at equal recall, with the advantage growing as recall requirements increase (38%–144%). SeeThroughput at equal recall.

  • Faiss requires tuning both ef_search and oversample_factor parameters to reach its ceiling; Lucene has only one effective knob (ef_search), making tuning simpler but with a lower ceiling. SeeParameter trade-offs.

  • Compared with FP32 full precision, Faiss-side 1-bit outperforms in most scenarios (reaching 145% in the high-recall top-10 range), while Lucene-side generally trails by 7–25%. SeeQPS at the same recall.

  • Test matrix navigation:

    Engine

    Quantization

    Build parameters

    top-k

    Search breadth control

    Concurrency levels

    Section

    Lucene HNSW

    SQ 1-bit

    M=32 / ef_construction=200

    10 and 100

    5 tiers + 5 tiers

    11 tiers

    Lucene SQ 1-bit test data

    Faiss HNSW

    BQ 1-bit

    M=32 / ef_construction=200

    10 and 100

    25 tiers + 25 tiers

    11 tiers

    Faiss BQ 1-bit test data

Scope of application

Cluster specifications and versions

Component

Configuration

Search node

32 cores, 128 GiB × 2

JVM heap memory

31 GiB/search node

Index shards

2 primary shards + 0 replicas

Engine version

3.6

Vector engine

Lucene HNSW, Faiss HNSW

Client

96 cores, 371 GiB, directly connected over the internal network to search nodes

Dataset

Category

Item

Details

Dataset

MSMARCO V2.1 (CohereLabs/msmarco-v2.1-embed-english-v3)

1,024-D

Total vectors

113,520,750

100M-scale

Data precision

1-bit quantization (storage includes full-precision copy for reranking)

32× compression

Distance metric

cosine

—

Query vectors

1,677

Independent test set

Ground truth set

Included with the dataset, providing the top 10 and top 100 exact nearest neighbors for each query

Precomputed by brute-force search

top-k

10 and 100

Two independent test sets

The recall baseline comes from the ground truth set included with the dataset. The top-10 and top-100 test sets each use an independent answer set, providing the top 10 and top 100 exact nearest neighbors sorted by similarity in descending order for each query.

Index parameters

Parameter

Lucene SQ 1-bit

Faiss BQ 1-bit

engine

lucene

faiss

mode

on_disk

on_disk

Quantization

1-bit

1-bit (encoder: binary, bits=1)

M

32

32

ef_construction

200

200

Distance metric

cosine

cosine

Shards / Segments

2 primary, 0 replicas / 1 segment per shard

2 primary, 0 replicas / 1 segment per shard

Test methodology

Query benchmarks use community edition VectorDBBench 1.0.22. Recall is measured against the ground truth set included with the dataset; throughput and latency are from actual readings during the concurrent phase. All data in this report was collected under VectorDBBench default settings. For the impact of HTTP request body compression and other settings on data comparability, seeScope of application.

Query parameters:ef_search and oversample_factor

Queries under 1-bit quantization proceed in two stages: first, 1-bit vectors quickly screen a batch of candidates (coarse ranking); then full-precision vectors are read to precisely rerank these candidates (reranking). Two query parameters control these stages respectively:

Parameter

Function

Description

ef_search

Controls coarse-ranking search breadth

Higher values yield candidates ofhigher qualityhigher quality

oversample_factor

Controls reranking candidate count (approximately k × oversample_factor), lower limit 100, upper limit 10000

Higher values provide more candidates for reranking in terms ofquantitymore

The two engines respond differently to these parameters, which directly affects tuning approach and explains why the two chapters use different parameter matrices:

  • Lucene: The larger of the two parameters takes effect, simultaneously affecting both coarse ranking and reranking—effectively a single knob. Therefore, the Lucene chapter fixes oversample_factor=1.0and relies on ef_search to step through tiers.

  • Faiss: The two parameters take effect at different stages and must be tuned together for optimal results. Therefore, the Faiss chapter performs a cross-scan of both parameters (5 ef_search × 5 oversample_factor).

Data preparation

This test shares the same dataset as thePerformance test report: 100M-scale vectors (1024-D / FP32). For dataset acquisition and format conversion steps, refer to that report'sData preparationsection: After obtaining the raw dataset from HuggingFace, convert it to the three Parquet files required by VectorDBBench (train.parquet / test.parquet / neighbors.parquet) and place them in the same data directory, linked at benchmark time via --custom-dataset-dir .

Execution flow

Each benchmark command corresponds to a fixed engine × top-k × query parameter configuration, running through 11 concurrency levels sequentially. After command submission, the tool automatically completes all of the following steps without manual intervention:

Stage

Tool behavior

Build index

Based on --engine, --m, --ef-construction, --on-disk and other parameters to create a quantized index

Import

Write all vectors

Segment merging

Trigger force merge to merge each shard into a single segment (--force-merge-enabled Default true, --number-of-segments Default 1; this test uses default values for both)

Recall measurement

Issue queries serially and compare top-k results against the ground truth set

Throughput and latency measurement

Runs benchmarks across 11 concurrency levels sequentially, 60 seconds per level

Once built, the same index can be reused for measurements with different query parameters. Each engine's index is built independently on the same cluster; during testing, only the index under test is opened while others are closed to prevent mutual page cache eviction.

Benchmark command

Both engines and all parameter combinations share the following command skeleton (parameters held constant).

Note
  • In the command, awsopensearch is the VectorDBBench subcommand for OpenSearch-compatible endpoints (PolarSearch is compatible with the OpenSearch protocol).

  • In the command, <endpoint>, <username>, <password> are placeholders that must be replaced with the cluster access endpoint and account credentials.

vectordbbench awsopensearch \
  --case-type PerformanceCustomDataset \
  --custom-dataset-name msmarco100m --custom-dataset-dir /data/msmarco_100m \
  --custom-dataset-size 113520750 --custom-dataset-dim 1024 --custom-dataset-file-count 1 \
  --custom-dataset-metric-type cosine --custom-dataset-with-gt \
  --host <endpoint> --port 9200 \
  --user <username> --password <password> \
  --metric-type cosine \
  --m 32 --ef-construction 200 \
  --number-of-shards 2 --number-of-replicas 0 \
  --num-concurrency 16,32,48,64,96,144,192,256,320,384,450 \
  --concurrency-duration 60 \
  --search-serial --search-concurrent

The following parameters are appended to enumerate all 660 data points (Lucene 10 configurations × 11 concurrency levels + Faiss 50 configurations × 11 concurrency levels):

Parameter

Lucene section

Faiss section

--engine

lucene

faiss

--on-disk

Required; enables two-stage retrieval and 1-bit quantization

Same as above

--quantization-type

Not applicable (1-bit is implicitly determined by --on-disk )

bq

--k

10 or 100

10 or 100

--ef-search

100/200/300/400/500

100/200/300/400/500

--oversample-factor

Not set (default 1.0)

  • For k=10: 10/30/50/70/100

  • For k=100: 1/3/5/7/10

--skip-load --skip-drop-old

Omit on first index build; add on subsequent runs reusing the same index to skip index build and import

Same as above

Faiss BQ uses 1-bit quantized vectors for coarse ranking, then reads full-precision vectors to correct the order during reranking.

Impact of HTTP compression on the measured data

When connecting to a standard OpenSearch endpoint, VectorDBBench enables request body gzip compression by default (client configuration option http_compress), All data in this report was measured under this default configuration. Disabling compression yields higher throughput, with diminishing gains as concurrency increases. Therefore, data obtained with different benchmark tools or compression settings cannot be directly compared. Before comparison, ensure both sides are aligned on compression configuration, concurrency model, top-k, and timing methodology.

Lucene SQ 1-bit test data

Storage and memory

Primary shards measured at 460.3 GiB (shard0 230.1 GiB + shard1 230.2 GiB, each merged to a single segment). Memory breakdown:

Data type

Size per node

Hot path

Description

HNSW graph + 1-bit quantized vectors

13.5 GiB

Yes

  • Read during coarse ranking, resident in page cache

  • Graph 6.75 GiB + quantized vectors 6.75 GiB, ~14% of node page cache (page cache estimated as 128 GiB node memory minus 31 GiB JVM heap)

Full-precision vectors (for reranking)

216.3 GiB

On demand

Read during reranking, not permanently resident

Query performance summary table

Notes on table data:

  • QPS stands for queries per second.

  • AVG, P95, and P99 represent the average, 95th percentile, and 99th percentile of query latency, respectively, all in milliseconds.

  • Column header cN indicates a concurrency level of N. For example, c16 means concurrency level 16, and c32 means concurrency level 32.

top-k = 10

Highest measured throughput 13,150 QPS(recall 0.9283), highest recall 0.9729(throughput 3,974 QPS).

ef_search

Recall

Metric

c16

c32

c48

c64

c96

c144

c192

c256

c320

c384

c450

100

0.9283

QPS

4,551

8,221

10,444

11,443

12,608

13,023

13,091

13,150

13,038

12,936

12,925

AVG

3.5

3.9

4.6

5.6

7.6

11.0

14.6

19.3

24.2

29.1

34.0

P95

4.0

4.6

5.8

7.4

10.7

17.5

25.9

36.9

42.4

48.0

53.8

P99

4.4

5.3

6.9

9.2

13.7

27.2

47.9

66.7

76.3

81.8

86.3

200

0.9530

QPS

3,968

6,737

7,670

8,058

8,154

8,141

8,140

8,139

8,132

8,135

8,055

AVG

4.0

4.7

6.2

7.9

11.7

17.6

23.4

31.2

38.9

46.6

54.9

P95

4.7

5.8

8.5

11.7

22.2

34.1

41.4

50.5

59.7

67.8

77.0

P99

5.3

6.9

10.6

14.6

34.8

56.0

66.5

74.8

84.6

90.5

99.9

300

0.9637

QPS

3,465

5,705

5,897

6,003

5,961

5,952

5,915

5,912

5,922

5,886

5,870

AVG

4.6

5.6

8.1

10.6

16.1

24.1

32.3

42.9

53.6

64.8

75.9

P95

5.5

7.3

12.0

17.8

32.2

43.6

52.6

65.7

81.5

92.9

107.7

P99

6.3

8.6

14.9

24.0

49.0

64.5

73.0

82.3

97.7

111.7

129.9

400

0.9695

QPS

3,052

4,697

4,705

4,675

4,696

4,684

4,688

4,682

4,717

4,696

4,560

AVG

5.2

6.8

10.2

13.7

20.4

30.7

40.8

54.3

67.5

81.2

97.9

P95

6.4

9.0

16.2

25.9

39.3

51.5

63.6

84.1

103.2

123.9

144.8

P99

7.2

10.7

20.6

36.0

55.9

69.0

79.3

96.9

118.1

144.0

174.3

500

0.9729

QPS

2,746

3,972

3,974

3,942

3,937

3,942

3,931

3,873

3,867

3,874

3,871

AVG

5.8

8.0

12.1

16.2

24.3

36.5

48.7

65.8

82.3

98.5

115.2

P95

7.2

11.0

19.2

31.2

46.1

58.9

76.0

105.2

133.4

157.6

170.7

P99

8.2

13.3

24.1

43.5

63.6

77.2

89.3

113.9

145.7

176.7

206.0

top-k = 100

The only variable compared to the top-k=10 group is k changed from 10 to 100; all other parameters, index, dataset, tool, concurrency levels, and timing methodology remain identical. Recall is measured as Rec

Highest measured throughput for top-100 12,233 QPS(recall 0.8246), highest recall 0.9598(throughput 4,101 QPS).

ef_search

Recall

Metric

c16

c32

c48

c64

c96

c144

c192

c256

c320

c384

c450

100

0.8246

QPS

4,270

7,715

10,198

11,363

12,233

9,846

9,173

8,670

8,546

9,028

10,011

AVG

3.7

4.1

4.7

5.6

7.8

14.6

20.8

29.2

36.9

41.9

44.3

P95

4.2

4.8

5.7

7.1

11.7

42.2

51.2

61.0

66.9

68.9

65.2

P99

4.6

5.5

6.8

8.7

17.1

49.8

58.1

67.1

73.6

77.6

79.4

200

0.9109

QPS

3,703

6,621

8,069

7,522

6,106

6,733

6,683

6,845

5,867

5,960

7,767

AVG

4.3

4.8

5.9

8.5

15.7

21.3

28.6

37.1

54.1

63.9

57.2

P95

5.0

5.8

7.5

17.0

41.2

40.2

47.2

55.0

76.3

84.4

75.2

P99

5.6

6.8

8.9

27.6

46.9

48.8

59.4

70.1

86.1

97.4

99.0

300

0.9389

QPS

3,294

5,638

6,089

5,558

5,758

5,764

5,183

5,151

5,602

5,618

4,693

AVG

4.9

5.7

7.9

11.5

16.6

24.9

36.8

49.5

56.5

67.6

95.1

P95

5.7

7.0

11.2

16.7

22.3

32.1

55.1

72.1

84.4

99.1

127.8

P99

6.4

8.1

15.0

28.8

24.9

39.5

67.8

86.8

96.4

114.6

158.8

400

0.9517

QPS

2,925

4,831

4,474

4,594

4,590

4,590

4,609

4,506

4,581

4,578

4,428

AVG

5.5

6.6

10.7

13.9

20.9

31.3

41.4

56.6

69.4

83.3

100.8

P95

6.6

8.5

20.5

18.3

27.3

47.2

64.9

92.2

112.7

136.3

154.8

P99

7.4

9.8

26.8

21.7

31.3

51.8

70.4

98.2

121.8

148.0

178.4

500

0.9598

QPS

2,647

4,101

3,855

3,780

3,790

3,782

3,700

3,708

3,709

3,643

3,644

AVG

6.0

7.8

12.4

16.9

25.3

38.0

51.6

68.8

85.9

104.7

122.5

P95

7.5

10.3

17.7

22.1

33.0

58.0

83.2

114.2

144.5

177.3

208.5

P99

8.4

12.1

23.3

26.1

37.7

60.5

86.0

118.3

150.8

186.2

219.7

Data analysis

  1. Recall increases monotonically with ef_search monotonically, and the improvement grows larger as top-k increases. As ef increases from 100 to 500, top-10 recall rises from 0.9283 to 0.9729 (+4.5 percentage points), and top-100 rises from 0.8246 to 0.9598 (+13.5 percentage points). Full-precision reranking effectively compensates for ordering errors introduced by quantization, and both groups reach usable recall levels at the 100M scale.

  2. top-100 recall is lower than top-10, and the gap narrows rapidly as ef_search increases. The gap between the two groups drops from 10.4 percentage points at ef=100 to 1.3 percentage points at ef=500. The root cause is that both groups share the same reranking candidate count of 100 (the floor value): top-10 can select the best 10 from 100 candidates, while top-100 must return all 100, leaving no filtering margin during reranking. At ef=100, the search breadth exactly equals the candidate count, maximizing the gap; at ef=500, the breadth reaches 5x the candidate count, and the candidates themselves already approximate the true nearest neighbors.

  3. Throughput decreases monotonically as ef_search monotonically decreases and is largely unaffected by top-k.ef_search As ef_search increases from 100 to 500, peak throughput drops to roughly one-third (top-10: 13,150 → 3,974; top-100: 12,233 → 4,101). The difference in peak throughput between the two groups at the same tier is only −7.0% to +3.2%, because server-side search and reranking overhead is determined by search breadth and candidate count,k affecting only the response body size.

  4. top-100 reaches peak throughput at significantly lower concurrency than top-10. top-10 peaks are spread across c48–c320 (ef=100 peaks at c256, ef=400 at c320, others fall in c48–c96), while top-100 all peak between c32 and c96.ef_search The higher the value, the heavier each individual query; top-100 returns 100 results per query (10x that of top-10), resulting in greater serialization and network transfer overhead, both of which saturate the link at lower concurrency levels. The peak throughput tier for top-10 does not strictly shift forward with increasing ef_search —for example, in the ef=400 group, throughput differences across tiers from c48 onward are under 1%, indicating a saturation plateau where the peak tier position is mainly determined by measurement noise.

  5. P99 rises in a step-like manner after c96–c144. P99 shows significant jumps in the medium-to-high concurrency range across all tiers: for top-10 with ef=100, P99 jumps from 13.7 ms at c96 to 47.9 ms at c192; for top-100 with ef=100, it jumps from 17.1 ms at c96 to 49.8 ms at c144. If the workload has P99 constraints, keep concurrency at or below c96.

Note

Recall is measured during the serial phase and depends only on index parameters and ef_search , independent of concurrency, so each group reports a single value. For top-100, the low-ef tiers (ef=100–300) after peak throughput show 20%–30% throughput fluctuations, caused by the increased proportion of client-side parsing and network transfer overhead when the response body is large; analysis should focus on peak values per tier and overall trends.

Resource saturation

Measured readings during the formal test phase are as follows. "Single node" takes the maximum value across the two search nodes.

Metric

Measured value

Single-node CPU

Peak 91%

Search thread pool rejections

0 throughout

Search thread pool queue

No overflow

k-NN circuit breaker

Not triggered throughout (Lucene does not use off-heap memory)

Graph memory (off-heap)

0 (Lucene uses memory mapping and does not load into off-heap memory)

Client CPU

Not a constraint

Faiss BQ 1-bit test data

Storage and memory

Primary shards measured at 478.7 GiB (shard0 239.3 GiB + shard1 239.4 GiB, each merged into a single segment). Memory breakdown:

Data type

Size per node

Hot path

Description

HNSW graph + 1-bit quantized vectors

21.6 GiB

Yes

Required for coarse ranking, resident in off-heap memory, accounting for 29.7% of the k-NN circuit breaker limit (stored as a single file, not breakable by component)

Full-precision vectors (for reranking)

216.3 GiB

On demand

Read during reranking, not permanently resident

Query performance summary table

Notes on table data:

  • QPS stands for queries per second.

  • AVG, P95, and P99 represent the average, 95th percentile, and 99th percentile of query latency, respectively, all in milliseconds.

  • Column header cN indicates a concurrency level of N. For example, c16 means concurrency level 16, and c32 means concurrency level 32.

top-k = 10

Highest measured throughput 18,251 QPS(recall 0.9258), highest recall 0.9786 (ef_search=500 / oversample_factor=70, throughput 7,520 QPS).

ef_search

oversample_factor

Recall

Metric

c16

c32

c48

c64

c96

c144

c192

c256

c320

c384

c450

100

10

0.9258

QPS

4,998

8,805

11,874

13,841

15,989

17,228

17,885

17,827

18,170

18,251

18,153

AVG

3.2

3.6

4.0

4.6

6.0

8.3

10.6

14.2

17.4

20.7

24.3

P95

3.6

4.3

4.9

6.0

8.4

12.3

17.2

32.5

38.1

49.1

62.7

P99

6.9

7.3

7.9

8.6

11.0

15.8

23.7

53.4

70.8

96.3

117.0

30

0.9364

QPS

2,901

8,034

10,754

12,085

13,831

14,732

14,999

15,257

15,156

15,099

14,353

AVG

5.5

4.0

4.4

5.3

6.9

9.7

12.7

16.6

20.8

25.0

30.8

P95

4.5

4.7

5.6

7.3

9.8

14.4

19.3

26.5

34.1

42.2

53.4

P99

79.6

7.7

8.3

9.7

12.5

18.3

25.5

37.2

53.4

89.2

118.3

50

0.9367

QPS

2,681

7,369

9,895

11,078

12,096

12,704

13,029

12,976

12,940

12,816

12,735

AVG

6.0

4.3

4.8

5.8

7.9

11.3

14.6

19.5

24.4

29.5

34.5

P95

6.9

5.2

6.4

8.1

11.5

17.2

23.2

32.7

41.5

50.9

59.5

P99

80.7

8.2

8.9

10.5

14.7

22.6

31.2

49.0

72.5

89.1

96.9

70

0.9357

QPS

2,251

7,080

8,946

10,050

10,715

11,381

11,536

11,606

11,418

11,318

11,215

AVG

7.1

4.5

5.4

6.3

8.9

12.6

16.5

21.9

27.8

33.4

39.4

P95

7.7

5.4

7.4

9.1

13.3

19.8

27.9

37.6

48.1

57.9

67.0

P99

85.1

8.5

9.8

11.7

16.8

26.2

40.3

57.0

80.1

90.9

98.7

100

0.9361

QPS

1,886

6,370

8,100

8,757

9,422

9,780

9,856

9,684

9,619

9,471

9,369

AVG

8.5

5.0

5.9

7.3

10.1

14.7

19.4

26.2

32.9

40.0

47.3

P95

7.9

6.0

8.3

10.8

15.7

25.3

34.2

46.4

55.2

65.9

76.7

P99

123.3

9.0

10.6

13.4

20.3

35.2

51.1

70.7

81.1

93.3

100.5

200

10

0.9457

QPS

4,476

8,180

10,801

12,389

13,852

14,635

15,176

15,319

15,363

15,242

15,026

AVG

3.6

3.9

4.4

5.1

6.9

9.8

12.6

16.5

20.6

24.8

29.2

P95

4.2

4.7

5.6

7.0

9.8

14.4

19.1

26.2

33.6

41.4

49.7

P99

7.4

7.8

8.3

9.6

12.6

18.4

25.1

37.1

48.3

70.2

103.8

30

0.9575

QPS

3,936

7,519

9,787

10,833

11,891

12,786

12,932

12,960

12,847

12,762

12,686

AVG

4.1

4.2

4.9

5.9

8.0

11.2

14.7

19.6

24.6

29.7

34.8

P95

4.7

5.1

6.3

8.4

11.8

17.1

23.4

32.4

41.5

50.7

59.6

P99

12.8

8.1

8.8

11.0

15.1

22.0

31.6

45.4

61.8

98.1

103.4

50

0.9588

QPS

3,606

6,827

8,966

9,586

10,654

11,208

11,332

11,222

11,182

11,096

11,027

AVG

4.4

4.7

5.3

6.7

9.0

12.8

16.8

22.6

28.3

34.1

40.0

P95

5.1

5.7

7.3

9.8

13.4

20.2

27.9

38.9

48.7

58.1

67.3

P99

24.1

8.6

9.5

12.3

17.0

26.4

38.2

57.3

81.7

94.3

102.2

70

0.9584

QPS

3,228

6,583

8,223

8,813

9,701

10,048

10,107

9,998

9,902

9,826

9,731

AVG

5.0

4.9

5.8

7.2

9.9

14.3

18.9

25.4

32.0

38.6

45.6

P95

5.8

5.9

8.1

10.8

15.7

24.2

33.1

44.6

53.8

63.4

74.0

P99

30.8

8.9

10.4

13.4

20.5

33.3

47.2

69.4

80.9

94.3

101.4

100

0.9593

QPS

2,790

5,873

7,399

7,826

8,394

8,714

8,612

8,429

8,334

8,274

8,151

AVG

5.7

5.4

6.5

8.2

11.4

16.5

22.2

30.0

37.9

45.9

54.5

P95

6.0

6.7

9.3

12.4

18.4

30.3

39.2

49.9

60.6

71.3

85.2

P99

47.8

9.5

11.7

15.4

23.8

42.8

59.1

72.7

84.9

95.1

102.9

300

10

0.9542

QPS

4,117

7,614

9,708

10,990

12,126

12,604

12,992

13,001

12,886

12,791

12,741

AVG

3.9

4.2

4.9

5.8

7.9

11.4

14.7

19.5

24.5

29.4

34.6

P95

4.6

5.1

6.6

8.1

11.5

17.2

23.3

32.3

41.4

50.3

58.9

P99

7.6

7.9

9.0

10.5

14.4

21.8

30.8

44.5

61.6

77.4

103.6

30

0.9660

QPS

3,912

7,191

9,107

9,867

10,831

11,301

11,534

11,512

11,348

11,365

11,228

AVG

4.1

4.4

5.3

6.5

8.8

12.7

16.5

22.1

27.9

33.3

39.4

P95

5.0

5.5

7.1

9.5

13.1

20.0

27.2

37.5

48.0

57.0

66.3

P99

8.3

8.3

9.3

11.9

16.4

26.4

36.8

52.2

74.3

94.9

109.5

50

0.9684

QPS

3,622

6,624

8,201

8,898

9,577

9,938

10,134

9,910

9,882

9,743

9,637

AVG

4.4

4.8

5.8

7.2

10.0

14.4

18.8

25.6

32.0

38.8

45.8

P95

5.3

6.0

8.3

10.7

15.7

23.9

32.9

44.7

53.5

63.0

73.2

P99

11.3

8.7

10.6

13.4

20.0

31.9

45.1

67.0

85.1

97.2

107.1

70

0.9682

QPS

3,333

6,075

7,521

8,042

8,730

9,068

8,984

8,892

8,746

8,688

8,550

AVG

4.8

5.3

6.4

7.9

11.0

15.8

21.3

28.5

36.2

43.7

51.8

P95

5.7

6.4

9.1

12.0

18.1

28.5

38.0

47.7

58.1

68.0

79.5

P99

14.0

9.1

11.4

15.0

23.8

39.3

54.3

71.1

87.0

102.5

104.9

100

0.9683

QPS

2,939

5,490

6,809

7,189

7,752

7,712

7,666

7,600

7,464

7,383

7,310

AVG

5.4

5.8

7.0

8.9

12.3

18.6

24.9

33.4

42.4

51.4

60.6

P95

6.2

7.3

10.3

13.7

21.9

34.0

42.4

53.0

66.1

75.3

91.3

P99

26.7

9.9

12.8

17.2

30.2

48.7

61.0

76.1

87.2

99.1

107.9

400

10

0.9583

QPS

3,747

7,079

9,137

9,743

10,734

11,236

11,391

11,359

11,246

11,237

11,112

AVG

4.3

4.5

5.2

6.5

8.9

12.8

16.7

22.3

28.1

33.7

39.6

P95

5.1

5.6

7.2

9.6

13.4

20.3

27.7

38.2

48.2

57.1

66.2

P99

8.0

8.4

9.4

12.0

16.8

26.7

37.0

52.8

69.6

84.8

114.4

30

0.9704

QPS

3,700

6,933

8,182

8,997

9,682

10,126

10,280

10,142

10,051

9,909

9,839

AVG

4.3

4.6

5.9

7.1

9.9

14.2

18.6

25.0

31.4

38.0

44.9

P95

5.3

5.8

8.4

10.6

15.0

23.7

32.3

43.4

52.9

62.7

72.1

P99

8.3

8.5

10.5

13.2

18.9

31.5

44.4

62.2

74.6

101.1

112.4

50

0.9733

QPS

3,411

6,244

7,646

8,067

8,650

9,053

9,047

8,867

8,788

8,691

8,549

AVG

4.7

5.1

6.3

7.9

11.1

15.8

21.1

28.5

35.9

43.6

51.7

P95

5.7

6.5

9.0

12.1

17.7

27.7

37.3

47.4

57.2

67.2

78.9

P99

9.1

9.0

11.2

15.1

22.8

37.6

52.9

68.8

89.1

103.7

105.6

70

0.9736

QPS

3,184

5,757

6,914

7,589

8,039

8,237

8,014

8,021

7,877

7,815

7,717

AVG

5.0

5.6

6.9

8.4

11.9

17.4

23.8

31.7

40.3

48.5

57.5

P95

6.1

7.0

10.2

13.0

20.3

32.5

41.0

50.6

63.2

72.3

88.7

P99

10.5

9.5

12.6

16.2

27.1

45.2

58.3

72.9

90.2

98.5

115.6

100

0.9735

QPS

2,843

5,214

6,245

6,784

7,185

7,109

7,044

6,888

6,825

6,730

6,667

AVG

5.6

6.1

7.7

9.4

13.3

20.2

27.1

36.8

46.4

56.5

66.7

P95

6.8

7.7

11.6

15.0

24.6

35.9

44.3

56.4

69.9

85.5

95.5

P99

20.3

10.2

14.2

19.1

34.0

50.0

63.4

77.7

87.9

100.5

111.3

500

10

0.9630

QPS

3,681

6,970

8,166

8,843

9,607

9,962

10,135

10,076

9,890

9,837

9,785

AVG

4.3

4.6

5.9

7.2

10.0

14.4

18.8

25.2

31.9

38.4

45.4

P95

5.4

5.9

8.4

10.9

15.7

23.7

33.3

43.7

53.1

61.5

71.7

P99

8.1

8.6

10.7

13.5

20.2

31.1

45.4

61.8

76.4

94.1

112.6

30

0.9754

QPS

3,381

6,362

7,722

8,343

8,836

9,093

9,081

9,019

8,893

8,774

8,700

AVG

4.7

5.0

6.2

7.7

10.8

15.8

21.0

28.2

35.6

43.2

50.9

P95

5.8

6.5

8.9

11.7

17.4

28.1

37.5

46.8

56.8

67.1

77.5

P99

8.5

9.0

11.1

14.6

22.4

38.5

51.4

66.0

82.3

100.7

113.8

50

0.9776

QPS

3,190

6,022

7,010

7,615

8,188

8,422

8,267

8,135

7,992

7,959

7,861

AVG

5.0

5.3

6.8

8.4

11.7

17.0

23.1

31.2

39.7

47.7

56.5

P95

6.2

6.8

10.1

12.9

20.6

31.0

39.9

50.0

62.0

71.4

87.8

P99

8.9

9.2

12.5

16.0

27.3

42.6

55.1

70.5

96.4

107.1

114.8

70

0.9786

QPS

2,903

5,781

6,506

6,985

7,502

7,520

7,355

7,263

7,181

7,094

7,027

AVG

5.5

5.5

7.4

9.1

12.8

19.1

26.0

34.9

44.1

53.4

63.3

P95

6.8

7.1

11.0

14.2

23.9

34.6

43.6

53.2

66.9

78.4

93.2

P99

10.1

9.6

13.5

17.9

33.0

47.8

61.1

73.9

93.8

104.2

112.3

100

0.9785

QPS

2,761

5,163

5,931

6,381

6,709

6,613

6,418

6,374

6,264

6,215

6,194

AVG

5.8

6.2

8.1

10.0

14.3

21.7

29.7

39.9

50.6

61.2

71.9

P95

7.2

8.0

12.3

16.2

27.0

37.3

47.8

61.7

72.7

90.3

99.3

P99

12.6

10.5

15.3

20.8

37.5

51.5

66.1

80.5

97.7

105.5

120.3

top-k = 100

The only difference from the top-k=10 group is that k changed from 10 to 100, and oversample_factor scaled down proportionally to 1/10, keeping the reranking candidate countsidentical across tiers; all other parameters, index, dataset, tool, concurrency levels, and timing methodology remain identical. Recall is measured as Recall@100, using the top-100 ground truth set as baseline.

Highest measured throughput for top-100 15,588 QPS(recall 0.7814), highest recall 0.9650 (ef_search=500 / oversample_factor=10, throughput 6,440 QPS).

ef_search

oversample_factor

Recall

Metric

c16

c32

c48

c64

c96

c144

c192

c256

c320

c384

c450

100

1

0.7814

QPS

4,418

7,958

10,847

12,770

14,334

15,177

15,403

15,515

15,408

15,588

15,321

AVG

3.6

4.0

4.4

5.0

6.7

9.4

12.4

16.3

20.4

24.2

28.9

P95

4.1

4.7

5.3

6.4

9.4

16.5

25.4

37.0

51.3

61.4

82.2

P99

7.3

7.9

8.3

9.1

12.2

24.2

40.1

63.5

90.7

102.1

127.7

3

0.8919

QPS

4,107

7,479

9,934

11,272

12,646

13,390

13,944

14,019

14,094

14,088

14,087

AVG

3.9

4.3

4.8

5.7

7.6

10.7

13.7

18.1

22.4

26.9

31.4

P95

4.4

5.0

5.9

7.6

10.6

15.8

20.6

28.9

37.5

48.3

59.5

P99

7.7

8.1

8.7

10.1

13.6

20.4

27.2

46.0

72.5

89.0

96.2

5

0.9042

QPS

3,807

7,019

9,215

10,172

11,311

11,894

12,095

12,163

11,930

12,000

11,845

AVG

4.2

4.6

5.2

6.3

8.5

12.0

15.8

20.9

26.5

31.5

37.2

P95

4.7

5.4

6.6

8.7

12.0

17.9

24.5

34.0

44.0

53.4

63.5

P99

7.9

8.4

9.3

11.4

15.3

23.2

34.3

55.1

79.5

84.7

92.5

7

0.9068

QPS

3,571

6,486

8,432

9,371

10,127

10,663

10,845

10,803

10,702

10,531

10,494

AVG

4.5

4.9

5.7

6.8

9.4

13.4

17.6

23.5

29.5

36.0

42.1

P95

5.0

5.8

7.6

9.6

13.8

20.4

28.1

39.4

50.5

61.5

72.0

P99

8.2

8.9

10.0

12.3

17.4

26.9

39.3

63.8

78.6

87.6

95.7

10

0.9085

QPS

3,279

6,011

7,591

8,213

8,911

9,235

9,276

9,156

9,081

8,992

8,933

AVG

4.9

5.3

6.3

7.8

10.7

15.5

20.6

27.8

35.0

42.1

49.7

P95

5.5

6.4

8.7

11.4

16.0

24.9

34.8

48.1

59.8

70.1

80.2

P99

8.6

9.4

11.1

14.2

20.2

33.3

53.0

70.6

79.7

90.5

97.7

200

1

0.7925

QPS

4,116

7,579

9,868

11,254

12,704

13,620

13,964

14,245

14,262

14,260

14,184

AVG

3.9

4.2

4.9

5.7

7.5

10.5

13.7

17.8

22.2

26.5

31.0

P95

4.5

4.9

6.1

7.7

10.7

15.5

20.5

28.0

37.4

47.0

60.3

P99

7.6

8.0

8.8

10.1

13.4

19.9

27.2

40.2

73.6

91.0

101.8

3

0.9145

QPS

3,792

6,865

9,138

10,068

11,296

11,869

12,056

12,170

11,987

11,807

11,879

AVG

4.2

4.7

5.2

6.3

8.5

12.1

15.8

20.9

26.4

32.0

37.2

P95

4.9

5.5

6.9

8.9

12.1

17.8

24.4

34.2

44.0

54.2

62.6

P99

7.9

8.5

9.3

11.3

15.2

22.6

33.0

50.9

75.0

92.1

95.4

5

0.9334

QPS

3,564

6,493

8,352

9,146

10,008

10,459

10,646

10,511

10,433

10,365

10,280

AVG

4.5

4.9

5.7

7.0

9.6

13.7

17.9

24.2

30.3

36.6

43.0

P95

5.1

5.9

7.7

9.9

14.0

20.8

28.8

40.3

51.7

62.5

72.9

P99

8.2

9.0

9.9

12.6

17.5

27.0

39.4

60.8

83.0

89.9

98.7

7

0.9387

QPS

3,317

6,074

7,667

8,419

9,110

9,460

9,577

9,450

9,361

9,239

9,187

AVG

4.8

5.3

6.2

7.6

10.5

15.2

19.9

26.9

33.8

41.0

48.2

P95

5.5

6.3

8.7

11.0

15.6

24.1

33.3

46.0

58.0

68.5

79.6

P99

8.6

9.3

11.0

13.7

19.5

31.9

48.0

70.2

81.7

91.6

101.0

10

0.9432

QPS

3,013

5,546

6,912

7,391

8,011

8,271

8,206

8,092

8,020

7,922

7,812

AVG

5.3

5.8

6.9

8.6

12.0

17.3

23.3

31.4

39.5

47.9

56.7

P95

6.1

7.0

9.9

12.9

18.7

29.9

40.9

53.0

63.7

74.9

87.5

P99

9.0

9.7

12.3

15.8

24.0

41.8

59.4

72.3

85.7

93.5

103.4

300

1

0.7965

QPS

3,833

6,903

9,132

10,359

11,226

11,895

12,199

12,292

12,122

12,088

11,960

AVG

4.2

4.6

5.2

6.2

8.5

12.0

15.6

20.7

26.1

31.3

36.9

P95

4.9

5.6

6.7

8.5

12.3

17.8

24.1

33.6

43.0

52.9

62.1

P99

7.8

8.4

9.2

10.9

15.5

22.4

32.4

47.7

67.1

92.5

102.0

3

0.9211

QPS

3,545

6,486

8,520

9,388

10,232

10,629

10,844

10,736

10,634

10,604

10,428

AVG

4.5

4.9

5.6

6.8

9.3

13.5

17.6

23.7

29.7

35.5

42.1

P95

5.3

5.9

7.5

9.6

13.6

20.5

28.0

39.3

50.1

60.5

71.1

P99

8.2

8.8

9.9

12.0

16.8

26.3

37.6

58.3

80.8

92.2

101.4

5

0.9437

QPS

3,326

5,980

7,690

8,430

9,012

9,403

9,459

9,390

9,230

9,245

9,149

AVG

4.8

5.3

6.2

7.6

10.6

15.3

20.2

27.0

34.1

40.9

48.5

P95

5.7

6.5

8.7

11.0

15.8

24.1

33.5

46.4

58.4

68.3

78.8

P99

8.6

9.3

11.1

13.7

19.8

32.0

45.8

70.2

85.7

93.0

101.7

7

0.9503

QPS

3,118

5,716

7,114

7,710

8,271

8,587

8,596

8,438

8,326

8,281

8,174

AVG

5.1

5.6

6.7

8.3

11.6

16.7

22.2

30.0

37.9

45.8

54.2

P95

6.0

6.8

9.5

12.3

17.7

27.9

38.5

51.1

62.4

72.9

84.7

P99

8.8

9.6

11.9

15.0

22.5

37.3

55.6

74.0

89.3

94.2

105.4

10

0.9552

QPS

2,872

5,201

6,395

6,865

7,374

7,526

7,377

7,259

7,194

7,109

7,029

AVG

5.6

6.1

7.5

9.3

13.0

19.1

25.9

34.9

44.1

53.5

63.3

P95

6.5

7.5

10.9

14.1

21.7

34.2

44.3

55.7

67.9

80.9

93.8

P99

9.4

10.2

13.4

17.3

28.6

47.9

64.4

74.0

86.7

99.4

109.8

400

1

0.7974

QPS

3,585

6,575

8,476

9,387

10,073

10,607

10,801

10,726

10,647

10,529

10,450

AVG

4.5

4.9

5.7

6.8

9.5

13.5

17.7

23.7

29.8

36.0

42.3

P95

5.3

5.9

7.5

9.7

14.1

20.4

28.2

39.2

50.5

60.8

71.0

P99

8.1

8.7

9.7

12.2

17.6

26.1

37.9

54.0

79.7

96.3

104.4

3

0.9235

QPS

3,306

5,976

7,810

8,524

9,148

9,591

9,668

9,574

9,490

9,397

9,305

AVG

4.8

5.3

6.1

7.5

10.5

15.0

19.8

26.5

33.3

40.2

47.5

P95

5.7

6.6

8.5

10.8

15.6

23.4

32.5

45.0

56.8

67.1

77.7

P99

8.5

9.2

10.8

13.4

19.5

30.1

43.9

66.8

87.5

97.1

108.3

5

0.9478

QPS

3,225

5,846

7,119

7,646

8,307

8,583

8,602

8,421

8,424

8,257

8,173

AVG

5.0

5.5

6.7

8.4

11.5

16.7

22.2

30.1

37.5

45.8

54.1

P95

6.0

6.8

9.6

12.5

17.6

27.9

38.4

50.9

60.9

72.9

85.2

P99

8.6

9.4

11.9

15.4

22.1

37.2

53.5

75.4

86.0

96.1

108.8

7

0.9552

QPS

2,979

5,398

6,592

7,128

7,629

7,799

7,771

7,682

7,522

7,456

7,365

AVG

5.4

5.9

7.3

9.0

12.6

18.4

24.6

33.1

42.1

50.8

60.1

P95

6.4

7.3

10.5

13.6

20.1

32.6

42.6

54.0

66.6

77.1

91.8

P99

9.0

9.9

13.1

16.7

26.0

44.9

62.2

76.9

87.5

100.2

109.0

10

0.9603

QPS

2,730

4,954

5,927

6,358

6,871

6,818

6,700

6,657

6,563

6,471

6,423

AVG

5.9

6.5

8.1

10.0

13.9

21.1

28.5

38.2

48.3

58.6

69.3

P95

7.0

8.0

12.0

15.4

24.8

37.5

46.6

59.3

71.8

87.5

98.7

P99

9.5

10.6

14.7

19.2

33.8

52.6

65.0

77.1

92.0

106.1

118.0

500

1

0.7995

QPS

3,409

6,060

7,806

8,457

9,108

9,506

9,645

9,513

9,407

9,324

9,306

AVG

4.7

5.3

6.1

7.6

10.5

15.1

19.8

26.7

33.6

40.6

47.5

P95

5.8

6.6

8.5

11.1

15.8

24.0

33.2

45.6

56.9

67.3

77.2

P99

8.4

9.3

10.7

13.8

19.7

31.3

44.9

63.0

83.2

102.0

107.6

3

0.9269

QPS

3,086

6,057

7,167

7,897

8,333

8,694

8,679

8,622

8,461

8,401

8,326

AVG

5.2

5.3

6.7

8.1

11.5

16.5

22.0

29.4

37.4

45.1

53.2

P95

6.3

6.7

9.5

12.0

17.7

27.3

37.8

50.3

61.1

72.1

83.5

P99

8.8

9.3

11.7

14.7

22.2

36.0

51.9

73.1

91.7

101.8

109.2

5

0.9505

QPS

2,967

5,611

6,758

7,225

7,786

8,019

7,908

7,813

7,715

7,635

7,531

AVG

5.4

5.7

7.1

8.8

12.3

17.9

24.2

32.5

41.1

49.7

58.9

P95

6.5

7.2

10.2

13.3

19.3

31.4

42.1

53.3

64.7

76.1

89.7

P99

9.0

9.7

12.6

16.3

24.6

42.5

59.7

76.6

89.0

97.7

111.4

7

0.9592

QPS

2,878

5,277

6,262

6,641

7,136

7,134

7,069

6,999

6,888

6,784

6,730

AVG

5.6

6.1

7.7

9.6

13.4

20.1

27.0

36.3

46.1

55.9

66.1

P95

6.8

7.6

11.0

14.8

23.2

36.0

44.9

56.8

69.8

84.2

96.3

P99

9.2

10.1

13.6

18.3

30.9

50.3

63.6

77.9

95.3

102.0

113.0

10

0.9650

QPS

2,590

4,836

5,671

6,089

6,440

6,290

6,220

6,109

6,044

5,969

5,954

AVG

6.2

6.6

8.4

10.5

14.9

22.8

30.6

41.6

52.5

63.8

74.8

P95

7.5

8.4

12.4

16.5

27.2

39.3

48.9

64.2

76.1

92.4

101.9

P99

9.8

10.8

15.2

20.8

37.3

54.1

67.7

81.1

94.1

108.1

121.5

Data analysis overview

Data analysisData analysis charts

  1. Both parameters must be tuned; adjusting only one will cap recall at a lower level. Taking top-100 as an example: with ef_search=100 only increasing oversample_factor, recall caps at 0.9085; with fixed oversample_factor=1 only increasing ef_search, capped at 0.7995; while setting both to their maximum values reaches 0.9650. The same pattern holds for top-10: individually tuning each parameter caps recall at 0.9367 and 0.9630 respectively, while joint tuning reaches 0.9786.

  2. In the low-recall region, increasing oversample_factor offers better cost-effectiveness, while in the high-recall region, relying on ef_search. is essential. Taking top-100 as an example, at ef_search=100 , increasing oversample_factor from 1 to 5 raises recall from 0.7814 to 0.9042 (+12.3 percentage points) with only a 22.0% drop in peak throughput, offering excellent cost-effectiveness; but further increasing to 10 adds only 0.4 more percentage points. At this point, switching to increase ef_searchfrom 100 to 500 gains another 5.65 percentage points (0.9085 → 0.9650).

  3. The two parameters reinforce each other: as candidates increase, the effect of raisingef_search becomes more pronounced. Increasing ef_search from 100 to 500, top-100 with oversample_factor=1 gains only 1.8 percentage points, whereas with oversample_factor=10 it gains 5.65 percentage points (approximately 3.1x). This is because once the candidate pool is sufficient, improving candidate quality can fully translate into better final results.

  4. For top-10, oversample_factor has a clear saturation point, while top-100 does not. At ef_search=100 , top-10 recall essentially stops improving starting from oversample_factor=30 onward (0.9364 → 0.9361, only −0.03 percentage points from 30 to 100), while peak throughput drops from 15,257 to 9,856 (−35.4%), representing pure negative return; top-100 continues to improve up to oversample_factor=10 (+0.43 percentage points). The reason is that top-10 already has a 10:1 filtering margin at the minimum candidate count (100), while top-100 operates at 1:1 with no margin at all.Oversample factor saturation analysis

  5. With the same candidate count, top-100 recall is lower than top-10, but throughput remains comparable. With reranking candidate counts aligned tier by tier, the recall gap narrows from 14.4–16.4 percentage points (candidate count 100) to 1.3–2.8 percentage points (candidate count 1000), while the peak throughput difference at the same tier mostly stays within 10%. This indicates that server-side overhead is primarily determined by the two query parameters,k affecting only the response body size.

  6. The larger the parameters, the earlier peak concurrency occurs. Lightweight configurations (ef_search=100) mostly peak at c192–c384, while the heaviest configuration (ef_search=500 with large oversample_factor) peaks as early as c96–c144. The heavier each individual query, the less concurrency is needed to saturate the CPU. Therefore, under high-precision configurations, excessively high client-side concurrency is neither necessary nor advisable.Peak concurrency analysis

  7. Latency grows approximately linearly with concurrency, and P99 rises in a step-like manner after peak throughput. For example, with ef_search=100 / oversample_factor=10 / k=10 For example, with ef_search=100 / oversample_factor=10 / k=10, average latency increases from 3.2 ms at c16 to 24.3 ms at c450 (approximately 7.6x) while concurrency grows 28x, indicating stable queuing behavior; P99 at the same tier jumps from 23.7 ms at c192 to 53.4 ms at c256. If the workload has P99 constraints, keep concurrency at or below the peak throughput point.

Note

Recall is measured during the serial phase and depends only on index parameters and the two query parameters, independent of concurrency, so each group reports a single value. Throughput decline from peak to c450 is within 8% for all combinations (most within 5%), representing normal drag within the saturation plateau; analysis should use peak values per tier. A few combinations show notably elevated c16 readings (e.g., ef_search=100 / oversample_factor=100 P99), attributable to page cache and JIT instability during warm-up—a cold-start characteristic that does not represent steady-state performance.

Resource saturation

Measured readings during the formal test phase are as follows. "Single node" takes the maximum value across the two search nodes.

Metric

Measured value

Single-node CPU

Peak 97%

Dual-node CPU balance difference

Within 3% at peak tier

Search thread pool rejections

0 throughout

Search thread pool queue

No overflow

k-NN circuit breaker

Not triggered throughout (graph memory accounts for 29.7% of the circuit breaker limit)

Graph memory (off-heap)

21.6 GiB/node, resident throughout

Disk read rate

< 1 MB/s in steady state (page cache nearly fully hit)

Client CPU

Not a constraint

Neither thread pools nor circuit breakers were triggered, dual-node CPU was balanced (within 3% difference), disk reads were near zero—the current throughput bottleneck is CPU compute capacity, not memory, queue capacity, or IO.

Faiss vs. Lucene comparison

The parameter matrices in the two chapters differ (Lucene has only one effective knob), so direct tier-by-tier alignment is not possible. The comparison below uses the common comparable range across both chapters: same dataset, same cluster specifications, same M=32 / ef_construction=200, same single segment per shard.

Reachable recall ceiling

Engine

Highest Recall@10

Corresponding throughput

Highest Recall@100

Corresponding throughput

Lucene

0.9729

3,974 QPS

0.9598

4,101 QPS

Faiss

0.9786

7,520 QPS

0.9650

6,440 QPS

At the shared ceiling ef_search=500, Faiss achieves higher measured recall than Lucene and also delivers higher throughput at that recall (89.2% and 57.0% higher respectively). The difference stems from the tunable dimensions: at the same ef_search, Faiss can additionally leverage oversample_factor to append reranking candidates, while Lucene takes the larger of its two parameters—effectively a single knob. Note that recall for both engines has not saturated as ef_search increases—the last tier (400 → 500) yields Lucene +0.34/+0.81 percentage points and Faiss +0.50/+0.47 percentage points. Continuing to raise ef_search leaves headroom for both. Values in the tables are the highest measured within this test's parameter matrix and do not represent the engines' recall ceilings.

Throughput at equal recall

A more meaningful comparison for practical engine selection is "how much throughput each engine can achieve at comparable recall levels":

top-k

Recall range

Faiss config / recall / peak QPS

Lucene config / recall / peak QPS

Faiss advantage

10

≈ 0.93

ef=100, osf=10 → 0.9258 → 18,251

ef=100 → 0.9283 → 13,150

+38.8%

10

≈ 0.954

ef=300, osf=10 → 0.9542 → 13,001

ef=200 → 0.9530 → 8,154

+59.4%

10

0.96

6,747

12,541

+86%

10

≈ 0.970

ef=400, osf=30 → 0.9704 → 10,280

ef=400 → 0.9695 → 4,717

+117.9%

100

≈ 0.91

ef=100, osf=10 → 0.9085 → 9,276

ef=200 → 0.9109 → 8,069

+15.0%

100

≈ 0.94

ef=200, osf=7 → 0.9387 → 9,577

ef=300 → 0.9389 → 6,089

+57.3%

100

≈ 0.96

ef=400, osf=10 → 0.9603 → 6,871

ef=500 → 0.9598 → 4,101

+67.5%

100

0.9598

4,101

6,991

+70%

Faiss vs Lucene throughput comparison

At the same recall target, Faiss throughput leads Lucene by 38%–144%, with the advantage growing as recall requirements increase. Each point in the chart represents a parameter configuration's recall–peak throughput pair:

  • At the same recall level, Faiss data points are consistently above the Lucene curve

  • The Lucene curve terminates earlier on the right side because at the shared peak ef_search=500 it only reaches 0.9729/0.9598, not because its recall capability has peaked.

The gap stems from Faiss being able to tune two parameters simultaneously, while Lucene can only raise ef_search, making the single-knob approach more costly.

Storage and memory

Engine

Primary shard storage

Hot data (per node)

Hot data residency location

Consumes circuit breaker quota

Lucene

460.3 GiB

13.5 GiB

Page cache

No

Faiss

478.7 GiB

21.6 GiB

Off-heap memory

Yes (measured at 29.7% of the circuit breaker limit)

Primary shard storage differs by only 4% between the two engines, but the volume and residency location of hot data differ. Faiss requires 21.6 GiB/node of resident hot data, about 60% more than Lucene's 13.5 GiB. Both have the same neighbor count and identical 1-bit quantized vector sizes; the difference comes from graph structure encoding—Faiss stores neighbor IDs as fixed-length integers while Lucene uses variable-length compression, causing the same graph connections to consume more space in Faiss. Both store a copy of full-precision vectors for reranking (216.3 GiB/node each). For capacity planning, Lucene only needs to ensure page cache can hold the hot data, while Faiss additionally requires confirming that graph memory stays within the k-NN circuit breaker limit.

Comparison with FP32 full precision

Compared withPerformance test report: 100M-scale vectors (1024-D / FP32)to assess the value of quantization. Both reports share the same dataset, query set, graph-building parameters (M=32 / ef_construction=200), engine version, and per-node CPU specifications. The difference lies in cluster scale: the 1-bit side uses 2 nodes × 128 GiB (256 GiB total memory, 2 primary shards), while the FP32 side uses 3 nodes × 256 GiB (768 GiB total memory, 3 primary shards). The 1-bit side has less than two-thirds the memory and shard count of the FP32 side, so the throughput comparison below is conservative for 1-bit and should be read accordingly.

Comparable recall ceilings

Engine

Precision

Highest Recall@10

Highest Recall@100

Faiss

FP32

0.9733

0.9728

Faiss

1-bit

0.9786

0.9650

Lucene

FP32

0.9715

0.9732

Lucene

1-bit

0.9729

0.9598

  • At top-100, the ef_search peak settings on both sides are both 500, comparable: 1-bit is 0.8–1.3 percentage points lower, with quantization error more exposed at tail positions.

  • At top-10, the peak settings differ between sides (FP32 at 350, 1-bit at 500), not directly comparable. The higher value on the 1-bit side only reflects a deeper parameter sweep, and the FP32 side from ef_search=256 to 350 still shows a +0.46 percentage point increase with no saturation observed. What can be confirmed is that both precision levels can reach the 0.96–0.97 range and quantization does not constitute a recall bottleneck. However, this test is insufficient to determine which has a higher recall ceiling.

QPS at the same recall

At each recall target, the maximum QPS configurations achievable by each side are compared (the 1-bit cluster resources are only 1/3 of the FP32 side):

Engine

top-k

Recall range

FP32 (768 GiB)

1-bit (256 GiB)

1-bit / FP32

Faiss

10

≥ 0.97

7,113 (ef=350)

10,280 (ef=400, osf=30)

145%

Faiss

100

0.94

11,520

9,546

83%

Faiss

10

≈ 0.95

12,277 (ef=128)

13,001 (ef=300, osf=10)

106%

Faiss

10

0.96

10,552

12,541

119%

Faiss

100

≈ 0.96

6,549 (ef=400)

6,871 (ef=400, osf=10)

105%

Faiss

100

≈ 0.95

9,618 (ef=200)

8,596 (ef=300, osf=7)

89%

Lucene

10

≥ 0.97

4,468 (ef=350)

3,974 (ef=500)

89%

Lucene

100

0.94

8,413

5,981

71%

Lucene

10

≈ 0.95

8,785 (ef=128)

8,154 (ef=200)

93%

Lucene

10

0.96

6,910

6,747

98%

Lucene

100

≈ 0.95

6,477 (ef=200)

4,831 (ef=400)

75%

Lucene

100

0.9598

5,596

4,101

73%

Reading methodology: For each side, first take the points with the highest throughput at the same recall from all parameter combinations to form the frontier, then linearly interpolate between adjacent frontier points to the recall target in the table. Compare only after aligning both sides to the same recall value. All recall targets fall within the measured range of both sides, with no extrapolation.

  • In the top-10 scenario, 1-bit achieves no less than full precision with 1/3 of the memory resources. Faiss side outperforms by 14%–25%, with the advantage growing as recall requirements increase; Lucene side is roughly on par (96%–99%). The 1-bit side cluster memory is only one-third of the FP32 side; this ratio actually underestimates the cost-effectiveness of quantization.

  • The reason Faiss outperforms in the high-recall range is an additional knob. Quantization gives Faiss an additional independent dimension—the reranking candidate pool, which can be scaled to 100x k; full-precision side recall is entirely determined by ef_search, so in the high-recall range it can only deepen graph traversal, with higher marginal cost.

  • In the top-100 scenario, 1-bit trails full precision by 16%–29%. When returning 100 results, the available multiplier for the reranking candidate pool is compressed by the upper limit, and quantization error is more exposed at the tail of the result list, requiring deeper graph traversal to compensate. Lucene side trails more (27%–29%), because it only has ef_search one effective knob.

  • The ratios in this table should not be taken as the true production cost-effectiveness of quantization. The test uses 1,677 query samples, and reranking reads on the quantized side mostly hit page cache. Under production random query workloads, page faults reading full-precision vectors (216 GiB/node) from disk become the primary bottleneck, and actual throughput will be lower than reported here, while the FP32 side avoids this risk because all data is resident. Choosing quantization requires reserving additional page cache for full-precision vectors.

Storage and memory

Engine

Precision

Primary shard storage

Hot data requiring resident memory (single node)

Faiss

FP32

898.7 GiB

154.2 GiB

Faiss

1-bit

478.7 GiB (down 47%)

21.6 GiB (down 7×)

Lucene

FP32

444.2 GiB

137.9 GiB

Lucene

1-bit

460.3 GiB (slightly up)

13.5 GiB (down 10×)

Two observations:

  • Primary shard storage changes depend on engine architecture. Under Faiss FP32, two complete copies of vectors are stored (graph file + flat recomputation file); after quantization, the second copy becomes 1-bit, cutting storage roughly in half. Under Lucene FP32, graph and vectors coexist in a single segment file; after quantization, an additional 1-bit vector copy is added, slightly increasing storage.

  • Hot data drops by an order of magnitude. The hot data that both engines must keep resident (HNSW graph + quantized vectors) drops from over a hundred GiB to tens of GiB, shifting full-precision vectors from "fully resident" to "read on demand during reranking." This is the core value of quantization: the same hardware can support significantly larger datasets.

Selection and production guidance

Scenarios

1-bit quantization suits scenarios where memory is constrained and the dataset is too large to fully cache all vectors. Quantization compresses the memory-resident portion by an order of magnitude, at the cost of two-stage retrieval (coarse ranking + full-precision reranking), resulting in lower throughput and additional reranking read overhead compared to full precision at the same specifications. If the hardware can cache all vectors, quantization offers no advantage and full precision should be preferred.

Engine selection

Both engines have comparable storage footprints; engine selection primarily depends on throughput, recall ceiling, and operational complexity (conclusions below are based on the test specifications in this report; actual values vary with workload characteristics and hardware):

Dimension

Lucene SQ 1-bit

Faiss BQ 1-bit

Throughput at equal recall

Baseline

Leads (advantage grows with higher recall requirements)

Recall ceiling

Relatively low

Relatively high

Tuning dimensions

1 (ef_search), simple

2 (ef_search + oversample_factor), requires joint tuning

Memory constraints

Whether page cache can hold hot data

Additionally constrained by the k-NN circuit breaker (graph memory)

Operational considerations

No special requirements

Requires evaluating graph memory against the circuit breaker limit; recommend calling the warmup API before queries to preload the graph

Choose Faiss for higher throughput and recall ceilings; choose Lucene for simpler tuning without memory circuit breaker concerns (verify specific differences on target hardware through benchmarking). Under the test specifications used here, Faiss leads by 15%–118% in equal-recall comparisons, with the gap widening at higher recall requirements.

Parameter trade-offs

First determine the recall target, then select the configuration with the highest throughput among those meeting the target.Optimal values and inflection points depend on the dataset, dimensionality, and scale, and must be determined by parameter sweeping on actual business data; the following provides only a parameter selection methodology.

  • Lucene has only one knob: only sweep ef_search; oversample_factor ; no need to set (values below ef_search have no effect).

  • For Faiss, prioritize tuning oversample_factor: when the two parameters take effect independently, increasing the reranking candidate count offers better recall cost-effectiveness than widening the coarse-ranking beam—the former only adds reranking computation while solidly improving recall; the latter lengthens graph traversal paths, and coarse-ranking quality quickly plateaus when the candidate count is insufficient.

  • Watch for diminishing marginal returns and stop in time: as oversample_factor increases, recall saturates (top-10 saturates earlier, top-100 later); beyond the inflection point, throughput keeps declining without recall improvement, yielding pure negative returns. In this test, top-10 saturates after osf=30 (further increasing to 100 causes recall to stagnate or slightly decrease while throughput drops 35%); top-100 continues to improve slowly up to osf=10 ; the exact inflection point varies with business data, so parameter sweeping on production traffic is recommended.

  • Costs escalate sharply in the high-recall range: pushing from a usable recall level (e.g., ~0.93) to the recall ceiling often requires several times the search breadth and sacrifices most of the throughput. If the workload tolerates moderate recall, lowering the recall target can yield significant throughput gains.

  • The two top-k values require separate tuning: with the same reranking candidate count, top-100 recall is inherently lower than top-10, so top-10 tuning results cannot be directly reused.

    Build parameters M / ef_construction are set at index build time and determine graph quality and memory footprint. This test fixes M=32 / ef_construction=200:

    • M=32: at the 100M scale with high dimensionality (1024-D), graph connectivity must be high enough to sustain 0.97+ recall;M=16 is acceptable at the 10M scale, but at the 100M scale the larger graph suffers insufficient connectivity and a limited recall ceiling.

    • ef_construction=200: graph construction candidate queue width—200 strikes a good balance between graph quality and indexing speed, and further increases yield diminishing returns on recall.

  • Additional considerations for quantization: under 1-bit quantization the graph is built from quantized vectors, so its navigation quality is inherently lower than that of full precision;M a value that is too small further amplifies this disadvantage, so for 100M-scale quantized scenarios we do not recommend going below M=32.

is not recommended. For 100M-scale scenarios, directly adopt M=32 / ef_construction=200and focus tuning on query-time parameters (ef_search / oversample_factor); for empirical comparisons of build parameters on the recall–throughput trade-off, refer toPerformance test report: 10M-scale vectors (1024-D / FP32).

Production environment considerations

  • Memory planning has two layers: hot data (HNSW graph + quantized vectors) must be fully resident to ensure zero disk reads during coarse ranking; full-precision vectors are read on demand during reranking, requiring as much page cache as possible. This report is based on a fixed query set with strong locality, where reranking mostly hits page cache; before going live, validate page cache hit rates with real query distributions—otherwise reranking disk reads may become the primary bottleneck, and actual QPS degradation will be significantly worse than reported.

  • Faiss requires circuit breaker quota verification: Faiss graphs occupy off-heap memory and count toward the k-NN circuit breaker; re-evaluation is needed when data volume, replica count, or quantization parameters change—otherwise graph loading may be blocked by the circuit breaker.

  • Replicas and capacity planning: this test uses 0 replicas to isolate variables; production requires ≥1 replica for availability, and hot data and storage must scale by the replica multiplier.

  • Segment count and continuous writes: all data in this report was measured under single-segment-per-shard steady state, representing the upper bound of query performance; continuous writes produce multiple segments that increase query traversal overhead—schedule force merge during low-write periods or rely on automatic merging.

FAQ

  • Why does ef_search below 100 have no effect?

    1-bit quantization has a minimum reranking candidate count of 100, and the actual search breadth cannot go below this floor, so when ef_search ≤ 100 , the effect is always equivalent to 100. The parameter matrix in this test starts from ef_search=100 without including lower tiers.

  • Why was the 32-core 128 GiB × 2 configuration chosen?

    The value of 1-bit quantization is compressing the hot data that must reside in memory. Its hot data (graph + 1-bit vectors) is only 13.5 GiB/node (Lucene, in page cache) and 21.6 GiB/node (Faiss, in off-heap memory)—a 128 GiB node can fully accommodate this with room left for full-precision reranking. Using larger-memory nodes would allow caching all full-precision vectors as well, negating the "small hot data" advantage of quantization and invalidating the test conclusions.

  • Why does 1-bit quantization require mode: on_disk?

    mode: on_disk mode: on_disk enables two-stage retrieval. 1-bit quantization retains only the sign bit per dimension, causing extreme information loss; using it alone for ranking lacks sufficient precision. After coarse ranking produces candidates, full-precision raw vectors must be read for exact reranking to achieve usable recall. Without this mode, reranking is skipped and recall drops to unusable levels.

  • Can the test data be extrapolated to larger scales or random query workloads?

    Direct extrapolation is not recommended. Growing data scale simultaneously affects graph depth, traversal node count, and memory usage; furthermore, this test uses a fixed query set with strong locality and near-zero steady-state disk reads, whereas under random query workloads, the proportion of disk reads for full-precision reranking is higher and latency is greater. For larger-scale or random workload scenarios, validation testing with actual business data is recommended.