Performance white paper

Run a test using the web UI

• On the Run Test page, select Lindorm as the database.

• Set the Lindorm instance connection details and the index name.

  The form also includes the version (optional, used to label the results), db_label (optional, used to label the results), and note (optional) fields. The default value for port is 30070.

  Parameter	Description
  host & port	The search engine's Elasticsearch-compatible connection address. To obtain this address, see Elasticsearch Compatible Address. Important Use a private network address.
  user & password	The username and password used to access the vector engine. To obtain the default username and password, go to the console. In the left-side navigation pane, choose Database Connections and click the Search Engine tab.
  index_name	The name of the vector index to create.

• Select test cases and configure the index parameters.

  • General parameters

    Parameter	Description
    filter_type	The query mode for hybrid search. Valid values: efficient_filter, pre_filter, and post_filter. efficient_filter is recommended.
    k_expand_scope	In post_filter mode, this value replaces the k value to expand the search scope.
    number_of_regions	The number of regions for the vector index. To test the performance of a single node, set this parameter to 1.

  • Index parameters

    HNSW index

    On the STEP 2: Choose the case(s) page, select the Search Performance Test (100M Dataset, 768 Dim) test case and select Lindorm as the database. Set IndexType to HNSW, filter_type to efficient_filter, k_expand_scope to 1000, and number_of_regions to 2.

    Parameter	Description
    M	The maximum number of outgoing edges per node at each graph layer. Recommended value: 32.
    efConstruction	The length of the dynamic list during index construction. Recommended value: 400.
    efSearch	The length of the dynamic list during an index query. This parameter is typically used to improve recall at the cost of performance overhead. Recommended value: 100.

    IVFPQ index

    In the Search Performance Test (100M Dataset, 768 Dim) configuration panel, set IndexType to IVF_PQ, filter_type to efficient_filter, k_expand_scope to 1000, and number_of_regions to 1.

    Parameter	Description
    nlist	The number of centroids. Recommended value: $$\sqrt{n}$$, where n is the total number of vectors.
    nprobe	The number of clusters to query. This parameter is typically used to improve recall at the cost of performance overhead. Recommended value: nlist * 0.004.
    reorder-factor	The factor used for re-ranking with raw vectors. The number of raw vectors fetched for re-ranking is topk * reorder-factor. This parameter is typically used to improve recall at the cost of performance overhead. Recommended value: 2.
    client_refactor	Specifies whether to perform re-ranking on the client. Recommended value: true.
    centroids_hnsw_M	If the HNSW algorithm is used for centroid search, this parameter specifies the maximum number of outgoing edges per node at each graph layer. Recommended value: 32.
    centroids_hnsw_efConstruction	If the HNSW algorithm is used for centroid search, this parameter specifies the length of the dynamic list during index construction. Recommended value: 500.
    centroids_hnsw_efSearch	If the HNSW algorithm is used for centroid search, this parameter specifies the length of the dynamic list during a query. Recommended value: 200.

    IVFBQ index

    Select the Search Performance Test (100M Dataset, 768 Dim) test case, set IndexType to IVF_BQ, filter_type to efficient_filter, k_expand_scope to 1000, and number_of_regions to 1. For other parameters and their recommended values, see the following table.

    Parameter	Description
    nlist	The number of centroids. Recommended value: $$\sqrt{n}$$, where n is the total number of vectors.
    nprobe	The number of clusters to query. This parameter is typically used to improve recall at the cost of performance overhead. Recommended value: nlist * 0.004.
    reorder-factor	The factor used for re-ranking with raw vectors. The number of raw vectors fetched for re-ranking is topk * reorder-factor. This parameter is typically used to improve recall at the cost of performance overhead. Recommended value: 2.
    client_refactor	Specifies whether to perform re-ranking on the client. Recommended value: true.
    centroids_hnsw_M	If the HNSW algorithm is used for centroid search, this parameter specifies the maximum number of outgoing edges per node at each graph layer. Recommended value: 32.
    centroids_hnsw_efConstruction	If the HNSW algorithm is used for centroid search, this parameter specifies the length of the dynamic list during index construction. Recommended value: 500.
    centroids_hnsw_efSearch	If the HNSW algorithm is used for centroid search, this parameter specifies the length of the dynamic list during a query. Recommended value: 200.
    exbits	Specifies the number of extra bytes to represent the quantized vector. A higher value improves recall but increases memory usage. Recommended value: 2.

Run a test using the CLI

• When running the performance test multiple times, you can add --skip-drop-old --skip-load to the end of the following commands to skip index creation and data import.

  #HNSW index
  vectordbbench lindormhnsw --case-type Performance768D10M --index-name <index_name> --k 10 --host <host> --port <port> --user <user> --password <password> --m <M> --ef-construction <efConstruction> --ef-search <efSearch>
  #IVFPQ index
  vectordbbench lindormivfpq --case-type Performance768D10M --index-name <index_name> --k 10 --host <host> --port <port> --user <user> --password <password> --lists <nlist> --probes <nprobe> --m <centroids_hnsw_M> --ef-construction <centroids_hnsw_efConstruction> --ef-search <centroids_hnsw_efSearch> --reorder-factor <reorder_factor>
  #IVFBQ index
  vectordbbench lindormivfbq --case-type Performance768D10M --index-name <index_name> --k 10 --host <host> --port <port> --user <user> --password <password> --lists <nlist> --probes <nprobe> --exbits <exbits> --m <centroids_hnsw_M> --ef-construction <centroids_hnsw_efConstruction> --ef-search <centroids_hnsw_efSearch> --reorder-factor <reorder_factor>

• Run the test.

  To skip index creation and data import, select the Index already exists checkbox.

  In the Task Label field, enter a label for the task. If needed, select the Dataset from China (Shanghai) checkbox to download the dataset from an Alibaba Cloud OSS bucket in the China (Shanghai) region. Set the K value (the number of nearest neighbors to search for, default is 10), num of concurrencies (the number of concurrent searches, such as 32,64,128,160), and concurrency duration (the duration for each concurrent search test, default is 30 seconds). After configuring the parameters, click Run Your Test.

Test metrics

HNSW index search performance