General vector index configuration

Vector dimension

• Purpose: Defines the number of features in a vector. This value must exactly match the output dimension of your embedding model.

• Recommendations:

  • Ensure consistency: If the configured dimension does not match the dimension of the written vector data, index building will fail.

  • Performance impact: Higher dimensions allow vectors to represent more information, but they also increase memory usage and computational overhead. Doubling the dimension approximately doubles the memory usage.

Distance type

• Purpose: Defines the method for calculating similarity between vectors. Choosing an appropriate distance type is critical to retrieval quality and depends on your data's characteristics and business scenario.

• Selection guide:

  Distance type	Score description
  Cosine distance	Scores range from [-1, 1], where higher scores indicate greater similarity. A score of 1 means the vectors are identical, while -1 means they point in opposite directions.
  Inner product distance	Higher scores indicate greater similarity.
  Squared Euclidean distance	Lower scores indicate greater similarity. A score of 0 means the vectors are identical.

Vector index algorithm

• Purpose: Selects the underlying algorithm for building the vector index. Different algorithms offer various trade-offs between index building speed, memory usage, query performance, and recall.

• Selection guide:

  Algorithm	Description	Distance type	Data scale	Recall	Latency	Memory usage	Primary use case
  FLAT (formerly Linear)	Vector scale: Suitable for tens of thousands of vectors. Scenario: Scenarios that require perfect data accuracy (100% recall).	Inner Product, Squared Euclidean, Cosine	Very small (< 10k) Tens of thousands	100% (Exact)	Very high	Very low	Benchmarking; exact re-ranking on very small datasets.
  HNSW	Vector scale: Suitable for tens of millions of vectors. Scenario: A high-performance option for use cases with strict requirements for accuracy and low latency.	Inner Product, Squared Euclidean, Cosine	Medium (10M+) Tens of millions	Very high	Low	Very high	High-performance, in-memory online retrieval.
  HNSW_RaBitQ	Vector scale: Suitable for billion-scale datasets. Scenario: Ideal for massive datasets with strict memory constraints where some accuracy trade-off is acceptable.	Squared Euclidean	Large (100M+) Hundreds of millions	High	Very low	Very low	Lightweight retrieval optimized with binary quantization.
  CagraHNSW	Vector scale: Suitable for hundreds of millions of vectors. Scenario: A GPU performance powerhouse for graph indexing, often used with multiple GPUs for datasets in the hundreds of millions.	Inner Product, Squared Euclidean	Large (100M+) Hundreds of millions	Very high	Very low (GPU)	Very high	GPU-accelerated search for high-throughput scenarios.
  HNSW_SQ (formerly QGraph)	Vector scale: Suitable for billion-scale datasets. Scenario: For use cases requiring high query performance and low latency, where accuracy requirements are less strict.	Inner Product, Squared Euclidean, Cosine	Very large (1B+) Billion-scale	High	Low	High
  IVF_SQ8	Vector scale: Suitable for hundreds of millions of vectors. Scenario: A balanced, traditional approach when requirements for accuracy and latency are moderate.	Inner Product, Squared Euclidean, Cosine	Large (~500M) 500-million-scale	Medium to high	Medium	Low	A classic, cost-effective solution for large datasets that uses vector compression to reduce memory usage. Ideal for balancing cost and scale, especially with cold/hot data tiering.
  DiskANN	Vector scale: Designed for maximum scale, handling datasets of a billion vectors or more. Scenario: Uses local disks, making it suitable for scenarios where higher latency is acceptable and memory usage must be minimal.	Inner Product, Squared Euclidean, Cosine	Massive (1B+) Billion-plus	High	Medium to high	Very low	Disk-resident search for ultra-large-scale datasets.

Real-time indexing

• Purpose: When enabled, incrementally written data is indexed and becomes searchable within seconds.

• How it works: The system first builds a temporary in-memory index for real-time data writes. Once this temporary index reaches a certain size, the system merges its data into the full index on disk.

• Recommendations:

  • Enable (true): Recommended for online services that require data to be searchable immediately after writing. This option consumes a small amount of additional memory and CPU resources.

  • Disable (false): Suitable for offline analytics or for data that is not frequently updated after a full import.

Linear build threshold

• Purpose: When the number of vectors in a shard is below this threshold, the system performs a brute-force search using the FLAT algorithm, overriding your selected vector index algorithm.

• Recommendations:

  • Default: 5000. For datasets of this size, a brute-force search often performs as well as or better than a complex index due to lower index building overhead.

  • When to adjust: You typically do not need to change this value. However, if you have very high query concurrency and your data volume is near the threshold, you can lower this value to force the use of a high-performance index like HNSW. This may increase index building overhead.

Ignore invalid vector data

• Purpose: Controls how the system handles invalid vectors (e.g., dimension mismatches or null values) encountered during full or incremental index building.

• Recommendations:

  Option	Behavior	Recommended use case
  true	The system skips any row with an invalid vector, logs a warning, and continues the index building task.	Development and testing. Allows you to debug quickly by preventing a few bad records from causing the entire build task to fail.
  false	If the system encounters any invalid vector, the index building task fails immediately and returns an error.	Production environment. Enforces data quality and prevents silent data loss. We recommend using this setting with a robust upstream data cleaning pipeline.

Real-time indexing parameters

• Purpose: When real-time indexing is enabled, these parameters fine-tune real-time data stream processing.

• Parameter example: {"proxima.oswg.streamer.segment_size":2048}

• Parameter explanation: proxima.oswg.streamer.segment_size controls how many real-time records accumulate in memory before flushing them to a small in-memory segment.

• Tuning recommendations:

  • High write QPS scenarios: Increase this value (e.g., to 4096) to reduce the number of in-memory segments and lower index management overhead. This slightly increases the write-to-search latency.

  • Low write QPS scenarios: Keep the default value of 2048 or decrease it to make newly written data searchable more quickly.

Real-time retrieval parameters

• Purpose: Dynamically adjusts the search behavior of the index algorithm at query time to balance recall and latency. The keys and values for this parameter vary depending on the selected vector index algorithm.

• General description: These parameters typically control the scope of the search. For example, with the HNSW algorithm, the ef parameter controls the number of neighbor nodes to traverse during a search. A larger ef value increases recall but also increases query latency.

• Configuration example (HNSW):

  • {"searcher_name":"HNSW", "ef":200}

  • The value of ef typically ranges from k (the number of top results to return) to 4096; a good starting value is 100, which you can adjust based on your business requirements for recall and latency.

Vector separator

• Purpose: Defines the separator used between dimension values in string-formatted vector data.

• Example: For the vector data 1.05,0.15,0.14, the separator is a comma (,), which is the system default. You typically do not need to change this setting.

Write volume control

• Limit batch size: Avoid writing too many documents in a single batch. Split data into smaller batches based on your vector dimension and instance type to prevent sudden spikes in data processing memory usage.

• Control write frequency: For high-frequency write scenarios, consider reducing write concurrency or increasing the interval between batches. This provides sufficient time for index building and segment merging.

• Monitor data processing memory: If data processing memory usage remains high after writes even when sufficient overall instance memory is available, it indicates high memory overhead from index building or segment merging. In this case, reduce the write rate and wait for merge operations to complete before resuming.

Resource assessment and scaling

If you experience insufficient data processing memory or a noticeable increase in query latency, assess whether your current instance type meets your business requirements.

• Use the Specification Calculator and enter your business parameters, such as vector dimension, data volume, and QPS, to get a recommended instance type.

• If the calculator indicates that your current instance type is insufficient, upgrade the memory and compute resources of your data nodes by changing the instance configuration.