Quantization algorithms for vector search
optimizes based on and supports multiple s, such as , , , and . These algorithms reduce and improve performance while maintaining a high .
Supported versions
Supported PolarDB for PostgreSQL versions:
-
( 2.0.14.20.44.0 and later)
-
(minor kernel version 2.0.16.13.16.0 or later)
-
(minor kernel version 2.0.17.9.6.0 or later)
-
(minor kernel version 2.0.18.3.2.0 or later)
Check your minor kernel version number in the console or run the SHOW polardb_version; statement. If your cluster does not meet the version requirements, you must upgrade the minor kernel version.
How quantization works
Each dimension of a full-precision vector uses 4 bytes. In high-dimensional scenarios, the vector index can consume significant memory, increasing cache miss and degrading performance. Quantization compresses full-precision vectors into compact codes. Combined with automatic reranking, random rotation, and ADC, it reduces memory usage and improves performance while maintaining high recall rate.
Using quantization algorithms
supports four s: , , , and . Currently, quantization is supported only for indexes on the vector ( floating-point) .
To enable a quantization algorithm, specify it in the WITH (quantization = pq/sq4/sq8/rabitq) clause.
The following table compares the quantization algorithms.
|
Algorithm |
Precision per dimension |
Compression ratio |
Maximum supported dimensions |
|
SQ8 |
8 bits |
4x |
8,000 |
|
SQ4 |
4 bits |
8x |
16,000 |
|
PQ |
8 bits/sub-vector |
|
16,000 |
|
RabitQ |
1 bit |
32x |
16,000 |
SQ4 and SQ8
The Scalar Quantization (SQ) algorithm compresses each dimension of a vector to 4 or 8 bits, yielding compression ratios of 8x and 4x, respectively.
-
SQ8: Supports up to 8,000 dimensions.
-
SQ4: Supports up to 16,000 dimensions.
CREATE INDEX ON vecs USING hnsw(embedding vector_l2_ops) WITH (m=16, ef_construction=256, quantization=sq4, train_samples=5000);
CREATE INDEX ON vecs USING hnsw(embedding vector_l2_ops) WITH (m=16, ef_construction=256, quantization=sq8, train_samples=5000);PQ
The PQ quantization algorithm divides an original vector into pq_m sub-vectors and then learns a separate codebook for each group by performing K-means clustering. Each sub-vector is stored using 1 byte. For example, for a 1,024-dimension full-precision vector, setting pq_m = 128 results in an 8x compression ratio. The pq_m parameter controls the trade-off between compression ratio and accuracy: a smaller pq_m value yields a higher compression ratio but results in greater accuracy loss.
-
Supports up to 16,000 dimensions.
-
Limitations: The vector dimension must be a multiple of
pq_mand must be greater thanpq_m.
CREATE INDEX ON vecs USING hnsw(embedding vector_l2_ops) WITH (m=16, ef_construction=256, quantization=pq, pq_m=128, train_samples=5000);RabitQ
RabitQ uses the Fast Walsh-Hadamard Transform (FWHT) combined with random sign flipping to randomly rotate the vector. It then quantizes each dimension to 1 bit, achieving a 32x compression ratio.
-
Supports up to 16,000 dimensions.
-
Limitations: The vector dimension must be a multiple of 8 (for example, 128, 256, 512, or 1024) to meet the byte-alignment requirements for binary encoding.
CREATE INDEX ON vecs USING hnsw(embedding vector_l2_ops) WITH (m=16, ef_construction=256, quantization=rabitq, train_samples=5000);Reranking
While quantization algorithms reduce memory usage, they may lose accuracy compared to full-precision vectors. Automatic reranking improves the recall rate by refining search results. It first retrieves a candidate set from the quantized vector index and then accesses the full-precision vector data from the table to recalculate similarity and reorder results. This adds overhead, but query performance stays high because quantization optimizes initial retrieval.
Parameters
SET hnsw.enable_rerank = on;
SET hnsw.rerank_k_factor = 2.0;Parameters:
-
hnsw.enable_rerank: Enables or disables reranking. Default:off. -
hnsw.rerank_k_factor: Sets the initial candidate set size from the vector index search. A larger value improves the recall rate during reranking.
Performance
On the Cohere 1M dataset, HNSW + SQ8 substantially reduces vector index memory usage and improves both query and index creation performance, while maintaining a recall rate greater than 95%.
