Scalar filtering

Filtering strategies

The optimizer selects one of three strategies based on filter selectivity:

• pre-filtering

• post-filtering

• inline-filtering

Pre-filtering

When a filter is highly selective, the execution plan first uses a scalar index (such as a BTree index) to build an initial candidate set, then calculates similarity only for vectors in that set.

Post-filtering

When a scalar filter has low selectivity, post-filtering is more efficient. The vector index first retrieves an initial candidate set by similarity, then the scalar filter is applied. A potential issue is that fewer candidates may remain than the top-K value specified in the query.

addresses this with iterative scan: if too few candidates remain after filtering, the system continues vector retrieval until enough candidates are found or a scan limit is reached. To enable iterative scan with an HNSW index:

SET hnsw.iterative_scan = 'relaxed_order';
SET hnsw.max_scan_tuples = 100000;

Parameters:

• iterative_scan: Controls result ordering by similarity.

  • relaxed_order: Better performance, approximate ordering.

  • strict_order: Strict similarity ordering with higher overhead.

  • off: Disables the feature.

• max_scan_tuples: Maximum tuples to scan during iterative scan. Prevents unlimited iterations.

Inline-filtering

For medium-selectivity filters where pre-filtering and post-filtering both perform poorly, supports inline-filtering, which combines scalar and vector indexes within a single query.

SELECT id, category_id, embedding, embedding <-> '[50,50,50]' AS distance
FROM items
WHERE category_id = 3                       -- B-Tree bitmap filter
ORDER BY embedding <-> '[50,50,50]'         -- HNSW ordered scan
LIMIT 10;

Limit
   ->  Bitmap Filtered Index Scan using idx_items_hnsw on items
         Filter: (category_id = 3)
         TIDs Checked: 40            -- The HNSW index returned 40 TIDs
         Bitmap Hits: 6              -- 6 TIDs matched the bitmap
         ->  Bitmap Index Scan on idx_items_category
               Index Cond: (category_id = 3)

Performance at different filter rates

maintains stable performance across different filter rates. The following table shows which execution plan is selected at each rate: