DocsICP FilingConsole
官方文档
首页

Grouped vector search

更新时间：
复制 MD 格式
产品详情
我的收藏

A standard vector search returns the globally top-k most similar vectors — but when multiple vectors come from the same source (e.g., multiple chunks of the same document, or multiple images of the same product), those top-k slots can be dominated by a single source. Grouped vector search fixes this by returning the best-matching results from distinct groups, so each group contributes at most group_topk results and at most group_count groups appear in total.

Use cases

  • Retrieval-augmented generation (RAG): Documents are split into segments, each vectorized and stored in DashVector. Without grouping, multiple chunks from the same document may fill the top results. Set group_by_field to the document ID field to ensure each group represents a distinct source document.

  • Product image retrieval: Multiple images represent a single product, each stored as a separate vector. Set group_by_field to the product ID field to return results from different products rather than multiple images of the same one.

Prerequisites

Before you begin, ensure that you have:

Run a grouped vector search

The following example uses a RAG scenario. Six document segments from three papers are stored in a collection. The search returns the two most similar papers (groups), with up to two segments per paper.

The collection uses fields_schema to define document_id (string) and chunk_id (integer). The group_by_field parameter must reference a field defined in fields_schema — schema-free fields are not supported for grouped search.

Step 1: Create a collection and insert documents

Replace YOUR_API_KEY and YOUR_CLUSTER_ENDPOINT with your actual values.

import dashvector
import numpy as np

client = dashvector.Client(
    api_key='YOUR_API_KEY',
    endpoint='YOUR_CLUSTER_ENDPOINT'
)

# Create a collection with schema-defined fields
ret = client.create(
    name='group_by_demo',
    dimension=4,
    fields_schema={'document_id': str, 'chunk_id': int}
)
assert ret

collection = client.get(name='group_by_demo')

# Insert six segments across three documents
ret = collection.insert([
    ('1', np.random.rand(4), {'document_id': 'paper-01', 'chunk_id': 1, 'content': 'xxxA'}),
    ('2', np.random.rand(4), {'document_id': 'paper-01', 'chunk_id': 2, 'content': 'xxxB'}),
    ('3', np.random.rand(4), {'document_id': 'paper-02', 'chunk_id': 1, 'content': 'xxxC'}),
    ('4', np.random.rand(4), {'document_id': 'paper-02', 'chunk_id': 2, 'content': 'xxxD'}),
    ('5', np.random.rand(4), {'document_id': 'paper-02', 'chunk_id': 3, 'content': 'xxxE'}),
    ('6', np.random.rand(4), {'document_id': 'paper-03', 'chunk_id': 1, 'content': 'xxxF'}),
])
assert ret

Step 2: Query with grouping

ret = collection.query_group_by(
    vector=[0.1, 0.2, 0.3, 0.4],
    group_by_field='document_id',  # Group results by document
    group_count=2,                 # Return at most 2 groups
    group_topk=2,                  # Return at most 2 segments per group
)

if ret:
    print('query_group_by success')
    print(len(ret))
    print('------------------------')
    for group in ret:
        print('group key:', group.group_id)
        for doc in group.docs:
            print(' -', doc)

The output groups results by document_id. Each group lists its segments in descending score order:

query_group_by success
4
------------------------
group key: paper-01
 - {"id": "2", "fields": {"document_id": "paper-01", "chunk_id": 2, "content": "xxxB"}, "score": 0.6807}
 - {"id": "1", "fields": {"document_id": "paper-01", "chunk_id": 1, "content": "xxxA"}, "score": 0.4289}
group key: paper-02
 - {"id": "3", "fields": {"document_id": "paper-02", "chunk_id": 1, "content": "xxxC"}, "score": 0.6553}
 - {"id": "5", "fields": {"document_id": "paper-02", "chunk_id": 3, "content": "xxxE"}, "score": 0.4401}

Two groups are returned (paper-01 and paper-02), each with up to two segments. paper-03 is excluded because group_count=2 limits the total number of groups returned.

Limitations

ConstraintDetails
group_by_field field typeMust reference a field defined in fields_schema at collection creation. Schema-free fields are not supported. See Create a collection and Schema-free.
group_count and group_topk are best-effortThe actual number of groups or segments returned may be fewer than the values you set. DashVector prioritizes group_count — it fills as many groups as possible first, then fills segments within each group up to group_topk.
Maximum valuesgroup_count max: 64. group_topk max: 16. Higher values increase index scan workload and extend API response time.

What's next

Previous：Dynamic vector quantizationNext：Multi-vector retrieval