Group Retrieval Documentation

更新时间:
复制 MD 格式

Use the query_group_by() method to search a collection and group results by a field value. This is useful in Retrieval-Augmented Generation (RAG) pipelines where a single document is split into multiple chunks — without grouping, the top results may contain several chunks from the same document, leaving other documents underrepresented.

Prerequisites

Before you begin, make sure you have:

API definition

Collection.query_group_by(
        self,
        vector: Optional[Union[List[Union[int, float]], np.ndarray]] = None,
        *,
        group_by_field: str,
        group_count: int = 10,
        group_topk: int = 10,
        id: Optional[str] = None,
        filter: Optional[str] = None,
        include_vector: bool = False,
        partition: Optional[str] = None,
        output_fields: Optional[List[str]] = None,
        sparse_vector: Optional[Dict[int, float]] = None,
        async_req: bool = False,
    ) -> DashVectorResponse:

Examples

Replace YOUR_API_KEY with your API key and YOUR_CLUSTER_ENDPOINT with the endpoint of your cluster before running the examples.

All examples below use the same dataset. Run this setup code first to create a collection and insert sample data:

import dashvector
import numpy as np

client = dashvector.Client(
    api_key='YOUR_API_KEY',
    endpoint='YOUR_CLUSTER_ENDPOINT'
)
ret = client.create(
    name='group_by_demo',
    dimension=4,
    fields_schema={'document_id': str, 'chunk_id': int}
)
assert ret

collection = client.get(name='group_by_demo')

ret = collection.insert([
    ('1', np.random.rand(4), {'document_id': 'paper-01', 'chunk_id': 1, 'content': 'xxxA'}),
    ('2', np.random.rand(4), {'document_id': 'paper-01', 'chunk_id': 2, 'content': 'xxxB'}),
    ('3', np.random.rand(4), {'document_id': 'paper-02', 'chunk_id': 1, 'content': 'xxxC'}),
    ('4', np.random.rand(4), {'document_id': 'paper-02', 'chunk_id': 2, 'content': 'xxxD'}),
    ('5', np.random.rand(4), {'document_id': 'paper-02', 'chunk_id': 3, 'content': 'xxxE'}),
    ('6', np.random.rand(4), {'document_id': 'paper-03', 'chunk_id': 1, 'content': 'xxxF'}),
])
assert ret

Search by vector

Pass a query vector and specify the field to group by. Use group_count to control how many groups are returned, and group_topk to control how many results appear per group.

ret = collection.query_group_by(
    vector=[0.1, 0.2, 0.3, 0.4],
    group_by_field='document_id',  # Group results by the document_id field.
    group_count=2,                 # Return up to two groups.
    group_topk=2,                  # Return up to two results per group.
)
if ret:
    print('query_group_by success')
    print(len(ret))
    print('------------------------')
    for group in ret:
        print('group key:', group.group_id)
        for doc in group.docs:
            prefix = ' -'
            print(prefix, doc)

Expected output:

query_group_by success
4
------------------------
group key: paper-01
 - {"id": "2", "fields": {"document_id": "paper-01", "chunk_id": 2, "content": "xxxB"}, "score": 0.6807}
 - {"id": "1", "fields": {"document_id": "paper-01", "chunk_id": 1, "content": "xxxA"}, "score": 0.4289}
group key: paper-02
 - {"id": "3", "fields": {"document_id": "paper-02", "chunk_id": 1, "content": "xxxC"}, "score": 0.6553}
 - {"id": "5", "fields": {"document_id": "paper-02", "chunk_id": 3, "content": "xxxE"}, "score": 0.4401}

The response contains two groups (paper-01 and paper-02), each with up to two results ranked by similarity score.

Search by primary key

Use the id parameter to perform a grouped search based on the vector stored for a specific document.

ret = collection.query_group_by(
    id='1',
    group_by_field='name',
)
if ret:
    print('query_group_by success')
    print(len(ret))
    for group in ret:
        print('group:', group.group_id)
        for doc in group.docs:
            print(doc)
            print(doc.id)
            print(doc.vector)
            print(doc.fields)

Search with a filter

Combine a vector or primary key with a filter expression to narrow results before grouping. The filter must follow SQL WHERE clause syntax.

ret = collection.query(
    vector=[0.1, 0.2, 0.3, 0.4],   # Use id='...' instead to search by primary key.
    group_by_field='name',
    filter='age > 18',              # Return only documents where age is greater than 18.
    output_fields=['name', 'age'],  # Return only the name and age fields.
    include_vector=True
)

For the full filter syntax, see Conditional filtering.

Search with dense and sparse vectors

Include both vector and sparse_vector to combine dense and sparse retrieval. This enables keyword-aware semantic search, where the sparse vector encodes keyword weights alongside the dense semantic embedding.

ret = collection.query(
    vector=[0.1, 0.2, 0.3, 0.4],
    sparse_vector={1: 0.3, 20: 0.7},
    group_by_field='name',
)

For details on keyword-aware semantic search, see Keyword-aware semantic vector search.

Request parameters

Specify either vector or id.
ParameterTypeDefaultDescription
group_by_fieldstrNoneRequired. The field by which to group results. Schema-free fields are not supported.
vectorOptional[Union[List[Union[int, float]], np.ndarray]]NoneThe query vector.
idOptional[str]NoneThe primary key. The search uses the vector stored for this document.
group_countint10The maximum number of groups to return. This is a best-effort parameter. In general, the specified number of groups can be returned.
group_topkint10The number of similar results to return per group. This is a best-effort parameter with lower priority than group_count.
filterOptional[str]NoneA conditional filter using SQL WHERE clause syntax. See Conditional filtering.
include_vectorboolFalseSpecifies whether to include vector data in the response.
partitionOptional[str]NoneThe partition to search in.
output_fieldsOptional[List[str]]NoneThe fields to return. Returns all fields by default.
sparse_vectorOptional[Dict[int, float]]NoneThe sparse vector for keyword-aware retrieval.
async_reqboolFalseSpecifies whether to send the request asynchronously.

Response parameters

Returns a DashVectorResponse object. For status codes, see Status codes.

ParameterTypeDescriptionExample
codeintThe status code.0
messagestrThe response message.success
request_idstrThe request ID.19215409-ea66-4db9-8764-26ce2eb5bb99
outputList[Group]The grouped search results.

Limitations

  • group_by_field must be a schema-defined field. Schema-free fields are not supported.

  • group_count and group_topk are best-effort parameters. The actual number of groups or results per group returned may be fewer than specified.