Multi-vector retrieval

In some AI retrieval scenarios, an entity can correspond to multiple vectors. To find the most likely entity, you can search across all these vectors. Examples include the following:

• In Natural Language Processing (NLP), a title and a document are extracted into separate vector features. You need to search based on both the title vector and the document vector at the same time.

• In product retrieval, a product's image and text are extracted into separate vector features. You need to search based on both the image vector and the text vector at the same time.

DashVector supports multi-vector retrieval for these scenarios.

Usage example

Prerequisites

• You have created a cluster. For more information, see Create a cluster.

• You have obtained an API key. For more information, see API key management.

• You have installed the latest version of the software development kit (SDK). For more information, see Install the DashVector SDK.

Create a multi-vector collection

import dashvector
import numpy as np
from dashvector import VectorParam, Doc, WeightedRanker, VectorQuery, RrfRanker

client = dashvector.Client(
    api_key='YOUR_API_KEY',
    endpoint='YOUR_CLUSTER_ENDPOINT'
)
ret = client.create(
    'multi_vector_demo',
    vectors={
        "title": VectorParam(4),
        "content": VectorParam(6, metric="euclidean"),
    },
    fields_schema={
        'author': str,
    }
)
assert ret

Insert data

collection = client.get(name='multi_vector_demo')
docs = []
for i in range(10):
    docs.append(Doc(id=str(i),
                    vectors={"title": np.random.random(4),
                             "content": np.random.random(6)
                            },
                    )
                )
ret = collection.insert(docs)
print(ret)

Execute retrieval

title_vector = [0.1, 0.2, 0.3, 0.4]
content_vector = [0.1, 0.2, 0.3, 0.4, 0.5, 0.6]
vectors = {
    "title": VectorQuery(vector=title_vector, num_candidates=10),
    "content": VectorQuery(vector=content_vector),
}
ret = collection.query(
    vector=vectors,
    include_vector=False,
    # Use RRF for fusion sorting
    # rerank=RrfRanker(rank_constant=100)
    # Use weighted fusion sorting
    rerank=WeightedRanker(weights={"title": 1.0, "content": 1.0}),
    topk=20
)
assert ret
print(ret.output)

# Perform retrieval using a single vector
ret = collection.query(vector={"title": VectorQuery(vector=title_vector)})
assert ret
print(ret.output)

# Perform grouped vector retrieval using a single vector
ret = collection.query_group_by(title_vector, group_by_field='author', vector_field='title')
assert ret
print(ret)

Limitations