Use open source embedding models from ModelScope to convert text into vectors

Prerequisites

Before you begin, ensure that you have:

• A VRS cluster — Create a cluster

• A VRS API key — Manage API keys

• The latest VRS SDK installed — Install DashVector SDK

• The ModelScope SDK installed:

  pip install -U modelscope

Choose a model

Browse all sentence embedding models on ModelScope to find a model that fits your language and domain. The tables below list the most commonly used model families. For each model, note the model ID and dimensions — you will use both values in the code.

Generate embeddings and store them in VRS

The code pattern is the same for all models listed above — CoROM, GTE, Udever, and the additional models. Replace <model-id> with your chosen model ID and <dimensions> with the corresponding dimensions value from the tables above.

StructBERT FAQ models use a different pipeline task. See Generate embeddings with StructBERT.

from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
from typing import List
from dashvector import Client

# Replace <model-id> with your chosen model ID, e.g., damo/nlp_corom_sentence-embedding_english-base
pipeline_se = pipeline(Tasks.sentence_embedding, model='<model-id>')


def generate_embeddings(texts: List[str]):
    inputs = {'source_sentence': texts}
    result = pipeline_se(input=inputs)
    return result['text_embedding']


# Create a VRS client.
client = Client(
    api_key='{your-dashvector-api-key}',
    endpoint='{your-dashvector-cluster-endpoint}'
)

# Create a collection.
# Set dimension to match the model's output dimensions from the table above.
# For example, use 768 for CoROM base models and 256 for CoROM tiny models.
rsp = client.create('text-embedding', dimension=<dimensions>)
assert rsp
collection = client.get('text-embedding')
assert collection

# Insert a vector
collection.insert(
    ('ID1', generate_embeddings(['Alibaba Cloud DashVector is one of the best vector databases in performance and cost-effectiveness.'])[0])
)

# Run a vector search
docs = collection.query(
    generate_embeddings(['The best vector database'])[0]
)
print(docs)

Replace the following placeholders in the code:

Generate embeddings with StructBERT

StructBERT FAQ models use the faq_question_answering task instead of sentence_embedding. Replace <model-id> with the StructBERT model ID you want to use.

from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
from typing import List

# Replace <model-id> with a StructBERT model ID,
# e.g., damo/nlp_structbert_faq-question-answering_chinese-base
pipeline = pipeline(Tasks.faq_question_answering, model='<model-id>')


def generate_embeddings(texts: List[str], max_len=30):
    return pipeline.get_sentence_embedding(texts)

To store the resulting vectors in VRS and run searches, use the same client, collection.insert(), and collection.query() calls shown in the previous section.

What's next

• Create a cluster

• Manage API keys

• Install DashVector SDK