Vectorize image data by using open source embedding models of ModelScope

Model overview

This model is designed for large-scale product image similarity search, particularly for luggage items. It automatically performs image matting on luggage products and extracts product characteristics from the matted result — no additional preprocessing input is required.

Note: This model is optimized for luggage product images. Results on other product categories may vary.

Use the following parameters when creating your DashVector collection:

For more information about this model, see Similarity search model based on product characteristics extracted from product images.

Example

The following example shows how to generate an embedding from a product image URL, insert it into a DashVector collection, and run a similarity query.

Replace the following placeholders before running the code:

from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks
from dashvector import Client


# Load the product embedding model
product_embedding = pipeline(
    Tasks.product_retrieval_embedding,
    model='damo/cv_resnet50_product-bag-embedding-models'
)


def generate_embeddings(img: str):
    result = product_embedding(img)
    return result['img_embedding']


# Create a DashVector client
client = Client(
    api_key='{your-dashvector-api-key}',
    endpoint='{your-dashvector-cluster-endpoint}'
)

# Create a collection with 512 dimensions to match the model output
rsp = client.create('resnet50-embedding', dimension=512)
assert rsp
collection = client.get('resnet50-embedding')
assert collection

# Generate an embedding from an image URL and insert it into the collection
img_url = 'https://mmsearch.oss-cn-zhangjiakou.aliyuncs.com/maas_test_img/tb_image_share_1666002161794.jpg'
collection.insert(
    ('ID1', generate_embeddings(img_url))
)

# Query for similar images
docs = collection.query(
    generate_embeddings(img_url)
)
print(docs)