DocsICP FilingConsole
官方文档
首页

Multimodal reranking

更新时间：
复制 MD 格式
产品详情
我的收藏

Scores the relevance between a query and a set of candidate documents. Both the query and documents can be text, images, or a mix of both. Each document receives a relevance score from 0 to 1—the higher the score, the more relevant the document is to the query.

Service Service ID Description API QPS limit
Multimodal reranking ops-mm-reranker-001 Scores the relevance between a query and candidate documents for image retrieval, retrieval-augmented generation (RAG), and multimodal search. Returns documents sorted by relevance score. 20 (shared across Alibaba Cloud accounts and RAM users). To apply for a higher limit, submit a ticket.

Prerequisites

Before you begin, make sure you have:

  • An API key: Required to authenticate API calls to the AI Search Open Platform.

  • A service endpoint: The endpoint for calling the service over the public network or a virtual private cloud (VPC). For details, see Get service registration address.

Make a request

Request method and URL

POST {host}/v3/openapi/workspaces/{workspace_name}/multi-modal-reranker/{service_id}

URL parameters:

Parameter Description Example
host The service endpoint. Call the service over the public network or through a VPC. ****-hangzhou.opensearch.aliyuncs.com
workspace_name The name of the workspace. default
service_id The built-in service ID. Must start with ops-. ops-mm-reranker-001

Constraints:

  • Request body size: 8 MB maximum.

Request headers

Parameter Type Required Description Example
Content-Type String Yes The request content type. application/json
Authorization String Yes The API key in Bearer token format. Bearer OS-d1**2a

Request body

Parameter Type Required Description Example
service_id String Yes The service ID. Must start with ops-. ops-mm-reranker-001
query ContentObject Yes The query content. Provide either text or image—not both, and not neither. {"text": "Is there a cake in the picture?"}
docs List[ContentObject] Yes The candidate documents to rank. Up to 100 documents per request. See ContentObject
options OptionObject No If the input includes an image, use options to configure the image resize settings. See OptionObject

ContentObject

Each ContentObject represents a single piece of content. Provide exactly one of text or image.

Parameter Type Required Description
text String No Plain text content.
image String No An image URL or a Base64-encoded image string. If using a URL, the URL must be publicly accessible.

Image input examples:

Using a URL:

{
  "image": "http://xxxxx/a.jpg"
}

Using a Base64-encoded string:

{
  "image": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAoHCB..."
}

OptionObject

Use options to control how the model resizes images before processing.

Parameter Type Required Description Example
resize_method String No The image resize method. Valid values: default (built-in scaling), none (no scaling), fixed_size (resize to exact dimensions), proportional_scaling (resize within a pixel range while maintaining aspect ratio). fixed_size
resize_options Object No Resize parameters. Required when resize_method is fixed_size or proportional_scaling. See below.

resize_options format:

If resize_method is fixed_size:

{
  "width": 224,
  "height": 224
}

If resize_method is proportional_scaling:

{
  "min_pixels": 409600,
  "max_pixels": 1638400
}

Both width and height are in pixels. min_pixels and max_pixels define the pixel count range for the resized image.

Response parameters

Parameter Type Description Example
request_id String The unique ID for the API call, generated by the system. A5B25952-4406-45BF-99EC-E8020246****
latency Float/Int The request processing time in milliseconds. 1765
usage.doc_count Int The number of documents in the request. 2
result.scores List The relevance scores, sorted by score in descending order. Each entry contains index and score. —
result.scores[].index Int The position of the document in the original docs array. Use this to map scores back to your input documents. 0
result.scores[].score Float The relevance score. A higher value indicates stronger relevance. 0.9183856248855591

Examples

cURL

curl --location 'http://****-hangzhou.opensearch.aliyuncs.com/v3/openapi/workspaces/default/multi-modal-reranker/ops-mm-reranker-001/' \
  --header 'Authorization: Bearer <YOUR_API_KEY>' \
  --header 'Content-Type: application/json' \
  --data '{
    "service_id": "ops-mm-reranker-001",
    "query": {
      "text": "Is there a cake in the picture?"
    },
    "docs": [
      {
        "image": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250408/syuvxh/%E7%89%A9%E4%BD%93%E5%AE%9A%E4%BD%8D.png"
      }
    ]
  }'

Successful response

{
  "request_id": "1b87e3c462079a32999a7c8bc173ca7a",
  "latency": 1765,
  "usage": {
    "image_token": 1225,
    "text_token": 16
  },
  "result": {
    "scores": [
      {
        "index": 0,
        "score": 0.9183856248855591
      }
    ]
  }
}

The scores array is sorted by score in descending order. Use scores[].index to map each score back to the corresponding document in your input docs array.

Status codes

For a full list of status codes and their meanings, see Status codes.

Previous：RerankNext：Object detection