Multimodal search API

更新时间:
复制 MD 格式

This topic describes the request and response parameters for the Multimodal Search API.

Overview

Multimodal Search delivers superior performance in retrieval effectiveness, cost, and latency. It currently supports searching for images by using text.

API reference

Request parameters

Parameter

Type

Required

Default

Description

Example

query

string

Yes

-

The search query. The value must be 1 to 50 characters long.

Important

For optimal performance, we recommend limiting the query to 30 characters. Rewrite longer queries on the client side before calling the API.

lion

advancedParams

map<string, string>

No

false

Advanced search parameters.

  • To exclude specific websites from the search, set the key to excludeSites and the value to a comma-separated list of websites.

{
    "excludeSites": "www.360doc.com,weibo.com"
}

Response parameters

Field

Type

Nullable

Description

Example (query: lion)

requestId

string

No

The ID of the request. Provide this ID when you contact technical support.

35E5608A-A737-2038-****-D9D34C6BFD9E

imageItems[]

title

string

No

The title of the webpage.

A lion's life of battle hides countless legends of life-and-death struggles

hostPageUrl

string

No

The URL of the image's source webpage.

https://example.html

imageUrl

string

No

The URL of the image.

https://imageexample.html

publishedTime

string

No

The webpage's publication time, in ISO 8601 format.

2024-12-18T22:24:24+08:00

height

int32

No

The height of the image in pixels.

1330

width

int32

No

The width of the image in pixels.

1000

searchInformation

searchTime

int64

No

The search duration in milliseconds.

1048

queryContext

originalQuery

query

string

No

The original search query sent in the request.

lion

Note

The server-side timeout is 5 seconds.

Examples

  • Request body

{
  "query": "lion",
  "advancedParams": {
    "excludeSites": "www.360doc.com,weibo.com"
  }
}
  • Response body

{
    "requestId": "********",
    "imageItems": [
        {
            "title": "A lion's life of battle hides countless legends of life-and-death struggles",
            "publishedTime": "2025-04-12T12:47:54+08:00",
            "imageUrl": "http://pic.rmb.bdstatic.com/bjh/bb9193e11fbc/250412/cb5ab496c1b690c6514da34db1ecabb6.jpeg?for=bg",
            "hostPageUrl": "https://quanmin.baidu.com/sv?source=share-h5&pd=qm_share_search&vid=4798422598618199036",
            "height": 1280,
            "width": 720
        },
        {
            "title": "Lion! Good fortune arrives with the sunlight! Lion! Good things arrive with the warm breeze! Prairie",
            "publishedTime": "2025-09-27T03:25:26+08:00",
            "imageUrl": "http://b0.bdstatic.com/ugc/_aq4Eq8b3neWg7bUSTpRMAb2d849c9780458737158f6dec8dc4bef.jpg",
            "hostPageUrl": "http://mbd.baidu.com/newspage/data/dtlandingsuper?nid=dt_5267989015389343625",
            "height": 1232,
            "width": 928
        },
        {
            "title": "Are lions really that smelly? Why are they one of the ten smelliest animals? How bad do they smell?",
            "publishedTime": "2025-10-25T22:17:00+08:00",
            "imageUrl": "http://q9.itc.cn/images01/20251021/2b6c94863d7a45929c8f67590a187329.jpeg",
            "hostPageUrl": "http://news.sohu.com/a/945934548_120545240",
            "height": 853,
            "width": 1280
        },
        {
            "title": "Top of the list: Leo. Lion master! Great news has arrived! The dome shines brightly, overflowing with joy! The entire Lion's Glory Prairie is boiling",
            "publishedTime": "2025-10-05T09:30:43+08:00",
            "imageUrl": "http://pic.rmb.bdstatic.com/bjh/3f119887b814/251005/f0e1de6994276136c35f8763cd79a989.png",
            "hostPageUrl": "http://mbd.baidu.com/newspage/data/dtlandingsuper?nid=dt_4244727221681996203",
            "height": 1121,
            "width": 934
        },
        {
            "title": "curation 146 / lions",
            "publishedTime": "2022-07-03T00:00:00+08:00",
            "imageUrl": "http://sns-img-qc.xhscdn.com/010273016jh26jo8ja7011bs4ue7rjcpzf?imageView2/2/w/1080/format/webp",
            "hostPageUrl": "http://www.xiaohongshu.com/discovery/item/62c1a8ad000000000102fff2",
            "height": 1441,
            "width": 1080
        },
        {
            "title": "This little lion is too boisterous, come and control it!",
            "publishedTime": "2025-07-17T17:57:00+08:00",
            "imageUrl": "http://b0.bdstatic.com/ugc/Hs065dVoGvbUAK9o9yptiAa61f2cd1e32a33239e119b885321c017.jpg",
            "hostPageUrl": "http://mbd.baidu.com/newspage/data/dtlandingsuper?nid=dt_4778610846744027573",
            "height": 1664,
            "width": 936
        },
        {
            "title": "Dao Quan - you sleeping lion of three years, is it time to wake up?!",
            "publishedTime": "2020-05-15T10:45:12+08:00",
            "imageUrl": "http://gbres.dfcfw.com/Files/picture/20200515/E22CB6AF03759A8791300FD476EE8D99_w1680h1050.jpg",
            "hostPageUrl": "http://guba.eastmoney.com/news,002852,930168103,4195112827546318.html",
            "height": 1050,
            "width": 1680
        },
        {
            "title": "Differences and survival status of lion subspecies",
            "publishedTime": "2025-12-13T11:52:08+08:00",
            "imageUrl": "http://pic.rmb.bdstatic.com/bjh/other/7e5c29616163c88db19504ea7dffba39.png?for=bg",
            "hostPageUrl": "https://quanmin.baidu.com/sv?source=share-h5&pd=qm_share_search&vid=18144687723668657466",
            "height": 1664,
            "width": 928
        },
        {
            "title": "When the sunset plates his mane with gold, the king of the prairie sheds all his sharpness. He hides the pride's majesty in an embrace",
            "publishedTime": "2025-09-19T16:05:08+08:00",
            "imageUrl": "http://b0.bdstatic.com/ugc/creative_center/RNbRETUMRKSxdLMpdzvAjAb7ae17de83fff0353bfa3828051f53b5.jpg",
            "hostPageUrl": "http://mbd.baidu.com/newspage/data/dtlandingsuper?nid=dt_4520815613738106269",
            "height": 2188,
            "width": 988
        },
        {
            "title": "This little lion is too boisterous, come and control it!",
            "publishedTime": "2025-07-17T17:57:00+08:00",
            "imageUrl": "http://b0.bdstatic.com/ugc/Hs065dVoGvbUAK9o9yptiA3e638a890ea5ad2223fe6bc41cbb4821.jpg",
            "hostPageUrl": "http://mbd.baidu.com/newspage/data/dtlandingsuper?nid=dt_4778610846744027573",
            "height": 1664,
            "width": 936
        },
        {
            "title": "A heartwarming moment for the lion family, the cub is being cute, and dad watches lovingly, the scene is so full of love #ai creation",
            "publishedTime": "2025-08-02T21:27:14+08:00",
            "imageUrl": "http://b0.bdstatic.com/ugc/Hs065dVoGvbUAK9o9yptiA986dd2160c75da1850b34393452b52ae.jpg",
            "hostPageUrl": "http://mbd.baidu.com/newspage/data/dtlandingsuper?nid=dt_3800333561420709257",
            "height": 1664,
            "width": 936
        },
        {
            "title": "About the zookeeper poking the lion's butt, wildlife up close, zoo, male lion",
            "publishedTime": "2025-08-27T00:00:00+08:00",
            "imageUrl": "http://q6.itc.cn/q_70/images03/20250827/b908b4f0e1234671a3327b5d325a030b.jpeg",
            "hostPageUrl": "http://www.sohu.com/a/928684234_100114195",
            "height": 1280,
            "width": 720
        },
        {
            "title": "HD pictures, Duitang, beauty wallpaper interest community",
            "publishedTime": "2023-12-24T03:23:39+08:00",
            "imageUrl": "http://c-ssl.dtstatic.com/uploads/item/202102/12/20210212165128_losrw.thumb.400_0.jpg",
            "hostPageUrl": "http://www.duitang.com/blogs/tag/?name=%CA%A8%D7%D3%B6%AF%CE%EF%CD%BC%C6%AC&start=144",
            "height": 711,
            "width": 400
        },
        {
            "title": "Unveiling the lion families of Maasai Mara National Park!",
            "publishedTime": "2024-12-25T10:54:17+08:00",
            "imageUrl": "http://miaobi-lite.bj.bcebos.com/miaobi/5mao/b%275Lit5Lic6ZuE54uuXzE3MzM4NDY3MjUuMTgzMjY2Mg%3D%3D%27/0.png",
            "hostPageUrl": "http://mbd.baidu.com/newspage/data/dtlandingsuper?nid=dt_4908065249726726296",
            "height": 1050,
            "width": 1621
        },
        {
            "title": "In a relationship, will a Leo be unfaithful? This is a question from a friend, and I personally think it's quite interesting",
            "publishedTime": "2024-03-16T10:31:08+08:00",
            "imageUrl": "http://pic.rmb.bdstatic.com/bjh/240316/events/40074e13b9df1b66808a4b74c985e4902381.jpeg@h_1280",
            "hostPageUrl": "https://mbd.baidu.com/newspage/data/dtlandingsuper?nid=dt_4520288067222930888",
            "height": 1280,
            "width": 1778
        },
        {
            "title": "Thinking painting Saturday afternoon wax crayon class at 4:50",
            "publishedTime": "2020-11-07T00:00:00+08:00",
            "imageUrl": "http://ss2.meipian.me/users/15565667/12a9d13824a1c862efac377825792599.jpg?meipian-raw/bucket/ivwen/key/dXNlcnMvMTU1NjU2NjcvMTJhOWQxMzgyNGExYzg2MmVmYWMzNzc4MjU3OTI1OTkuanBn/sign/5ff30d4d147e1d9192e168a902c52927.jpg",
            "hostPageUrl": "http://www.meipian.cn/38vc9yej",
            "height": 934,
            "width": 640
        },
        {
            "title": "Male lions flirting with each other",
            "publishedTime": "2026-01-01T20:12:31+08:00",
            "imageUrl": "http://i2.hdslb.com/bfs/archive/e251d2e1386aa0a84a4cf0b9e30168fa80d32a1d.jpg",
            "hostPageUrl": "http://www.bilibili.com/video/BV171vDBtEeA",
            "height": 1440,
            "width": 2560
        },
        {
            "title": "Wallpaper Savanna, lion, animal 1920x1080 full hd 2k hd wallpaper, image, photo",
            "publishedTime": "2020-11-05T12:31:16+08:00",
            "imageUrl": "http://s2.best-wallpaper.net/wallpaper/iphone/1412/Savanna-lion-animals_iphone_640x960.jpg",
            "hostPageUrl": "http://cn.best-wallpaper.net/savanna-lion-animals_wallpapers.html",
            "height": 960,
            "width": 640
        },
        {
            "title": "The scruffy lion of Qianling Mountain took a bath, why is its mane still messy? How to solve the 'dirty' lion's hairstyle problem",
            "publishedTime": "2025-12-26T19:17:10+08:00",
            "imageUrl": "http://i0.hdslb.com/bfs/archive/b9e47b113dae41d8589aceade346a5ee34bb0d3d.jpg",
            "hostPageUrl": "http://www.bilibili.com/video/BV1MdBYBBEke",
            "height": 1125,
            "width": 2000
        },
        {
            "title": "Why is business getting harder and more bosses are going bankrupt. The reason is simple, the more lions on the prairie...",
            "publishedTime": "2023-09-21T08:53:07+08:00",
            "imageUrl": "http://pic.rmb.bdstatic.com/bjh/114531eef1b85ed9aeb34acc4103d73f1885.jpeg@h_1280",
            "hostPageUrl": "https://mbd.baidu.com/newspage/data/dtlandingsuper?nid=dt_4712827839063249262",
            "height": 1280,
            "width": 1918
        }
    ],
    "searchInformation": {
        "searchTime": 808
    },
    "queryContext": {
        "originalQuery": {
            "query": "lion"
        }
    }
}

Error codes

Status

Error code

Error message

Solution

404

InvalidAccessKeyId.NotFound

The specified access key is not found.

Verify that your access key and access key secret are correct.

403

Retrieval.NotActivate

The Information Query Service (IQS) is not activated.

Purchase the service or contact your account manager to activate it.

403

Retrieval.Arrears

Your account has an overdue balance.

Your account balance is insufficient. Please top up your account.

403

Retrieval.NotAuthorised

The sub-account is not authorized.

The sub-account is not authorized. See Create a RAM user and grant permissions.

403

Retrieval.TestUserPeriodExpired

The test period has expired.

The trial period has expired (valid for 15 days after purchase). Contact your account manager to upgrade to a paid plan.

429

Retrieval.Throttling.User

The request was denied due to user-level throttling.

The request rate exceeds the throttling limit. Contact your account manager to upgrade your plan.

429

Retrieval.TestUserQueryPerDayExceeded

The daily query limit has been exceeded.

The daily query limit for the trial has been exceeded (1,000 queries/day). Contact your account manager to upgrade to a paid plan.

API calls

SDK call

To make API calls with an Alibaba Cloud SDK, you must use an access key and access key secret for authentication.

Python SDK

Prerequisite

Make sure that you have Python 3.8 or later installed.

Install SDK
pip3 install alibabacloud_iqs20241111==1.7.1
Sample code
  • Synchronous call

import os

from Tea.exceptions import TeaException
from alibabacloud_iqs20241111 import models
from alibabacloud_iqs20241111.client import Client
from alibabacloud_tea_openapi import models as open_api_models


class Sample:
    def __init__(self):
        pass

    @staticmethod
    def create_client() -> Client:
        config = open_api_models.Config(
            # TODO: Replace with your access key and access key secret. We recommend loading them from environment variables.
            access_key_id='$YOUR_ACCESS_KEY',
            access_key_secret='$YOUR_ACCESS_SECRET'
        )
        config.endpoint = f'iqs.cn-zhangjiakou.aliyuncs.com'
        return Client(config)

    @staticmethod
    def main() -> None:
        client = Sample.create_client()
        advancedParams = {
            "excludeSites": "www.360doc.com,weibo.com"
        }
        run_instances_request = models.MultimodalSearchRequest(
            body=models.MultimodalSearchBody(
                query='lion',
                advanced_params=advancedParams
            )
        )
        try:
            response = client.multimodal_search(run_instances_request)
            print(
                f"api success, request_id:{response.body.request_id}, size :{len(response.body.image_items)}, server_cost:{response.body.search_information.search_time}")
            for index, item in enumerate(response.body.image_items):
                print(f"{index}. {'-' * 20}")
                print(f"title:{item.title}")
                print(f"image_url:{item.image_url}")
                print(f"host_page_url:{item.host_page_url}")
                print(f"published_time:{item.published_time}")
                print(f"width:{item.width}")
                print(f"height:{item.height}")

        except TeaException as e:
            code = e.code
            request_id = e.data.get("requestId")
            message = e.data.get("message")
            print(f"api exception, requestId:{request_id}, code:{code}, message:{message}")


if __name__ == '__main__':
    Sample.main()
  • Asynchronous call

import os
import asyncio

from Tea.exceptions import TeaException
from alibabacloud_iqs20241111 import models
from alibabacloud_iqs20241111.client import Client
from alibabacloud_tea_openapi import models as open_api_models


class Sample:
    def __init__(self):
        pass

    def create_client(self) -> Client:
        config = open_api_models.Config(
            # TODO: Replace with your access key and access key secret. We recommend loading them from environment variables.
            access_key_id='$YOUR_ACCESS_KEY',
            access_key_secret='$YOUR_ACCESS_SECRET'
        )
        config.endpoint = f'iqs.cn-zhangjiakou.aliyuncs.com'
        return Client(config)

    async def main_async(self) -> None:
        client = self.create_client()
        advancedParams = {
            "excludeSites": "www.360doc.com,weibo.com"
        }
        run_instances_request = models.MultimodalSearchRequest(
            body=models.MultimodalSearchBody(
                query='lion',
                advanced_params=advancedParams
            )
        )
        try:
            response = await client.multimodal_search_async(run_instances_request)
            print(
                f"api success, request_id:{response.body.request_id}, size :{len(response.body.image_items)}, server_cost:{response.body.search_information.search_time}")
            for index, item in enumerate(response.body.image_items):
                print(f"{index}. {'-' * 20}")
                print(f"title:{item.title}")
                print(f"image_url:{item.image_url}")
                print(f"host_page_url:{item.host_page_url}")
                print(f"published_time:{item.published_time}")
                print(f"width:{item.width}")
                print(f"height:{item.height}")

        except TeaException as e:
            code = e.code
            request_id = e.data.get("requestId")
            message = e.data.get("message")
            print(f"api exception, requestId:{request_id}, code:{code}, message:{message}")


if __name__ == '__main__':
    sample = Sample()
    asyncio.run(sample.main_async())

Java SDK

Prerequisite

Make sure that you have Java 8 or later installed.

Maven dependency
<dependency>
    <groupId>com.aliyun</groupId>
    <artifactId>iqs20241111</artifactId>
    <version>1.7.1</version>
</dependency>
Sample code
package com.aliyun.iqs.example;

import com.aliyun.iqs20241111.Client;
import com.aliyun.iqs20241111.models.*;
import com.aliyun.teaopenapi.models.Config;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;

import java.util.HashMap;
import java.util.Map;

public class MultimodalSearchExample {
    public static void main(String[] args) throws Exception {
        Client client = initClient();
        invoke(client, "lion", "www.360doc.com,weibo.com");
    }

    private static Client initClient() throws Exception {
        // The following code loads the access key and access key secret from environment variables for security.
        String accessKeyId = System.getenv("ACCESS_KEY");
        String accessKeySecret = System.getenv("ACCESS_SECRET");

        Config config = new Config()
            .setAccessKeyId(accessKeyId)
            .setAccessKeySecret(accessKeySecret);

        config.setEndpoint("iqs.cn-zhangjiakou.aliyuncs.com");
        return new Client(config);
    }

    private static void invoke(Client client, String query, String excludeSites) {

        MultimodalSearchBody input = new MultimodalSearchBody();
        input.setQuery(query);
        Map<String, String> advancedParams = new HashMap<>();
        advancedParams.put("excludeSites",excludeSites);
        input.setAdvancedParams(advancedParams);
        MultimodalSearchRequest request = new MultimodalSearchRequest().setBody(input);

        try {
            long start = System.currentTimeMillis();
            MultimodalSearchResponse response = client.multimodalSearch(request);
            MultimodalSearchOutput result = response.getBody();
            long end = System.currentTimeMillis() - start;
            System.out.println("cost:" + end + " requestId:" + result.getRequestId());
            printOutput(result);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    private static void printOutput(MultimodalSearchOutput output) {
        // Create a Gson instance with pretty printing using GsonBuilder.
        Gson gson = new GsonBuilder()
            .setPrettyPrinting()
            .disableHtmlEscaping()
            .create();

        // Print the formatted JSON.
        String prettyJson = gson.toJson(output);
        System.out.println(prettyJson);
    }
}

Go SDK

Prerequisite

Make sure that your Go environment is version 1.10.x or later.

Install SDK
require (
  github.com/alibabacloud-go/iqs-20241111 v1.7.1
  github.com/alibabacloud-go/darabonba-openapi/v2 v2.0.10
)
Sample code
package main

import (
  "fmt"
  "log"

  openapi "github.com/alibabacloud-go/darabonba-openapi/v2/client"
  iqs20241111 "github.com/alibabacloud-go/iqs-20241111/client"
  util "github.com/alibabacloud-go/tea-utils/v2/service"
  "github.com/alibabacloud-go/tea/tea"
)

const endpointURL = "iqs.cn-zhangjiakou.aliyuncs.com"

func createClient() (*iqs20241111.Client, error) {
  // TODO: Replace with your access key and access key secret.
  accessKeyID := "YOUR_ACCESS_KEY"
  accessKeySecret := "YOUR_ACCESS_SECRET"

  if accessKeyID == "" || accessKeySecret == "" {
    return nil, fmt.Errorf("ACCESS_KEY or ACCESS_SECRET environment variable is not set")
  }

  config := &openapi.Config{
    AccessKeyId:     tea.String(accessKeyID),
    AccessKeySecret: tea.String(accessKeySecret),
    Endpoint:        tea.String(endpointURL),
  }

  return iqs20241111.NewClient(config)
}

func runGenericSearch(client *iqs20241111.Client) error {
  body := &iqs20241111.MultimodalSearchBody{
    Query: tea.String("lion"),
  }
  request := &iqs20241111.MultimodalSearchRequest{
    body,
  }
  runtime := &util.RuntimeOptions{}

  resp, err := client.MultimodalSearchWithOptions(request, nil, runtime)
  if err != nil {
    return fmt.Errorf("multimodal search failed: %w", err)
  }

  fmt.Printf("[%s] response: %s\n", *resp.Body.RequestId, resp.Body)
  return nil
}

func main() {
  client, err := createClient()
  if err != nil {
    log.Fatalf("Failed to create client: %v", err)
  }

  if err := runGenericSearch(client); err != nil {
    log.Fatalf("Error running multimodal search: %v", err)
  }
}

HTTP call

Use the credential (API-KEY) of the Information Query Service (IQS) product to authenticate and make API calls over HTTP. To obtain an API-KEY, see Create and view a credential Key.

curl  -X POST https://cloud-iqs.aliyuncs.com/search/multimodal \
--header "Authorization: Bearer $API_KEY" \
--header "Content-Type: application/json" \
--data '{
  "query": "lion"
}'
Important

Replace $API_KEY with the API key you created in the Information Query Service (IQS) console (see Create and view API keys). It may take up to 5 minutes for a new API key to become effective.