SearchMediaByAILabel-Intelligent Media Services(IMS)-阿里云帮助中心

Performs a multimodal search based on person names, subtitles, and AI categories.

Operation description

Performs a multimodal search based on person names, subtitles, and AI categories. Coarse search returns results at the media asset granularity, while precise search returns matched segment information within the media asset.

Try it now

Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage.

Test

RAM authorization

The table below describes the authorization required to call this API. You can define it in a Resource Access Management (RAM) policy. The table's columns are detailed below:

Action: The actions can be used in the Action element of RAM permission policy statements to grant permissions to perform the operation.
API: The API that you can call to perform the action.
Access level: The predefined level of access granted for each API. Valid values: create, list, get, update, and delete.
Resource type: The type of the resource that supports authorization to perform the action. It indicates if the action supports resource-level permission. The specified resource must be compatible with the action. Otherwise, the policy will be ineffective.
- For APIs with resource-level permissions, required resource types are marked with an asterisk (*). Specify the corresponding Alibaba Cloud Resource Name (ARN) in the Resource element of the policy.
- For APIs without resource-level permissions, it is shown as All Resources. Use an asterisk (*) in the Resource element of the policy.
Condition key: The condition keys defined by the service. The key allows for granular control, applying to either actions alone or actions associated with specific resources. In addition to service-specific condition keys, Alibaba Cloud provides a set of common condition keys applicable across all RAM-supported services.
Dependent action: The dependent actions required to run the action. To complete the action, the RAM user or the RAM role must have the permissions to perform all dependent actions.

Action

Access level

Resource type

Condition key

Dependent action

ice:SearchMediaByAILabel

get

*All Resource

*

None

Request parameters

Parameter	Type	Required	Description	Example
SearchLibName	string	No	The name of the search library.	test-1
MediaType	string	No	The media asset type. Valid values: image: images video: videos audio: audio Valid values: image : image video : video audio : audio	video
SortBy	string	No	The sorting of query results. Valid values: CreationTime:Desc (default): sorts by creation time in descending order. CreationTime:Asc: sorts by creation time in ascending order.	CreationTime:Desc
PageNo	integer	No	The current page number. Default value: 1.	1
PageSize	integer	No	The number of entries to return on each page. Default value: 10. Maximum value: 50.	20
Text	string	No	The search content.	kite
MultimodalSearchType	string	No	The multimodal search type. Default value: FullSearch. Valid values: PersonName: person name search Ocr: subtitle search AiCategory: AI categorization search FullSearch: full text search Valid values: PersonName : PersonName AiCategory : AiCategory FullSearch : FullSearch Ocr : Ocr	Ocr
SpecificSearch	boolean	No	Specifies whether to perform a precise search. Valid values: true: performs a precise search. false: performs a coarse search.	true
MediaId	string	No	The media asset ID. This parameter is required for precise search.	**c469e944b5a856828dc2**
MatchingMode	string	No	The matching mode. Default value: fuzzy. Valid values: fuzzy: fuzzy match. precise: exact match. This mode takes effect only for newly added media assets. Valid values: precise : precise fuzzy : fuzzy	fuzzy
Namespace	string	No	The namespace.	name-1
CustomFilters	string	No	The custom filters. The value is a JSON string. Supported fields include the integer field intField1 and the string fields strField1 and strField2. You can select at most one match type for the same field. Filters for different fields use the AND logical operator. Exact match, such as {"intField1":12,"strField1":"abc"}. Multi-value match, such as {"intField1":[12,13],"strField1":["abc","cd"]}. Range match, such as {"intField1":{"gte":12,"lte":13}}.	{"intField1":{"gte":12,"lte":13},"strField2":["cd","de"],"strField1":"abc"}
UtcCreate	string	No	The creation time, in milliseconds. gte indicates greater than or equal to, and lte indicates less than or equal to. Range match, such as {"gte":1761205662998,"lte":1771205662998}.	{"gte":1761205662998,"lte":1771205662998}

Response elements

Element	Type	Description	Example
	object	Schema of Response
RequestId	string	The request ID.	****11-DB8D-4A9A-875B-275798****
Success	string	Indicates whether the request is successful.	true
Code	string	The return code.	200
MediaList	array<object>	The collection of matched media assets.
	array<object>	The media asset details.
AiData	object	The detailed AI information.
AiLabelInfo	array<object>	The AI label information list.
	array<object>	The AI label information.
FaceId	string	The face ID.	5FE19530C7A422197535FE74F5DB****
LabelType	string	The label type.	object
Source	string	The source.	vision
Category	string	The category.	Means of transportation
LabelName	string	The entity name.	Car
LabelId	string	The entity ID.	103102503**
Occurrences	array<object>	The segment information.
	array<object>	The segment.
Score	number	The score.	0.75287705
Image	string	The optimal face image.	https://service-**-public.oss-cn-hangzhou.aliyuncs.com/1563457438522/service-image/f788974f-9595-43b2-a478-7c7a1afb**.jpg
TableBatchSeqId	string	The vector table sequence ID.	85010D1**
ClipId	string	The segment ID.	158730355E4B82257D8AA1583A58****
From	number	The start time of the segment.	1.4
To	number	The end time of the segment.	2.5
FinegrainName	string	The fine-grained name of the entity.	Car
FinegrainId	string	The fine-grained ID of the entity.	103102503**
Content	string	The text content.	Smart label test
Tracks	array<object>	The track sequence.
	object	The sequence.
Size	number	The area of the face bounding box.	50
Position	string	The position coordinates of the bounding box.	468.0;67.0;615.0;267.0
Timestamp	number	The timestamp of the track.	1.4
AsrInfo	array<object>	The speech information list.
	object	The speech information.
From	number	The start time of the segment.	1.4
To	number	The end time of the segment.	2.5
ClipId	string	The segment ID.	5FE19530C7A422197535FE74F5DB****
Content	string	The text content.	Smart label test
Timestamp	number	The segment timestamp.	1.4
OcrInfo	array<object>	The subtitle information list.
	object	The subtitle information.
From	number	The start time of the segment.	1.4
To	number	The end time of the segment.	2.5
ClipId	string	The segment ID.	5FE19530C7A422197535FE74F5DB****
Content	string	The text content.	Pipi
Timestamp	number	The segment timestamp.	1.4
StorageLocation	string	The storage location.	out-****.oss-cn-shanghai.aliyuncs.com
Status	string	The video status. Valid values: PrepareFail : PrepareFail UploadFail : UploadFail Init : Init UploadSucc : UploadSucc Transcoding : Transcoding TranscodeFail : TranscodeFail Deleted : Deleted Normal : Normal Uploading : Uploading Preparing : Preparing Blocked : Blocked Checking : Checking	Normal
CreationTime	string	The creation time. The format is yyyy-MM-ddTHH:mm:ssZ (UTC time).	2017-11-14T09:15:50Z
ModificationTime	string	The modification time. The format is yyyy-MM-ddTHH:mm:ssZ (UTC time).	2017-11-14T09:15:50Z
MediaId	string	The media asset ID.	1c6ce34007d571ed94667630a6bc****
Tags	string	The tag list.	Tag1, Tag2
Description	string	The description.	Alibaba Cloud VOD video description
AppId	string	The application ID. Default value: app-1000000.	app-1000000
Size	integer	The size of the source file. Unit: bytes.	10897890
CoverUrl	string	The cover URL.	http://example.aliyundoc.com/snapshot/**.jpg?auth_key=1498476426-0-0-f00b9455c49a423ce69cf4e27333**
Duration	number	The duration. Unit: seconds.	12.2
Title	string	The title.	The greatest music player in history.
Snapshots	array	The array of video snapshot URLs.
	string	The video snapshot URL.	http://example.aliyundoc.com/snapshot/**.jpg?auth_key=1498476426-0-0-f00b9455c49a423ce69cf4e27333**
Total	integer	The total number of matched media assets.	30

Examples

Success response

JSON format

{
  "RequestId": "******11-DB8D-4A9A-875B-275798******",
  "Success": "true",
  "Code": "200",
  "MediaList": [
    {
      "AiData": {
        "AiLabelInfo": [
          {
            "FaceId": "5FE19530C7A422197535FE74F5DB****\n",
            "LabelType": "object",
            "Source": "vision",
            "Category": "Means of transportation",
            "LabelName": "Car",
            "LabelId": "103102503**",
            "Occurrences": [
              {
                "Score": 0.75287705,
                "Image": "https://service-****-public.oss-cn-hangzhou.aliyuncs.com/1563457****438522/service-image/f788974f-9595-43b2-a478-7c7a1afb****.jpg",
                "TableBatchSeqId": "85010D1**",
                "ClipId": "158730355E4B82257D8AA1583A58****",
                "From": 1.4,
                "To": 2.5,
                "FinegrainName": "Car",
                "FinegrainId": "103102503**",
                "Content": "Smart label test",
                "Tracks": [
                  {
                    "Size": 50,
                    "Position": "468.0;67.0;615.0;267.0",
                    "Timestamp": 1.4
                  }
                ]
              }
            ]
          }
        ],
        "AsrInfo": [
          {
            "From": 1.4,
            "To": 2.5,
            "ClipId": "5FE19530C7A422197535FE74F5DB****",
            "Content": "Smart label test",
            "Timestamp": 1.4
          }
        ],
        "OcrInfo": [
          {
            "From": 1.4,
            "To": 2.5,
            "ClipId": "5FE19530C7A422197535FE74F5DB****\n",
            "Content": "Pipi",
            "Timestamp": 1.4
          }
        ]
      },
      "StorageLocation": "out-****.oss-cn-shanghai.aliyuncs.com\n",
      "Status": "Normal",
      "CreationTime": "2017-11-14T09:15:50Z\n",
      "ModificationTime": "2017-11-14T09:15:50Z\n",
      "MediaId": "1c6ce34007d571ed94667630a6bc****\n",
      "Tags": "Tag1, Tag2\n",
      "Description": "Alibaba Cloud VOD video description\n",
      "AppId": "app-1000000",
      "Size": 10897890,
      "CoverUrl": "http://example.aliyundoc.com/snapshot/****.jpg?auth_key=1498476426-0-0-f00b9455c49a423ce69cf4e27333****",
      "Duration": 12.2,
      "Title": "The greatest music player in history.\n",
      "Snapshots": [
        "http://example.aliyundoc.com/snapshot/****.jpg?auth_key=1498476426-0-0-f00b9455c49a423ce69cf4e27333****"
      ]
    }
  ],
  "Total": 30
}

Error codes

See Error Codes for a complete list.

Release notes

See Release Notes for a complete list.