QueryVideoCognitionJob-Intelligent Media Services(IMS)-阿里云帮助中心

Queries the task result of the video understanding node.

Try it now

Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage.

Test

RAM authorization

The table below describes the authorization required to call this API. You can define it in a Resource Access Management (RAM) policy. The table's columns are detailed below:

Action: The actions can be used in the Action element of RAM permission policy statements to grant permissions to perform the operation.
API: The API that you can call to perform the action.
Access level: The predefined level of access granted for each API. Valid values: create, list, get, update, and delete.
Resource type: The type of the resource that supports authorization to perform the action. It indicates if the action supports resource-level permission. The specified resource must be compatible with the action. Otherwise, the policy will be ineffective.
- For APIs with resource-level permissions, required resource types are marked with an asterisk (*). Specify the corresponding Alibaba Cloud Resource Name (ARN) in the Resource element of the policy.
- For APIs without resource-level permissions, it is shown as All Resources. Use an asterisk (*) in the Resource element of the policy.
Condition key: The condition keys defined by the service. The key allows for granular control, applying to either actions alone or actions associated with specific resources. In addition to service-specific condition keys, Alibaba Cloud provides a set of common condition keys applicable across all RAM-supported services.
Dependent action: The dependent actions required to run the action. To complete the action, the RAM user or the RAM role must have the permissions to perform all dependent actions.

Action

Access level

Resource type

Condition key

Dependent action

ice:QueryVideoCognitionJob

get

*All Resource

*

None

Request parameters

Parameter	Type	Required	Description	Example
JobId	string	Yes	The ID of the intelligent tagging job to query. You can obtain this ID from the response parameters of the Submit an intelligent tagging job operation.	**20b48fb04483915d4f2cd8ac**
Params	string	No	The additional request parameters, represented as a JSON string.	{}
IncludeResults	object	No	Specifies whether to include the complete algorithm results in the query results.
NeedAsr	boolean	No	Specifies whether to include the automatic speech recognition (ASR) results.	true
NeedOcr	boolean	No	Specifies whether to include the optical character recognition (OCR) results.	true
NeedProcess	boolean	No	Specifies whether to include the links to the original results of the operators.	true

Response elements

Element	Type	Description	Example
	object
JobStatus	string	The status of the job. Valid values: Success: The job is completed. Fail: The job failed. Processing: The job is being processed. Submitted: The job is submitted and waiting to be processed.	Success
RequestId	string	The ID of the request.	****11-DB8D-4A9A-875B-275798****
UserData	string	The custom user information.	{"userId":"123432412831"}
Results	object
result	array<object>	The array of analysis result objects.
	object	The video content understanding results.
Type	string	The type of the analysis result. Valid values: TextLabel: text labels. VideoLabel: video labels. ASR: original automatic speech recognition (ASR) results, which are not returned by default. OCR: original optical character recognition (OCR) results, which are not returned by default. NLP: natural language processing (NLP) results, which are not returned by default. Process: links to the original results of the operators, which are not returned by default.	ASR
Data	string	The specific data of the analysis result, represented as a JSON string. For the data structures of different types, see the Result parameter description below.	{"title":"example-title-****"}
TemplateId	string
Params	string
Input	object	The input media asset information of the job.
Type	string	The type of the input file. Valid value: OSS.
Media	string	The URL of the input file.

Result parameter description

VideoLabel data structure

Name	Type	Description
persons	JSONArray	The detected person results.
persons.name	String	The name of the recognized person.
persons.category	String	The type of the person. Valid values: `celebrity` (celebrity), `politician` (political figure), `sensitive` (sensitive figure), `unknown` (unknown person), and the ID of the custom person library (returned when the recognized person is a custom person).
persons.ratio	double	The appearance rate of the person. Valid values: 0 to 1.
persons.occurrences	JSONArray	The detailed information about the person's appearances.
persons.occurrences.score	double	The confidence score.
persons.occurrences.from	double	The start time when the person is detected, in seconds.
persons.occurrences.to	double	The end time when the person is detected, in seconds.
persons.occurrences.position	JSONObject	The coordinates of the face.
persons.occurrences.position.leftTop	int[]	The x and y coordinates of the upper-left corner.
persons.occurrences.position.rightBottom	int[]	The x and y coordinates of the lower-right corner.
persons.occurrences.timestamp	double	The timestamp of the face coordinates, in seconds.
persons.occurrences.scene	String	The shot type of the person. Valid values: `closeUp` (close-up), `medium-closeUp` (medium close-up), `medium` (medium shot), and `medium-long` (medium long shot).
tags	JSONArray	The detected tags for objects and scenes. For more information, see the following table.
tags.mainTagName	String	The main tag.
tags.subTagName	String	The sub-tag.
tags.ratio	double	The appearance rate. Valid values: 0 to 1.
tags.occurrences	JSONArray	The detailed information about the appearances.
tags.occurrences.score	double	The confidence score.
tags.occurrences.from	double	The start time, in seconds.
tags.occurrences.to	double	The end time, in seconds.
classifications	JSONArray	The video classification information.
classifications.score	double	The confidence score.
classifications.category1	String	The level-1 category, such as life, anime, and automobile.
classifications.category2	String	The level-2 category. For example, the life category can be subdivided into health and home.

Examples of video tags

mainTagName	subTagName
Program	For example, Where Are We Going, Dad? and Happy Comedian.
Character role	For example, doctor, nurse, and teacher.
Object	For example, piano, water cup, desk, scrambled eggs with tomatoes, car, and cosmetics.
Logo	For example, CCTV1, CCTV2, Youku, and Dragon TV.
Action and event	For example, dancing, kissing, hugging, meeting, singing, making a phone call, horseback riding, and fighting.
Region	For example, Tiananmen Square in Beijing, Statue of Liberty, Leshan Giant Buddha, China, and the United States.
Scene	For example, bedroom, subway station, terraced fields, beach, and desert.

ImageLabel data structure

Name	Type	Description
persons	JSONArray	The detected person results.
persons.name	String	The name of the recognized person.
persons.category	String	The type of the person. Valid values: `celebrity` (celebrity), `politician` (political figure), and `sensitive` (sensitive figure).
persons.score	double	The confidence score of the person.
persons.position	JSONObject	The coordinates of the face.
persons.position.leftTop	int[]	The x and y coordinates of the upper-left corner.
persons.position.rightBottom	int[]	The x and y coordinates of the lower-right corner.
persons.scene	String	The shot type of the person. Valid values: `closeUp` (close-up), `medium-closeUp` (medium close-up), `medium` (medium shot), and `medium-long` (medium long shot).
tags	JSONArray	The detected tags for objects and scenes. For more information, see the following table.
tags.mainTagName	String	The main tag.
tags.subTagName	String	The sub-tag.
tags.score	double	The confidence score.

Examples of image tags

mainTagName	subTagName
Character role	For example, doctor, nurse, and teacher.
Region	For example, Tiananmen Square in Beijing, Statue of Liberty, Leshan Giant Buddha, China, and the United States.
Action and event	For example, speaking.
Logo	For example, CCTV1, CCTV2, Youku, and Dragon TV.
Action and event	For example, dancing, kissing, hugging, meeting, singing, making a phone call, horseback riding, and fighting.
Object	For example, piano, water cup, desk, scrambled eggs with tomatoes, car, and cosmetics.
Scene	For example, bedroom, subway station, terraced fields, beach, and desert.

TextLabel data structure (derived from ASR and OCR)

Name	Type	Description
tags	JSONArray	The text tag information. For more information, see the following table.
tags.name	String	The tag category.
tags.value	String	The tag value. Multiple tag values are separated by commas (,).

Examples of text tags

name	value
Region	For example, Tiananmen Square in Beijing, Statue of Liberty, Leshan Giant Buddha, China, and the United States.
Organization	For example, China Wildlife Conservation Association and China Media Group.
Logo	For example, Nike and Li-Ning.
Keyword	For example, core force.

CPVLabel data structure

cates: categories (level-1, level-2, and level-3 categories)
entities: category attributes (with knowledge graph information)
hotwords: hot words (user-followed lists)
freeTags: free tags (keywords)

Name	Type	Example	Description
type	String	hmi	The result type. Valid values: `hmi` (human-machine collaboration result) and `autp` (machine tagging result).
cates	JSONArray	-	The classification category results.
cates.labelLevel1	String	Tourism	The level-1 tag.
cates.labelLevel2	String	Tourism scenery	The level-2 tag.
cates.label	String	""	The tag name. The algorithm may return an empty string.
cates.appearanceProbability	double	0.96	The appearance rate.
cates.detailInfo	JSONArray	-	-
cates.detailInfo.score	double	0.9	The confidence score.
cates.detailInfo.startTime	double	0.021	The start time.
cates.detailInfo.endTime	double	29.021	The end time.
entities	JSONArray	-	-
entities.labelLevel1	String	Region	The level-1 tag.
entities.labelLevel2	String	Landmark	The level-2 tag.
entities.label	String	Huangguoshu Waterfall	The tag name.
entities.appearanceProbability	double	0.067	The appearance rate.
entities.knowledgeInfo	String	{"name": "Huangguoshu Waterfall", "nameEn": "Huangguoshu Waterfall", "description": "One of the four major waterfalls in Asia"}	The knowledge graph information. For all fields, see the attached tables: Film and Variety IP Graph, Music Graph, Person Graph, Landmark Graph, and Object Graph.
entities.detailInfo	JSONArray	-	-
entities.detailInfo.score	double	0.33292606472969055	The confidence score.
entities.detailInfo.startTime	double	6.021	The start time.
entities.detailInfo.endTime	double	8.021	The end time.
entities.detailInfo.trackData	JSONArray	-	The structured information of the entity tag.
entities.detailInfo.trackData.score	double	0.32	The confidence score.
entities.detailInfo.trackData.bbox	integer[]	23, 43, 45, 67	The bounding box coordinates.
entities.detailInfo.trackData.timestamp	double	7.9	The timestamp.
hotwords	JSONArray	-	-
hotwords.labelLevel1	String	Keyword	The level-1 tag.
hotwords.labelLevel1	String	Keyword	The level-2 tag.
hotwords.labelLevel2	String	""	The level-2 tag.
hotwords.label	String	China Meteorological Administration	The highlight word content.
hotwords.appearanceProbability	double	0.96	The appearance rate.
hotwords.detailInfo	JSONArray
hotwords.detailInfo.score	double	1.0	The confidence score.
hotwords.detailInfo.startTime	double	0.021	The start time.
hotwords.detailInfo.endTime	double	29.021	The end time.
freeTags	JSONArray
freeTags.labelLevel1	String	Keyword	The level-1 tag.
freeTags.labelLevel2	String	""	The level-2 tag.
freeTags.label	String	National Meteorological Center	The highlight word content.
freeTags.appearanceProbability	double	0.96	The appearance rate.
freeTags.detailInfo	JSONArray
freeTags.detailInfo.score	double	0.9	The confidence score.
freeTags.detailInfo.startTime	double	0.021	The start time.
freeTags.detailInfo.endTime	double	29.021	The end time.

Automatic speech recognition (ASR) results

Name	Type	Description
details	JSONArray	The detailed results of the job.
details.from	double	The start timestamp, in seconds.
details.to	double	The end timestamp, in seconds.
details.content	String	The recognized text content.

Optical character recognition (OCR) results

Name	Type	Description
details	JSONArray	The detailed results of the job.
details.timestamp	double	The timestamp information, in seconds.
details.info	JSONArray	The specific information recognized at the timestamp.
details.info.score	double	The confidence score.
details.info.position	JSONObject	The coordinates of the text.
details.info.position.leftTop	int[]	The x and y coordinates of the upper-left corner.
details.info.position.rightBottom	int[]	The x and y coordinates of the lower-right corner.
details.info.content	String	The recognized text content.

Meta tagging results

Note

If you do not use human-machine tagging and specify needMetaData when submitting a job by calling SubmitSmarttagJob, the original title entered by the user is returned when you call QuerySmarttagJob to query the results.

Name	Type	Description
title	String	The title.

Subtitle extraction results

Name	Type	Description
details	JSONArray	The detailed results of the job.
details.allResultUrl	String	The URL of all subtitle results. The URL is valid for half a year after the job is completed.
details.chResultUrl	String	The URL of the Chinese subtitle results. The URL is valid for half a year after the job is completed.
details.engResultUrl	String	The URL of the English subtitle results. The URL is valid for half a year after the job is completed.

Note

Content of the subtitle result URL: Sequence number + Time period + Subtitle content (each subtitle sentence occupies one line).

Natural language processing (NLP) results

Parameter	Type	Description
transcription	object	The speech transcription results.
autoChapters	object	The chapter overview results.
summarization	object	The large model summarization results.
meetingAssistance	object	The intelligent meeting minutes results.
translation	object	The text translation results.

transcription (Speech transcription)

Parameter	Type	Description
transcription	object	The speech transcription result object.
transcription.paragraphs	list[]	The collection of speech transcription structures organized in paragraphs.
transcription.paragraphs[i].paragraphId	string	The paragraph-level ID.
transcription.paragraphs[i].speakerId	string	The speaker ID.
transcription.paragraphs[i].words	list[]	The word information contained in the paragraph.
transcription.paragraphs[i].words[i].id	int	The word sequence number, which usually does not require attention.
transcription.paragraphs[i].words[i].sentenceId	int	The sentence ID. Word information with the same SentenceId can be assembled into a sentence.
transcription.paragraphs[i].words[i].start	long	The start time of the word relative to the start time of the audio. It is a relative timestamp, in milliseconds.
transcription.paragraphs[i].words[i].end	long	The end time of the word relative to the start time of the audio. It is a relative timestamp, in milliseconds.
transcription.paragraphs[i].words[i].text	string	The word text.

summarization (Full-text summary, speaker summary, and question summary)

Parameter	Type	Description
summarization	object	The summary result object, which may contain zero or more results of different summary types.
summarization.paragraphSummary	string	The full-text summary result.
summarization.conversationalSummary	list[]	The list of speaker summary results.
summarization.conversationalSummary[i].speakerId	string	The speaker ID.
summarization.conversationalSummary[i].speakerName	string	The speaker name.
summarization.conversationalSummary[i].summary	string	The summary corresponding to the speaker.
summarization.questionsAnsweringSummary	list[]	The list of question answering summary results.
summarization.questionsAnsweringSummary[i].question	string	The question.
summarization.questionsAnsweringSummary[i].sentenceIdsOfQuestion	list[]	The list of SentenceIds of the original speech transcription from which the question is extracted.
summarization.questionsAnsweringSummary[i].answer	string	The answer corresponding to the question.
summarization.questionsAnsweringSummary[i].sentenceIdsOfAnswer	list[]	The list of SentenceIds of the original speech transcription from which the answer is summarized.
summarization.mindMapSummary	list[object]	The list of mind map summary results, which may contain summaries of various topics and relationships between topics.
summarization.mindMapSummary[i].title	string	The title of the topic.
summarization.mindMapSummary[i].topic	list[object]	The array containing each topic and its subtopics.
summarization.mindMapSummary[i].topic[i].title	string	The title of the topic.
summarization.mindMapSummary[i].topic[i].topic	list[object]	The array of subtopics of the topic, which can be an empty array.

translation (Full-text translation)

Parameter	Type	Description
translation	object	The translation result object.
translation.paragraphs	list[]	The collection of translation results organized in paragraphs, corresponding to the speech recognition result message.
translation.paragraphs.paragraphId	string	The paragraph segmentation ID, corresponding to the ParagraphId in the speech recognition results.
translation.paragraphs.sentences	list[]	The collection of translated texts.
translation.paragraphs.sentences[i].sentenctId	long	The sentence ID.
translation.paragraphs.sentences[i].start	long	The start time of the paragraph relative to the start time of the audio. It is a relative timestamp, in milliseconds.
translation.paragraphs.sentences[i].end	long	The end time of the paragraph relative to the start time of the audio. It is a relative timestamp, in milliseconds.
translation.paragraphs.sentences[i].text	string	The translated text, corresponding to the speech recognition result message.

autoChapters (Chapter recognition)

Parameter	Type	Description
autoChapters	list[]	The chapter overview collection, containing zero, one, or more chapter overview information.
autoChapters[i].id	int	The sequence number of the chapter.
autoChapters[i].start	long	The start time of the chapter relative to the start time of the audio. It is a relative timestamp, in milliseconds.
autoChapters[i].end	long	The end time of the chapter relative to the start time of the audio. It is a relative timestamp, in milliseconds.
autoChapters[i].headline	string	The one-sentence title of the chapter.
autoChapters[i].summary	string	The chapter summary.

meetingAssistance (Intelligent minutes extraction, including keyword, key sentence, and to-do item extraction)

Parameter	Type	Description
meetingAssistance	object	The intelligent minutes result object, which may contain zero or more results of different types.
meetingAssistance.keywords	list[]	The keyword extraction results.
meetingAssistance.keySentences	list[]	The key sentence extraction results, also known as key content.
meetingAssistance.keySentences[i].id	long	The key sentence sequence number.
meetingAssistance.keySentences[i].sentenceId	long	The sentence ID corresponding to the key sentence in the original ASR transcription.
meetingAssistance.keySentences[i].start	long	The start time relative to the start time of the audio. It is a relative timestamp, in milliseconds.
meetingAssistance.keySentences[i].end	long	The end time relative to the start time of the audio. It is a relative timestamp, in milliseconds.
meetingAssistance.keySentences[i].text	string	The key sentence information.
meetingAssistance.actions	list[]	The collection of to-do content and to-do summaries.
meetingAssistance.actions[i].id	long	The to-do sequence number.
meetingAssistance.actions[i].sentenceId	long	The sentence ID corresponding to the key sentence in the original ASR transcription.
meetingAssistance.actions[i].start	long	The start time relative to the start time of the audio. It is a relative timestamp, in milliseconds.
meetingAssistance.actions[i].end	long	The end time relative to the start time of the audio. It is a relative timestamp, in milliseconds.
meetingAssistance.actions[i].text	string	The to-do content.
meetingAssistance.classifications	object	The scene classification. Currently, there are only three scene classifications.
meetingAssistance.classifications.interview	float	The confidence score for the interview scene.
meetingAssistance.classifications.lecture	float	The confidence score for the lecture scene.
meetingAssistance.classifications.meeting	float	The confidence score for the meeting scene.

Examples

Success response

JSON format

{
  "JobStatus": "Success",
  "RequestId": "******11-DB8D-4A9A-875B-275798******\n",
  "UserData": "{\"userId\":\"123432412831\"}\n",
  "Results": {
    "result": [
      {
        "Type": "ASR",
        "Data": "{\"title\":\"example-title-****\"}\t\n"
      }
    ]
  },
  "TemplateId": "",
  "Params": "",
  "Input": {
    "Type": "",
    "Media": ""
  }
}

Error codes

See Error Codes for a complete list.

Release notes

See Release Notes for a complete list.