QuerySmarttagJob-Intelligent Media Services(IMS)-阿里云帮助中心

Queries smart tagging jobs.

Try it now

Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage.

Test

RAM authorization

The table below describes the authorization required to call this API. You can define it in a Resource Access Management (RAM) policy. The table's columns are detailed below:

Action: The actions can be used in the Action element of RAM permission policy statements to grant permissions to perform the operation.
API: The API that you can call to perform the action.
Access level: The predefined level of access granted for each API. Valid values: create, list, get, update, and delete.
Resource type: The type of the resource that supports authorization to perform the action. It indicates if the action supports resource-level permission. The specified resource must be compatible with the action. Otherwise, the policy will be ineffective.
- For APIs with resource-level permissions, required resource types are marked with an asterisk (*). Specify the corresponding Alibaba Cloud Resource Name (ARN) in the Resource element of the policy.
- For APIs without resource-level permissions, it is shown as All Resources. Use an asterisk (*) in the Resource element of the policy.
Condition key: The condition keys defined by the service. The key allows for granular control, applying to either actions alone or actions associated with specific resources. In addition to service-specific condition keys, Alibaba Cloud provides a set of common condition keys applicable across all RAM-supported services.
Dependent action: The dependent actions required to run the action. To complete the action, the RAM user or the RAM role must have the permissions to perform all dependent actions.

Action

Access level

Resource type

Condition key

Dependent action

ice:QuerySmarttagJob

get

*All Resource

*

None

Request parameters

Parameter	Type	Required	Description	Example
JobId	string	Yes	The ID of the smart tagging job to query. You can obtain the ID from the response parameters of Submit a smart tagging job.	88c6ca184c0e47098a5b665e2****
Params	string	No	The additional request parameters, represented as a JSON string. Example: {"labelResultType":"auto"}. The labelResultType parameter is of the String type. Valid values: auto: Machine tagging results. hmi: Human-machine collaboration results.	{"labelResultType":"auto"}

Response elements

Element	Type	Description	Example
	object
JobStatus	string	The job status. Valid values: Success: The job is processed successfully. Fail: The job failed to be processed. Processing: The job is being processed. Submitted: The job is submitted and waiting to be processed.	Success
RequestId	string	The request ID.	****11-DB8D-4A9A-875B-275798****
UserData	string	The information passed through during an MNS callback. For the message format, see the callback message format definition below.	{"userId":"123432412831"}
Results	object
Result	array<object>	The array of analysis result objects.
	object	The smart tagging analysis results.
Type	string	The type of the analysis result. - Smart tag 1.0 analysis result types: TextLabel: Text tags. VideoLabel: Video tags. ASR: Original automatic speech recognition (ASR) results. Not returned by default. OCR: Original optical character recognition (OCR) results. Not returned by default. NLP: Natural language processing (NLP) results. Not returned by default. - Smart tag 2.0 analysis result types: CPVLabel Meta: Information such as video titles. Not returned by default. - Smart tag 2.0-custom analysis result types: CPVLabel Meta: Information such as video titles. Not returned by default.	Meta
Data	string	The specific data of the analysis result, represented as a JSON string. For the data structures of different types, see the Result parameter description below.	{"title":"example-title-****"}
Usages	object
Usage	array<object>	The list of resource usage for the job.
	object	The resource usage information for the job.
Type	string
Quota	integer

Callback message format definition When the execution status of a smart tagging job changes, MPS sends a message to the queue specified by the user. For more information about how to specify a callback queue, see the UpdatePipeline operation of MPS. The message body is a JSON string that contains the following fields:

Name	Type	Description
Type	String	The fixed string "smarttag", which indicates a smart tagging job.
JobId	String	The unique ID of the job.
State	String	The current status of the job, which is the same as JobStatus in QuerySmarttagJob.
UserData	String	The UserData passed by the user in SubmitSmarttagJob.

Result parameter description

VideoLabel data structure

Name	Type	Description
persons	JSONArray	The detected person results.
persons.name	String	The name of the recognized person.
persons.category	String	The person type. Valid values: `celebrity` (celebrity); `politician` (politician); `sensitive` (sensitive person); `unknown` (unknown person); custom person library ID (returned when the recognized person is a custom person).
persons.ratio	double	The appearance rate of the person. Valid values: 0 to 1.
persons.occurrences	JSONArray	The detailed information about the person's appearances.
persons.occurrences.score	double	The confidence score.
persons.occurrences.from	double	The start time when the person is detected, in seconds.
persons.occurrences.to	double	The end time when the person is detected, in seconds.
persons.occurrences.position	JSONObject	The face coordinates.
persons.occurrences.position.leftTop	int[]	The x and y coordinates of the upper-left corner.
persons.occurrences.position.rightBottom	int[]	The x and y coordinates of the lower-right corner.
persons.occurrences.timestamp	double	The timestamp of the face coordinates, in seconds.
persons.occurrences.scene	String	The person shot. Valid values: `closeUp` (close-up); `medium-closeUp` (medium close-up); `medium` (medium shot); `medium-long` (full shot).
tags	JSONArray	The detected tags for objects and scenes. For examples, see the following table.
tags.mainTagName	String	The main tag.
tags.subTagName	String	The sub-tag.
tags.ratio	double	The appearance rate. Valid values: 0 to 1.
tags.occurrences	JSONArray	The detailed information about the appearances.
tags.occurrences.score	double	The confidence score.
tags.occurrences.from	double	The start time, in seconds.
tags.occurrences.to	double	The end time, in seconds.
classifications	JSONArray	The video classification information.
classifications.score	double	The confidence score.
classifications.category1	String	The level-1 category, such as life, anime, and automobile.
classifications.category2	String	The level-2 category, such as health and home under the life category.

Examples of video tags

mainTagName	subTagName
Program	For example, Where Are We Going, Dad? and Happy Comedian.
Character role	For example, doctor, nurse, and teacher.
Object	For example, piano, water cup, desk, scrambled eggs with tomatoes, car, and cosmetics.
Logo	For example, CCTV1, CCTV2, Youku, and Dragon TV.
Action and event	For example, dancing, kissing, hugging, meeting, singing, making a phone call, horse riding, and fighting.
Region	For example, Tiananmen Square in Beijing, Statue of Liberty, Leshan Giant Buddha, China, and the United States.
Scene	For example, bedroom, subway station, terraced field, beach, and desert.

ImageLabel data structure

Name	Type	Description
persons	JSONArray	The detected person results.
persons.name	String	The name of the recognized person.
persons.category	String	The person type. Valid values: celebrity (celebrity); politician (politician); sensitive (sensitive person).
persons.score	double	The confidence score of the person.
persons.position	JSONObject	The face coordinates.
persons.position.leftTop	int[]	The x and y coordinates of the upper-left corner.
persons.position.rightBottom	int[]	The x and y coordinates of the lower-right corner.
persons.scene	String	The person shot. Valid values: closeUp (close-up); medium-closeUp (medium close-up); medium (medium shot); medium-long (full shot).
tags	JSONArray	The detected tags for objects and scenes. For examples, see the following table.
tags.mainTagName	String	The main tag.
tags.subTagName	String	The sub-tag.
tags.score	double	The confidence score.

Examples of image tags

mainTagName	subTagName
Character role	For example, doctor, nurse, and teacher.
Region	For example, Tiananmen Square in Beijing, Statue of Liberty, Leshan Giant Buddha, China, and the United States.
Action and event	For example, speaking.
Logo	For example, CCTV1, CCTV2, Youku, and Dragon TV.
Action and event	For example, dancing, kissing, hugging, meeting, singing, making a phone call, horse riding, and fighting.
Object	For example, piano, water cup, desk, scrambled eggs with tomatoes, car, and cosmetics.
Scene	For example, bedroom, subway station, terraced field, beach, and desert.

TextLabel data structure (derived from ASR and OCR)

Name	Type	Description
tags	JSONArray	The text tag information. For examples, see the following table.
tags.name	String	The tag category.
tags.value	String	The tag value. Multiple tag values are separated by commas (,).

Examples of text tags

name	value
Region	For example, Tiananmen Square in Beijing, Statue of Liberty, Leshan Giant Buddha, China, and the United States.
Organization	For example, China Wildlife Conservation Association and China Media Group.
Logo	For example, Nike and Li-Ning.
Keyword	For example, core force.

CPVLabel data structure

cates: categories (level-1, level-2, and level-3 categories).
entities: category attributes (with knowledge graph information).
hotwords: hot words (list of words followed by users).
freeTags: free tags (keywords).

Name	Type	Example	Description
type	String	hmi	The result type. Valid values: hmi (human-machine collaboration results) and autp (machine tagging results).
cates	JSONArray	-	The classification category results.
cates.labelLevel1	String	Tourism	The level-1 tag.
cates.labelLevel2	String	Tourism scenery	The level-2 tag.
cates.label	String	""	The tag name. The algorithm may return an empty string.
cates.appearanceProbability	double	0.96	The appearance rate.
cates.detailInfo	JSONArray	-	-
cates.detailInfo.score	double	0.9	The confidence score.
cates.detailInfo.startTime	double	0.021	The start time.
cates.detailInfo.endTime	double	29.021	The end time.
entities	JSONArray	-	-
entities.labelLevel1	String	Region	The level-1 tag.
entities.labelLevel2	String	Landmark	The level-2 tag.
entities.label	String	Huangguoshu Waterfall	The tag name.
entities.appearanceProbability	double	0.067	The appearance rate.
entities.knowledgeInfo	String	{"name": "Huangguoshu Waterfall", "nameEn": "Huangguoshu Waterfall", "description": "One of the four major waterfalls in Asia"}	The knowledge graph information. For all fields, see the attached tables: Film and Variety IP Graph, Music Graph, Person Graph, Landmark Graph, and Object Graph.
entities.detailInfo	JSONArray	-	-
entities.detailInfo.score	double	0.33292606472969055	The confidence score.
entities.detailInfo.startTime	double	6.021	The start time.
entities.detailInfo.endTime	double	8.021	The end time.
entities.detailInfo.trackData	JSONArray	-	The structured information of the entity tag.
entities.detailInfo.trackData.score	double	0.32	The confidence score.
entities.detailInfo.trackData.bbox	integer[]	23, 43, 45, 67	The bounding box.
entities.detailInfo.trackData.timestamp	double	7.9	The timestamp.
hotwords	JSONArray	-	-
hotwords.labelLevel1	String	Keyword	The level-1 tag.
hotwords.labelLevel1	String	Keyword	The level-2 tag.
hotwords.labelLevel2	String	""	The level-2 tag.
hotwords.label	String	China Meteorological Administration	The highlight word content.
hotwords.appearanceProbability	double	0.96	The appearance rate.
hotwords.detailInfo	JSONArray
hotwords.detailInfo.score	double	1.0	The confidence score.
hotwords.detailInfo.startTime	double	0.021	The start time.
hotwords.detailInfo.endTime	double	29.021	The end time.
freeTags	JSONArray
freeTags.labelLevel1	String	Keyword	The level-1 tag.
freeTags.labelLevel2	String	""	The level-2 tag.
freeTags.label	String	National Meteorological Center	The highlight word content.
freeTags.appearanceProbability	double	0.96	The appearance rate.
freeTags.detailInfo	JSONArray
freeTags.detailInfo.score	double	0.9	The confidence score.
freeTags.detailInfo.startTime	double	0.021	The start time.
freeTags.detailInfo.endTime	double	29.021	The end time.

Speech recognition ASR results

Name	Type	Description
details	JSONArray	The detailed results of the node.
details.from	double	The start timestamp, in seconds.
details.to	double	The end timestamp, in seconds.
details.content	String	The recognized text content.

OCR results

Name	Type	Description
details	JSONArray	The detailed results of the job.
details.timestamp	double	The timestamp information, in seconds.
details.info	JSONArray	The specific information recognized at the timestamp.
details.info.score	double	The confidence score.
details.info.position	JSONObject	The text coordinates.
details.info.position.leftTop	int[]	The x and y coordinates of the upper-left corner.
details.info.position.rightBottom	int[]	The x and y coordinates of the lower-right corner.
details.info.content	String	The recognized text content.

Meta tagging results

Note

If you do not use human-machine tagging and specify needMetaData when submitting a job by calling SubmitSmarttagJob, QuerySmarttagJob returns the original title entered by the user.

Name	Type	Description
title	String	The title.

Subtitle extraction results

Name	Type	Description
details	JSONArray	The detailed results of the job.
details.allResultUrl	String	The URL of all subtitle results. The URL is valid for half a year after the job is completed.
details.chResultUrl	String	The URL of the Chinese results. The URL is valid for half a year after the job is completed.
details.engResultUrl	String	The URL of the English results. The URL is valid for half a year after the job is completed.

Note

Subtitle result URL content: Sequence number + Time period + Subtitle content (each subtitle sentence is on a separate line).

NLP results

Parameter	Type	Description
transcription	object	The speech transcription results.
autoChapters	object	The chapter overview results.
summarization	object	The large model summarization results.
meetingAssistance	object	The smart meeting minutes results.
translation	object	The text translation results.

transcription (Speech transcription)

Parameter	Type	Description
transcription	object	The speech transcription result object.
transcription.paragraphs	list[]	The collection of speech transcription structures organized in paragraphs.
transcription.paragraphs[i].paragraphId	string	The paragraph-level ID.
transcription.paragraphs[i].speakerId	string	The speaker ID.
transcription.paragraphs[i].words	list[]	The word information contained in the paragraph.
transcription.paragraphs[i].words[i].id	int	The word sequence number, which usually does not need attention.
transcription.paragraphs[i].words[i].sentenceId	int	The sentence ID. Word information belonging to the same SentenceId can be assembled into a sentence.
transcription.paragraphs[i].words[i].start	long	The start time of the word relative to the start time of the audio. It is a relative timestamp, in milliseconds.
transcription.paragraphs[i].words[i].end	long	The end time of the word relative to the start time of the audio. It is a relative timestamp, in milliseconds.
transcription.paragraphs[i].words[i].text	string	The word text.

summarization (Full-text summary, speaker summary, and Q&A summary)

Parameter	Type	Description
summarization	object	The summary result object, which may contain zero or more results of different summary types.
summarization.paragraphSummary	string	The full-text summary result.
summarization.conversationalSummary	list[]	The list of speaker summary results.
summarization.conversationalSummary[i].speakerId	string	The speaker ID.
summarization.conversationalSummary[i].speakerName	string	The speaker name.
summarization.conversationalSummary[i].summary	string	The summary corresponding to the speaker.
summarization.questionsAnsweringSummary	list[]	The list of Q&A summary results.
summarization.questionsAnsweringSummary[i].question	string	The question.
summarization.questionsAnsweringSummary[i].sentenceIdsOfQuestion	list[]	The list of SentenceIds of the original speech transcription from which the question is extracted.
summarization.questionsAnsweringSummary[i].answer	string	The answer to the question.
summarization.questionsAnsweringSummary[i].sentenceIdsOfAnswer	list[]	The list of SentenceIds of the original speech transcription from which the answer is summarized.
summarization.mindMapSummary	list[object]	The list of mind map summary results, which may contain summaries of various topics and relationships between topics.
summarization.mindMapSummary[i].title	string	The title of the topic.
summarization.mindMapSummary[i].topic	list[object]	The array containing each topic and its subtopics.
summarization.mindMapSummary[i].topic[i].title	string	The title of the topic.
summarization.mindMapSummary[i].topic[i].topic	list[object]	The array of subtopics of the topic, which can be an empty array.

translation (Full-text translation)

Parameter	Type	Description
translation	object	The translation result object.
translation.paragraphs	list[]	The collection of translation results organized in paragraphs, corresponding to the speech recognition result message.
translation.paragraphs.paragraphId	string	The paragraph segmentation ID, corresponding to the ParagraphId in the speech recognition results.
translation.paragraphs.sentences	list[]	The collection of translated texts.
translation.paragraphs.sentences[i].sentenctId	long	The sentence ID.
translation.paragraphs.sentences[i].start	long	The start time of the paragraph relative to the start time of the audio. It is a relative timestamp, in milliseconds.
translation.paragraphs.sentences[i].end	long	The end time of the paragraph relative to the start time of the audio. It is a relative timestamp, in milliseconds.
translation.paragraphs.sentences[i].text	string	The translated text, corresponding to the speech recognition result message.

autoChapters (Chapter recognition)

Parameter	Type	Description
autoChapters	list[]	The chapter overview collection, containing zero, one, or more chapter overview information.
autoChapters[i].id	int	The sequence number of the chapter.
autoChapters[i].start	long	The start time of the chapter relative to the start time of the audio. It is a relative timestamp, in milliseconds.
autoChapters[i].end	long	The end time of the chapter relative to the start time of the audio. It is a relative timestamp, in milliseconds.
autoChapters[i].headline	string	The one-sentence title of the chapter.
autoChapters[i].summary	string	The chapter summary.

meetingAssistance (Smart meeting minutes extraction, including keywords, key sentences, and to-do items)

Parameter	Type	Description
meetingAssistance	object	The smart meeting minutes result object, which may contain zero or more results of different types.
meetingAssistance.keywords	list[]	The keyword extraction results.
meetingAssistance.keySentences	list[]	The key sentence extraction results, also known as key content.
meetingAssistance.keySentences[i].id	long	The sequence number of the key sentence.
meetingAssistance.keySentences[i].sentenceId	long	The sentence ID corresponding to the key sentence in the original ASR transcription.
meetingAssistance.keySentences[i].start	long	The start time relative to the start time of the audio. It is a relative timestamp, in milliseconds.
meetingAssistance.keySentences[i].end	long	The end time relative to the start time of the audio. It is a relative timestamp, in milliseconds.
meetingAssistance.keySentences[i].text	string	The key sentence information.
meetingAssistance.actions	list[]	The collection of to-do content and to-do summaries.
meetingAssistance.actions[i].id	long	The to-do sequence number.
meetingAssistance.actions[i].sentenceId	long	The sentence ID corresponding to the key sentence in the original ASR transcription.
meetingAssistance.actions[i].start	long	The start time relative to the start time of the audio. It is a relative timestamp, in milliseconds.
meetingAssistance.actions[i].end	long	The end time relative to the start time of the audio. It is a relative timestamp, in milliseconds.
meetingAssistance.actions[i].text	string	The to-do content.
meetingAssistance.classifications	object	The scene classification. Currently, there are only three scene classifications.
meetingAssistance.classifications.interview	float	The confidence score for the interview scene.
meetingAssistance.classifications.lecture	float	The confidence score for the lecture scene.
meetingAssistance.classifications.meeting	float	The confidence score for the meeting scene.

Examples

Success response

JSON format

{
  "JobStatus": "Success",
  "RequestId": "******11-DB8D-4A9A-875B-275798******",
  "UserData": "{\"userId\":\"123432412831\"}",
  "Results": {
    "Result": [
      {
        "Type": "Meta",
        "Data": "{\"title\":\"example-title-****\"}\t\n"
      }
    ]
  },
  "Usages": {
    "Usage": [
      {
        "Type": "",
        "Quota": 0
      }
    ]
  }
}

Error codes

See Error Codes for a complete list.

Release notes

See Release Notes for a complete list.