This service analyzes the semantics of video content to automatically generate a title, summary, and tags. It also provides an enhanced transcript and metadata for each video segment, which is useful for video content understanding, automated tagging, and multimodal RAG.
Service name | Service ID | Description | QPS limit |
video summarization | ops-video-summarize-001 | Performs semantic analysis on a video to generate a title, summary, and tags, and outputs an enhanced transcript and metadata for video segments. | 5 Note To increase your QPS limit, submit a ticket to technical support. |
Prerequisites
Obtain authentication credentials
API calls to the OpenSearch AI-Native Search Platform service require identity authentication. For more information, see Get your API-KEY.
Obtain the service endpoint
You can call the service over the public network or through a VPC. For details, see Get service endpoints.
Request
General notes
The video summarization service supports only asynchronous calls. You must first create an asynchronous task and then query the result by using the task ID.
The request body cannot exceed 8 MB.
Create an asynchronous task
Request method
POST
URL
{host}/v3/openapi/workspaces/{workspace_name}/video-summarization/{service_id}/asyncParameters:
host: The service endpoint. You can call the API service over the public network or through a VPC. For more information, see Obtain a service endpoint.
workspace_name: The name of your workspace, for example,
default.service_id: The built-in service ID. Currently, only
ops-video-summarize-001is supported.
Get the status of an asynchronous task
Request method
GET
URL
{host}/v3/openapi/workspaces/{workspace_name}/video-summarization/{service_id}/async/task-status?task_id={task_id}Parameters:
host: The service endpoint.
workspace_name: The name of your workspace, for example,
default.service_id: The built-in service ID. Currently, only
ops-video-summarize-001is supported.task_id: The ID of the task to query. This ID is returned when you create the task.
Request parameters
Header parameters
API-KEY authentication
Parameter | Type | Required | Description | Example |
Content-Type | String | Yes | The content type of the request. This must be | application/json |
Authorization | String | Yes | Your API-KEY. | Bearer OS-d1**2a |
Body parameters (create task)
Parameter | Type | Required | Description | Example |
input | InputObject | Yes | The video file to process. | |
parameters | ParametersObject | No | Parameters that control the summarization behavior. | |
output | OutputObject | Yes | The output configuration for the result. |
InputObject
Parameter | Type | Required | Description | Example |
oss | String | No | The OSS path of the input file. Required if | oss://bucket-name/example/test.mp4 |
url | String | No | The HTTP URL of the input file. Required if | http://path/to/video.mp4 |
file_name | String | No | The name of the video file. If not specified, the name is parsed from the file path. | test.mp4 |
chunks | List<VideoChunk> | No | A list of video segments to summarize. If not specified, the entire video is summarized. |
VideoChunk
Parameter | Type | Required | Description | Example |
start_time | Float | Yes | The start time of the segment, in seconds. | 0 |
end_time | Float | Yes | The end time of the segment, in seconds. | 10 |
snapshots | List<Snapshot> | No | A list of snapshots for the segment. | |
transcript | String | No | The transcript for the segment. |
Snapshot
Parameter | Type | Required | Description | Example |
frame_index | Int | No | The index of the snapshot in the current segment, starting from 0. | 0 |
path | String | No | The OSS path where the snapshot image is stored. | oss://bucket-name/frame0.jpg |
frame_time | Float | Yes | The absolute time of the snapshot in the video, in seconds. | 5.0 |
ParametersObject
Parameter | Type | Required | Description | Example |
rewrite_transcript | Boolean | No | Specifies whether to rewrite the Automatic Speech Recognition (ASR) result to remove noise and add punctuation. The default value is | true |
generate_tags | Boolean | No | Specifies whether to extract video tags. The default value is | true |
extract_snapshot | Boolean | No | Specifies whether to extract snapshots. The default value is | true |
OutputObject
Parameter | Type | Required | Description | Example |
type | String | No | The output type. Defaults to | oss |
oss | String | Yes | The OSS path for the output file. This parameter is required when | oss://bucket-name/dump/result/path |
Query parameters (get task status)
Parameter | Type | Required | Description | Example |
task_id | String | Yes | The unique ID of the asynchronous task. This ID is returned when you create the task. | summarization-xxxx-abc-123 |
Response parameters
Create an asynchronous task
Parameter | Type | Description | Example |
request_id | String | The unique system-assigned identifier for the API call. | A5B25952-4406-45BF-99EC-E8020246**** |
latency | Float/Int | The request latency, in milliseconds (ms). | 10 |
result.task_id | String | The unique identifier of the asynchronous task. | summarization-xxxx-abc-123 |
Get the status of an asynchronous task
Parameter | Type | Description | Example |
request_id | String | The unique system-assigned identifier for the API call. | A5B25952-4406-45BF-99EC-E8020246**** |
latency | Float/Int | The request latency, in milliseconds (ms). | 10 |
usage.image_token | Int | The number of image tokens consumed. | 100 |
usage.audio_token | Int | The number of audio tokens consumed. | 200 |
usage.input_token | Int | The number of input tokens used by the large model. | 300 |
usage.output_token | Int | The number of tokens generated by the large model. | 300 |
result.task_id | String | The unique identifier of the asynchronous task. | summarization-xxxx-abc-123 |
result.status | String | The task status. Valid values: | SUCCESS |
result.error | String | The error message if the task fails. This field is empty on success. | |
result.data | VideoSummarizeData | The video summarization result. |
VideoSummarizeData
Parameter | Type | Description |
video_metadata | VideoMetaData | The overall metadata for the video. |
chunks | List<VideoSummary> | The summarization results for each video segment. |
VideoMetaData
Parameter | Type | Description |
title | String | The title of the video or segment. |
summary | String | The summary of the video or segment. |
tags | List<String> | A list of extracted video tags. You must set |
VideoSummary
Parameter | Type | Description |
index | Int | The index of the segment in the request, starting from 0. |
enhanced_transcript | String | Enhanced ASR transcript. You need to set |
metadata | VideoMetaData | The metadata for this video segment. |
cURL examples
Create an asynchronous task
curl --location 'http://****-hangzhou.opensearch.aliyuncs.com/v3/openapi/workspaces/default/video-summarization/ops-video-summarize-001/async' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--header 'Content-Type: application/json' \
--data '{
"input": {
"oss": "oss://my-bucket-name/example/test.mp4"
},
"parameters": {
"rewrite_transcript": true,
"generate_tags": true,
"extract_snapshot": true
},
"output": {
"type": "oss",
"oss": "oss://my-bucket-name/dump/result/path"
}
}'Get the status of an asynchronous task
curl --location 'http://****-hangzhou.opensearch.aliyuncs.com/v3/openapi/workspaces/default/video-summarization/ops-video-summarize-001/async/task-status?task_id=summarization-xxxx-abc-123' \
--header 'Authorization: Bearer YOUR_API_KEY'Example responses
Create an asynchronous task
{
"request_id": "f4a95b66efd25e6c94d0a1861cd3cf88",
"latency": 39,
"usage": {},
"result": {
"task_id": "video_summarize-2026031310xxxx-xxxxxxxxxxxxxxxx-e19ee37c",
"status": "PENDING"
}
}Get task status (completed)
{
"request_id": "1",
"latency": 0,
"usage": {
"audio_token": 200,
"image_token": 100,
"input_token": 300,
"output_token": 300
},
"result": {
"task_id": "test-summary-001",
"status": "SUCCESS",
"data": {
"video_metadata": {
"title": "Video Title",
"summary": "This is a summary of a product introduction video.",
"tags": [
"Product Introduction",
"Technology"
]
},
"chunks": [
{
"index": 0,
"enhanced_transcript": "This is the enhanced transcript text.",
"metadata": {
"title": "Title of the First Segment",
"summary": "Summary of the first segment.",
"tags": [
"Segment Tag 1",
"Segment Tag 2"
]
}
}
]
}
}
}Status codes
See Status Code Description for the OpenSearch AI-Native Search Platform.