Video summarization

更新时间:
复制 MD 格式

This service analyzes the semantics of video content to automatically generate a title, summary, and tags. It also provides an enhanced transcript and metadata for each video segment, which is useful for video content understanding, automated tagging, and multimodal RAG.

Service name

Service ID

Description

QPS limit

video summarization

ops-video-summarize-001

Performs semantic analysis on a video to generate a title, summary, and tags, and outputs an enhanced transcript and metadata for video segments.

5

Note

To increase your QPS limit, submit a ticket to technical support.

Prerequisites

  • Obtain authentication credentials

    API calls to the OpenSearch AI-Native Search Platform service require identity authentication. For more information, see Get your API-KEY.

  • Obtain the service endpoint

    You can call the service over the public network or through a VPC. For details, see Get service endpoints.

Request

General notes

  • The video summarization service supports only asynchronous calls. You must first create an asynchronous task and then query the result by using the task ID.

  • The request body cannot exceed 8 MB.

Create an asynchronous task

Request method

POST

URL

{host}/v3/openapi/workspaces/{workspace_name}/video-summarization/{service_id}/async

Parameters:

  • host: The service endpoint. You can call the API service over the public network or through a VPC. For more information, see Obtain a service endpoint.

  • workspace_name: The name of your workspace, for example, default.

  • service_id: The built-in service ID. Currently, only ops-video-summarize-001 is supported.

Get the status of an asynchronous task

Request method

GET

URL

{host}/v3/openapi/workspaces/{workspace_name}/video-summarization/{service_id}/async/task-status?task_id={task_id}

Parameters:

  • host: The service endpoint.

  • workspace_name: The name of your workspace, for example, default.

  • service_id: The built-in service ID. Currently, only ops-video-summarize-001 is supported.

  • task_id: The ID of the task to query. This ID is returned when you create the task.

Request parameters

Header parameters

API-KEY authentication

Parameter

Type

Required

Description

Example

Content-Type

String

Yes

The content type of the request. This must be application/json.

application/json

Authorization

String

Yes

Your API-KEY.

Bearer OS-d1**2a

Body parameters (create task)

Parameter

Type

Required

Description

Example

input

InputObject

Yes

The video file to process.

parameters

ParametersObject

No

Parameters that control the summarization behavior.

output

OutputObject

Yes

The output configuration for the result.

InputObject

Parameter

Type

Required

Description

Example

oss

String

No

The OSS path of the input file. Required if url is not specified.

oss://bucket-name/example/test.mp4

url

String

No

The HTTP URL of the input file. Required if oss is not specified.

http://path/to/video.mp4

file_name

String

No

The name of the video file. If not specified, the name is parsed from the file path.

test.mp4

chunks

List<VideoChunk>

No

A list of video segments to summarize. If not specified, the entire video is summarized.

VideoChunk

Parameter

Type

Required

Description

Example

start_time

Float

Yes

The start time of the segment, in seconds.

0

end_time

Float

Yes

The end time of the segment, in seconds.

10

snapshots

List<Snapshot>

No

A list of snapshots for the segment.

transcript

String

No

The transcript for the segment.

Snapshot

Parameter

Type

Required

Description

Example

frame_index

Int

No

The index of the snapshot in the current segment, starting from 0.

0

path

String

No

The OSS path where the snapshot image is stored.

oss://bucket-name/frame0.jpg

frame_time

Float

Yes

The absolute time of the snapshot in the video, in seconds.

5.0

ParametersObject

Parameter

Type

Required

Description

Example

rewrite_transcript

Boolean

No

Specifies whether to rewrite the Automatic Speech Recognition (ASR) result to remove noise and add punctuation. The default value is false.

true

generate_tags

Boolean

No

Specifies whether to extract video tags. The default value is false.

true

extract_snapshot

Boolean

No

Specifies whether to extract snapshots. The default value is false.

true

OutputObject

Parameter

Type

Required

Description

Example

type

String

No

The output type. Defaults to oss, which stores the result in OSS.

oss

oss

String

Yes

The OSS path for the output file. This parameter is required when type is set to oss.

oss://bucket-name/dump/result/path

Query parameters (get task status)

Parameter

Type

Required

Description

Example

task_id

String

Yes

The unique ID of the asynchronous task. This ID is returned when you create the task.

summarization-xxxx-abc-123

Response parameters

Create an asynchronous task

Parameter

Type

Description

Example

request_id

String

The unique system-assigned identifier for the API call.

A5B25952-4406-45BF-99EC-E8020246****

latency

Float/Int

The request latency, in milliseconds (ms).

10

result.task_id

String

The unique identifier of the asynchronous task.

summarization-xxxx-abc-123

Get the status of an asynchronous task

Parameter

Type

Description

Example

request_id

String

The unique system-assigned identifier for the API call.

A5B25952-4406-45BF-99EC-E8020246****

latency

Float/Int

The request latency, in milliseconds (ms).

10

usage.image_token

Int

The number of image tokens consumed.

100

usage.audio_token

Int

The number of audio tokens consumed.

200

usage.input_token

Int

The number of input tokens used by the large model.

300

usage.output_token

Int

The number of tokens generated by the large model.

300

result.task_id

String

The unique identifier of the asynchronous task.

summarization-xxxx-abc-123

result.status

String

The task status. Valid values: PENDING, SUCCESS, and FAIL.

SUCCESS

result.error

String

The error message if the task fails. This field is empty on success.

result.data

VideoSummarizeData

The video summarization result.

VideoSummarizeData

Parameter

Type

Description

video_metadata

VideoMetaData

The overall metadata for the video.

chunks

List<VideoSummary>

The summarization results for each video segment.

VideoMetaData

Parameter

Type

Description

title

String

The title of the video or segment.

summary

String

The summary of the video or segment.

tags

List<String>

A list of extracted video tags. You must set generate_tags to true in the request.

VideoSummary

Parameter

Type

Description

index

Int

The index of the segment in the request, starting from 0.

enhanced_transcript

String

Enhanced ASR transcript. You need to set rewrite_transcript to true in the request.

metadata

VideoMetaData

The metadata for this video segment.

cURL examples

Create an asynchronous task

curl --location 'http://****-hangzhou.opensearch.aliyuncs.com/v3/openapi/workspaces/default/video-summarization/ops-video-summarize-001/async' \
--header 'Authorization: Bearer YOUR_API_KEY' \
--header 'Content-Type: application/json' \
--data '{
  "input": {
    "oss": "oss://my-bucket-name/example/test.mp4"
  },
  "parameters": {
    "rewrite_transcript": true,
    "generate_tags": true,
    "extract_snapshot": true
  },
  "output": {
    "type": "oss",
    "oss": "oss://my-bucket-name/dump/result/path"
  }
}'

Get the status of an asynchronous task

curl --location 'http://****-hangzhou.opensearch.aliyuncs.com/v3/openapi/workspaces/default/video-summarization/ops-video-summarize-001/async/task-status?task_id=summarization-xxxx-abc-123' \
--header 'Authorization: Bearer YOUR_API_KEY'

Example responses

Create an asynchronous task

{
  "request_id": "f4a95b66efd25e6c94d0a1861cd3cf88",
  "latency": 39,
  "usage": {},
  "result": {
    "task_id": "video_summarize-2026031310xxxx-xxxxxxxxxxxxxxxx-e19ee37c",
    "status": "PENDING"
  }
}

Get task status (completed)

{
  "request_id": "1",
  "latency": 0,
  "usage": {
    "audio_token": 200,
    "image_token": 100,
    "input_token": 300,
    "output_token": 300
  },
  "result": {
    "task_id": "test-summary-001",
    "status": "SUCCESS",
    "data": {
      "video_metadata": {
        "title": "Video Title",
        "summary": "This is a summary of a product introduction video.",
        "tags": [
          "Product Introduction",
          "Technology"
        ]
      },
      "chunks": [
        {
          "index": 0,
          "enhanced_transcript": "This is the enhanced transcript text.",
          "metadata": {
            "title": "Title of the First Segment",
            "summary": "Summary of the first segment.",
            "tags": [
              "Segment Tag 1",
              "Segment Tag 2"
            ]
          }
        }
      ]
    }
  }
}

Status codes

See Status Code Description for the OpenSearch AI-Native Search Platform.