Caption extraction-ApsaraVideo VOD(VOD)-阿里云帮助中心

ApsaraVideo for VOD workflow 2.0 supports subtitle extraction through Intelligent Media Service (IMS). This feature automatically recognizes and extracts hard subtitles from a video and generates a standard SRT file. This topic describes how to use subtitle extraction in a workflow.

Note

This feature is currently available in the following regions:

China (Shanghai), Asia Pacific SE 1 (Singapore), and US West 1 (Silicon Valley).

Prerequisites

To use the subtitle extraction node in a workflow, you must activate ApsaraVideo for VOD and Intelligent Media Service (IMS) and subscribe to the Standard Edition of IMS. For more information, see Subscription-based billing.

Billing

The subtitle extraction feature for ApsaraVideo for VOD workflows is provided by Intelligent Media Service (IMS). To use this feature, you must purchase the IMS subscription service and pay usage fees. For more information, see Intelligent Production.

Usage

Step 1: Configure a subtitle extraction workflow

Currently, workflows can be created only in the ApsaraVideo for VOD console.

Log on to the ApsaraVideo for VOD console.
In the left-side navigation pane, choose Configuration Management > Media Processing > Workflows to open the Workflows page.
Click Add Workflow and enter a name for the workflow.
On the workflow editing page, click the + icon to the right of the Start node and add a Subtitle Extraction node.
In the right-side panel, configure the following parameters:
- Node Name: A custom name for the node.
- Subtitle Language: Chinese, English, and a mix of Chinese and English are supported.
- Frame Rate (fps): A higher frame rate improves recognition accuracy but increases processing time. The valid range is 2 to 10. We recommend 5 for standard videos and 10 for fast-paced videos.
- Sample Material: A sample video used to configure subtitle settings. The workflow does not process this video. MP4, WebM, MOV, and M3U8 formats are supported.
- Recognition Area: The bottom quarter of the video by default. You can also manually select an area.
- Main Subtitle Track Extraction: If enabled, the system extracts only the main subtitle track. This is suitable for TV series and movies. If disabled, the system extracts all subtitles in the specified area. This is suitable for live streams and user-generated content (UGC).
After you complete the configuration, click OK to submit the workflow template. The system generates a workflow ID. Record this ID to use for specifying the workflow during uploads. The created Subtitle Extraction template appears in the workflow management list with a status of Activating. In the Actions column, you can click Details or Deactivate.

Step 2: Trigger the workflow

Use the created workflow to process videos. You can start a workflow job either during or after the video upload.

Console

During upload

Log on to the ApsaraVideo for VOD console.
In the left-side navigation pane, choose Media Files > Audio/Video, and then click Upload Media.
On the Upload Audio/Video page, click to add an audio or video file. In the Add Audio/Video dialog box that appears, set the upload method to Local Upload and click Add Audio/Video to add the files that you want to upload. Select Use Workflow and specify the workflow that you created in the previous step. In the Transcoding Template Group/Workflow Template column, select the target workflow, such as Subtitle Extraction test, and then click Start Upload.

After upload

Log on to the ApsaraVideo for VOD console.
In the left-side navigation pane, choose Media Files > Audio/Video to open the audio and video list.
In the Actions column for the target media file, click Processing. Then, select Process with Workflow and choose the workflow that you created.

API

During upload

The CreateUploadVideo operation is used only to obtain the upload URL and credentials and create basic media asset information. You must implement the file upload logic. For a complete example of an API-based upload, see Upload media files by using ApsaraVideo for VOD APIs.
When you call the CreateUploadVideo or UploadMediaByURL operation to upload a media file, set the WorkflowId parameter to the ID of the workflow you created. After the upload is complete, the system automatically processes the media file using the specified workflow.

After upload

Call the SubmitWorkflowJob operation and set the WorkflowId parameter to the ID of the workflow you created. The system immediately starts to process the media file with the specified workflow.

Step 3: Query the results

Console

Log on to the ApsaraVideo for VOD console.
In the left-side navigation pane, choose Media Files > Short Video Material to open the Short-video materials page.
On the short video materials list page, find the subtitle file generated by the workflow. You can filter the list by source video name, creation time, or other criteria.
After the video processing is complete, you can view the following information:
- The media ID of the subtitle extraction output.
- The storage address of the output file.
- Details such as file size and file URL. You can copy the URL to download the file.

Workflow callbacks

Configure HTTP or Lightweight Message Queue (formerly MNS) callbacks.
When a workflow job is complete, the system triggers a workflow task completion event and sends the full results through the configured callback. The key fields in the callback message are described below.
1. Status: The job status. Valid values are Succeed and Failed.
2. ActivityResults: A JSON string that contains execution details for each node, such as the SubtitleExtraction node. The Result field contains key information about the output, such as MediaId and JobId.
3. TaskInput: The original input media information, such as the Media ID and filename.
Parse the ActivityResults field in the callback message to extract the MediaId of the generated subtitle file for subsequent playback or distribution.

API

The workflow submits a subtitle extraction job by calling the SubmitAIJob operation. This operation returns a job ID. Use this job ID with the GetAIJob operation to check the job status.

Request parameters:

JobId: The job ID that is returned when the workflow internally calls the SubmitAIJob operation to submit the subtitle extraction job.

Sample response:

{
  "createTime": "2024-09-24T06:17:09Z",
  "finishTime": "2024-09-24T06:17:31Z",
  "functionName": "CaptionExtraction",
  "input": {
    "type": "OSS",
    "media": "oss://example-bucket/example-object.mp4"
  },
  "jobId": "270d****0d1a",
  "jobParams": "{\"fps\":10,\"roi\":[[0.5,1],[0,1]],\"sep\":false}",
  "output": {
    "type": "OSS",
    "media": "oss://example-bucket/output.srt"
  },
  "outputFiles": [
    "output.srt"
  ],
  "outputUrls": [
    "http://example-bucket.oss-region.aliyuncs.com/output.srt"
  ],
  "requestId": "********-****-****-****-************",
  "result": "{}",
  "status": "Success"
}

Note

A Status of Success indicates that the job is complete.
You can use the URL in OutputUrls to download the extracted SRT file.