Subtitle removal
ApsaraVideo VOD workflow 2.0 integrates the subtitle removal feature from Intelligent Media Services (IMS). It identifies and removes subtitles from videos, then restores the background using smart inpainting for automated post-production.
This feature is available in the following regions:
-
China (Shanghai), China (Beijing), Asia Pacific SE 1 (Singapore), and US West 1 (Silicon Valley).
Prerequisites
-
To use the subtitle removal node in a workflow, you must enable ApsaraVideo VOD and Intelligent Media Services (IMS) and subscribe to the standard edition. For more information, see subscription-based billing.
Benefits
Two versions of subtitle removal are available:
-
Advanced (Recommended): Seamlessly removes subtitles and resolves mosaic and shadow artifacts for a more natural-looking video. To enable this version, use the
ModelIdparameter. For more information, see ModelId parameter. -
The advanced version uses a superior algorithm, but it processes videos more slowly and costs more than the basic version. Choose the version that best fits your business needs.
Billing
Subtitle removal in ApsaraVideo VOD workflows is powered by Intelligent Media Services (IMS). You must purchase an IMS subscription service and pay usage fees. For more information, see Subtitle removal.
Usage
Step 1: Configure the workflow
You can create workflows only in the ApsaraVideo VOD console.
-
Log on to the ApsaraVideo VOD console.
-
In the left-side navigation pane, choose Configuration Management > Media Processing > Workflows.
-
Click Add Workflow Template and enter a name for the workflow.
-
On the workflow editor page, click the + icon to the right of the Start node to add a subtitle removal node.
-
Configure the following parameters in the right-side panel:
-
Node Name: Enter a custom name for the subtitle removal node.
-
Sample Material: A sample video used to configure subtitle settings. This video is not processed by the workflow. Supported formats are MP4, WebM, MOV, and M3U8.
-
Recognition Area: The default area is the bottom 1/4 of the video. You can also manually select an area.
-
Algorithm Version: Two versions are available: Basic and Advanced.
-
Advanced (Recommended): Seamlessly removes subtitles across multiple time ranges and resolves mosaic and shadow artifacts for a more natural-looking result. To enable this version, set the
ModelIdparameter. For more information, see ModelId parameter. -
Basic: The output may contain mosaic or shadow artifacts, and the restoration quality is average.
-
-
Time Range: You can specify a custom start time and end time.
Note-
You must specify the start and end times as a pair.
-
The start time cannot be later than the end time.
-
If you do not specify a time range, the entire video is processed by default.
-
The advanced version supports up to five time ranges, while the basic version supports only one.
-
-
-
After you finish the configuration, click OK. The system generates a workflow ID. Record this ID for use when uploading videos. The new Subtitle Removal workflow template appears in the template list on the Workflow Management page with a status of Enabled. In the Actions column, you can click Details to view the configuration or Disable to disable the template.
Step 2: Trigger the workflow
You can trigger this workflow during or after video upload.
Console
During upload
-
Log on to the ApsaraVideo VOD console.
-
In the left-side navigation pane, choose Media Files > Audio/Video, and then click Upload Audio/Video.
-
On the Upload Audio/Video page, select Use Workflow as the upload method and add your video. In the processing options for the file, select Process with Workflow, choose the Subtitle Removal workflow from the Transcoding Template Group/Workflow Template list, and then click Start Upload.
After upload
-
Log on to the ApsaraVideo VOD console.
-
In the left-side navigation pane, choose Media Files > Audio/Video.
-
Find the target video and click Processing in the Actions column. In the Media Processing dialog box, set Processing Type to Process with Workflow, select the Subtitle Removal workflow, and then click OK.
API
During upload
-
The CreateUploadVideo API operation obtains an upload URL and credentials and creates basic media asset information but does not upload the media file itself. You must implement the complete upload logic. For a full example of uploading by using an API, see Upload media files by using the ApsaraVideo VOD API.
-
When you call the CreateUploadVideo or UploadMediaByUrl API operation to upload an audio or video file, set the
WorkflowIdparameter to the ID of the workflow that you created in the previous step. After the upload is complete, ApsaraVideo VOD automatically processes the file based on the specified workflow.
After upload
Call the SubmitWorkflowJob API operation and set the WorkflowId parameter to your workflow ID. This immediately starts processing for the specified media file.
Step 3: Query results
VOD console
-
Log on to the ApsaraVideo VOD console.
-
In the left-side navigation pane, choose Media Files > Audio/Video.
-
On the Audio/Video page, find the video generated by the workflow. You can filter by the source video name or creation time.
IMS console
-
Log on to the Intelligent Media Services console.
-
In the left-side navigation pane, choose VOD Media Processing > Task Management, and then click the Smart Removal tab.
-
On the Smart Removal tab, find the removal task generated by the workflow. You can filter by name or creation time.
-
When the task status is Success, click View to see the details:
-
Basic parameters and configuration of the removal task.
-
Input information for the removal video.
-
Output information and output file of the removal video.
-
Callback
-
Configure HTTP or Message Service (MNS) callbacks.
-
When a workflow task is complete, the system triggers a WorkflowTaskComplete event and pushes the results through the configured HTTP or Message Service (MNS) callback. Key fields in the callback include:
-
Status: The overall status of the task (Succeed/Failed). -
ActivityResults: A JSON string containing the execution details of each node, such asVideoDetext. TheResultfield contains key output information, such as theMediaIdandJobId. -
TaskInput: The original input media information, such as theMediaID and filename.
-
-
Parse
ActivityResultsin the callback payload to extract theMediaIdof the generated video for playback or distribution.
API
Call the GetAIMediaAuditJob API operation to query the task result. For the JobId parameter, use the ID from ActivityResults.Result.JobId in the workflow task callback. The API response contains the detailed status and output of the job.
-
JobId: The job ID returned when the workflow internally calls the SubmitAIMediaAuditJob operation to submit the subtitle removal task.
Sample response:
{
"RequestId": "****20b48fb04483915d4f2cd8ac****",
"JobId": "****20b48fb04483915d4f2cd8ac****",
"FunctionName": "VideoDetext",
"Input": {
"Type": "OSS",
"Media": "oss://example-bucket/input.mp4"
},
"Output": {
"Type": "OSS",
"Media": "oss://example-bucket/output.mp4",
"OutputUrl": "http://example-bucket.oss-cn-shanghai.aliyuncs.com/output.mp4"
},
"Status": "Success",
"CreateTime": "2024-09-24T06:17:09Z",
"FinishTime": "2024-09-24T06:17:31Z",
"OutputFiles": ["output.mp4"],
"OutputUrls": ["http://example-bucket.oss-cn-shanghai.aliyuncs.com/output.mp4"],
"Result": {}
}
-
A
StatusofSuccessindicates that the subtitle removal is complete. -
You can access the processed video using the URLs in the
OutputUrlsfield.