ApsaraVideo Media Processing (MPS) converts media files to suit different network bandwidths, devices, and user needs. It analyzes media content, text, speech, and scenes to provide automated review, content recognition, and smart editing.
Audio and video transcoding
Transcoding converts media streams into different resolutions, encoding formats, or container formats for various network bandwidths and playback devices. MPS supports mainstream encoding and container formats, and allows adding watermarks, captions, or simple edits during transcoding. For features unavailable in the console or API, contact your account manager.
To use the following features, you must submit a transcoding job. Standard transcoding fees apply, based on the specifications and duration of the output video. For billing details, see Transcoding Pricing.
|
Parameter |
Type |
Description |
|
Input file |
Container format |
|
|
Video encoding format |
Apple ProRes, AVS+, AVS, AVS2, H.263, H.263+, H.264/AVC, H.265/HEVC, H.266/VVC, MJPEG, MPEG-1, MPEG-2, MPEG-4, QuickTime, RealVideo, VP8, VP9, and Windows Media Video. |
|
|
Audio encoding format |
AAC, AC3, ADPCM, AMR, DSD, EAC3, MP1, MP2, MP3, PCM, RealAudio, Vorbis, and Windows Media Audio. |
|
|
File size |
Up to 100 GB. |
|
|
Chroma subsampling |
Examples: 4:2:2, 4:2:0. |
|
|
Output file |
Container format |
Note
|
|
Encoding format |
|
|
|
Encoding profile |
|
|
|
Resolution |
|
|
|
Bitrate |
|
|
|
Frame rate |
The output frame rate range is (0, 60] fps. |
|
|
Bit depth |
|
|
|
Pixel format |
yuv420p, yuvj420p, yuv422p, yuvj422p, yuv444p, yuvj444p, etc. |
|
|
Bitrate control |
VBR (Variable Bitrate), CBR (Constant Bitrate), ABR (Average Bitrate), and CRF (Constant Rate Factor). |
|
|
Scan mode |
Supported modes include the original video mode, automatic deinterlacing, interlaced scan, and progressive scan. |
Narrowband HDTM
Narrowband HDTM uses Alibaba Cloud's transcoding technology to improve video compression, reducing file sizes, minimizing playback buffering, and lowering storage and bandwidth costs.
To use these features, you must select the corresponding Narrowband HD transcoding template when you submit a transcoding job. You are charged for Narrowband HD transcoding based on the specifications and duration of the output video. For information about billing, see Transcoding Pricing.
|
Type |
Description |
|
Narrowband HDTM 1.0 |
Analyzes scenes, motion, content, and textures to reduce bitrate by 20%–40% while maintaining quality, or deliver higher definition at the same bandwidth. Supports H.264 and H.265. Other parameters match standard transcoding. Start a free trial. |
|
Narrowband HDTM 2.0 |
Surpasses standard encoders by integrating lightweight definition restoration and quality enhancement, reducing bitrate by 40%–60% while maintaining quality or delivering premium HD on low-bandwidth networks. Supports H.264 and H.265. Other parameters match standard transcoding. Start a free trial. |
Audio and video enhancement
Audio enhancement
ApsaraVideo Audio Lab combines signal processing with deep learning to enhance and restore audio across various scenarios.Try it online.
To use the following features, you must configure the corresponding parameters when you submit a transcoding job. The audio portion incurs audio and video enhancement fees based on the output audio's specifications and duration. The video portion incurs transcoding fees based on the services used. For billing details, see Audio and video enhancement fees and Transcoding fees. To enable an audio enhancement transcoding template, contact Alibaba Cloud technical support by joining DingTalk group 32171220.
|
Feature |
Description |
|
Dolby Audio |
|
|
Spatial audio |
Delivers a more immersive and realistic listening experience. |
|
Virtual surround sound |
Transcodes 5.1/7.1 multi-channel audio to stereo while preserving surround sound, enabling high-quality surround on mobile phones, tablets, headphones, and TVs. |
|
Sound enhancement |
Enhances mono, stereo, 5.1, and 7.1 audio to deliver cinematic-quality sound across headphones and speakers. Ideal for music, speeches, and film. |
|
Volume normalization |
Adjusts and normalizes loudness to resolve volume fluctuations from inconsistent sources in short video feeds and continuous music playback. |
Video enhancement
To use the following features, select the corresponding video enhancement transcoding template when you submit a transcoding job. Transcoding fees are based on the output video's specifications and frame count. For billing details, see Audio and video enhancement fees. To enable a video enhancement transcoding template, contact Alibaba Cloud technical support by joining DingTalk group 32171220.
This feature improves video definition, color accuracy, and smoothness for media processing needs such as 4K/8K ultra-high-definition (UHD) production, standard-definition (SD) to high-definition (HD) upscaling, optimization of internet user-generated content (UGC), and legacy video restoration. Delivering higher-quality audiovisual content enhances the user experience, leading to increased customer conversion and longer viewing times.Try it online.
|
Feature |
Description |
|
super-resolution |
Restores fine details and repairs minor artifacts such as blur and noise, with a special focus on enhancing facial regions. It upscales low-resolution videos to 720p or 1080p HD versions, and converts HD films into high-quality 2K or 4K sources. |
|
frame interpolation |
Applies adaptive frame interpolation strategies based on motion characteristics by combining intelligent analysis and dynamic compensation technologies. It supports seamless interpolation up to 120 fps to resolve issues like panning stutter and frame jitter from low frame rates. |
|
video restoration |
Intelligently repairs severe damage from video acquisition, compression, or physical projection, including compression noise, blocking artifacts, scratches, and mold spots, while maintaining inter-frame stability and detail preservation. |
|
HDR |
|
|
SDR+ |
Uses scene and content analysis to intelligently optimize a video's exposure, contrast, saturation, and white balance within the SDR color gamut, improving the subjective viewing experience while maintaining stable and consistent colors. |
High-speed transcoding
High-speed transcoding splits a video into segments and transcodes them in parallel, boosting speed 5–30x. Ideal for time-sensitive content such as news, sports events, and premieres.
To accelerate a transcoding job, submit it to a high-speed transcoding queue. In addition to the standard fees for audio and video transcoding or audio-visual enhancement, an extra fee applies for high-speed transcoding. This fee is based on the output video's specifications, duration, and the actual speed boost. For billing details, see High-speed transcoding.
|
Type |
Description |
|
Speed boost |
The speed boost is typically 5 to 30 times, depending on the input video. You can set an expected speed boost on the high-speed transcoding queue, such as 5x, 10x, 20x, or 30x. |
|
Applicable videos |
Recommended for videos longer than 30 minutes or those requiring complex processing such as high frame rates, UHD resolutions, or audio-visual enhancement. Limits on high-speed transcoding. |
|
Fallback policy |
Segmentation is not supported for all videos. If a video that is incompatible with high-speed transcoding is submitted to a high-speed transcoding queue, the job automatically falls back to the regular transcoding mode. |
Additional processing
Media information
Retrieve metadata from audio and video files in OSS, such as resolution, bitrate, frame rate, codec, and file format.
To use this feature, you must submit a media information job. This feature is billed for API calls based on the number of requests. For billing details, see Pricing for API calls.
Simple editing
Perform simple video editing, such as audio extraction, video extraction, merging, clipping, and audio mixing.
To use the following features, you must set the corresponding parameters when you submit a transcoding job. This feature incurs transcoding fees based on the specifications and duration of the output video. For billing details, see Transcoding pricing.
|
Feature |
Description |
Related API parameters |
Console operation |
|
Audio extraction |
Extracts the audio track from a video file by disabling the video track. |
Remove |
Supported |
|
Video extraction |
Extracts the video track from a video file by disabling the audio track. |
Remove |
Supported |
|
Black bar removal |
Detects and automatically crops black bars from the video frame. |
Crop |
Not supported |
|
Video cropping |
Specifies the width and height of the area to retain after cropping, and the position of this area relative to the top-left corner of the input video frame. The surrounding areas are automatically cropped. |
Crop |
Not supported |
|
Black bar addition |
Specifies the width and height of the output video after black bars are added, and the position of the input video frame relative to the top-left corner of the output frame. The surrounding gaps are filled with black bars. |
Pad |
Not supported |
|
Orientation adaptation |
Defines output resolution by long and short sides instead of width and height. Recommended when input videos include both landscape and portrait orientations. |
LongShortMode |
Supported |
|
Video rotation |
Sets the rotation angle of the video. |
Rotate |
Supported |
|
Video merging |
Merge up to 100 videos. You can set the start time and duration for each video to be merged. |
MergeList or MergeConfigUrl |
Not supported |
|
Video clipping |
|
Clip |
Supported |
|
Opening and tail slates |
Overlays a dynamic logo at the beginning and appends tail slate content for brand recognition and copyright display. |
OpeningList and TailSlateList |
Supports tail slate addition |
|
Blurring |
Blurs a specified area within the video. |
DeWatermark |
Not supported |
|
Audio mixing |
Merges two audio tracks, which is ideal for adding background music. |
Amix |
Not supported |
Video snapshots
Take snapshots of a specified size at specified time points in a video. You can use snapshots for video covers, sprites, and thumbnails on player progress bars.
To use the following features, you must submit a snapshot job. This feature is billed for API calls based on the number of snapshots taken. For billing details, see Pricing for API calls.
|
Feature |
Description |
Related API parameters |
Console operation |
|
Static snapshot |
Takes JPG snapshots of a specified size at specified time points in a video. The following sampling methods are available:
|
SnapshotConfig |
Supported |
|
Sprite snapshot |
Stitches static snapshots into a single large image (sprite) based on layout rules. Output is in JPG format. Supports only asynchronous calls. A single sprite request retrieves multiple images, reducing request count and improving client performance. |
TileOut, TileOutputFile |
Not supported |
|
WebVTT snapshot |
Generates a VTT file for static snapshots or a sprite, containing timestamps, file URLs, and coordinate information. To display an image, retrieve and parse the VTT file first. Useful for player progress bar thumbnails. |
SubOut |
Supported |
|
Keyframe snapshot |
This feature takes snapshots only at keyframes. If a specified time point is not a keyframe, the service uses the nearest keyframe instead. |
FrameType |
Supported |
|
First-frame black screen detection |
You can enable black screen detection for the first frame (time=0). A black screen is defined by the percentage of black pixels and a color value threshold. The service scans the first 5 seconds: if a non-black frame is found, it is captured. Otherwise, a single-snapshot job fails; a multi-snapshot job captures the first black frame. |
BlackLevel, PixelBlackThreshold |
Supported |
Video watermarks (visible watermarks)
Adds visible watermarks, such as a company logo or TV station icon, to a video to enhance brand identity, highlight copyright, and increase product recognition. To add invisible blind watermarks for copyright tracing, see the Digital watermarking (blind watermarks) section below.
To use the following features, you must submit a transcoding job and specify the watermark assets and an optional watermark template. This feature incurs transcoding fees based on the specifications and duration of the output video, and is billed for API calls based on the number of watermarks. For billing details, see Audio and video transcoding fees and Pricing for API calls.
|
Feature |
Description |
Related API parameters |
Console operation |
|
Image watermark |
|
WaterMarks |
|
|
Text watermark |
|
WaterMarks |
Not supported |
Subtitles
Add subtitles to videos for multi-language support or for viewing in silent or noisy environments. This feature also makes video content accessible to users with hearing impairments.
To use the following features, you must submit a transcoding job or initiate transcoding through a workflow. This feature incurs transcoding fees based on the specifications and duration of the output video. For billing details, see Audio and video transcoding fees.
|
Feature |
Description |
Related API parameters |
Console operation |
|
Hard subtitle |
Burns the subtitle file into video frames, replacing pixels with subtitle content. Supports SRT and ASS formats with configurable fonts. Hard subtitles display in all playback scenarios regardless of player support. |
SubtitleConfig |
Not supported |
|
Caption packaging |
Uses a workflow to package subtitle files with audio and video streams into a master playlist (.m3u8 or .mpd). Supports up to four subtitle tracks for in-player switching. Formats: VTT for HLS; VTT, STL, and TTML for DASH. |
|
Supported |
Video packaging
Packaging is the process of generating a master playlist file from multiple bitrate renditions, audio tracks, and subtitle tracks. In streaming media scenarios, this enables:
-
Adaptive bitrate streaming: Adapts the stream quality to the user's network conditions to reduce startup latency and playback buffering.
-
Ad insertion: Flexibly inserts video ads between segments.
To use the following features, you must initiate transcoding through a workflow. This feature incurs transcoding fees based on the specifications and duration of the output video. For billing details, see Audio and video transcoding fees.
|
Feature |
Description |
Related API parameters |
Console operation |
|
HLS protocol |
Packages content using the Apple HLS protocol. It supports a two-level index structure. The index file is in M3U8 format and the video file is in TS format. |
Supported |
|
|
DASH protocol |
Packages content using the DASH ISO protocol. It supports a single-level index. The index file is in MPD format and the video file is in fMP4 format. |
Not supported |
|
|
CMAF protocol |
Packages content using the CMAF protocol. It supports output in HLS or DASH format. |
None |
Not supported |
|
Custom segment duration |
Specifies up to 10 segment time points and a separate interval for the remaining duration. The interval can range from 1 to 60 seconds. This feature helps you set the segment duration based on the client's bandwidth conditions to reduce the time to first frame. |
Segment |
Not supported |
Video encryption
To use the following features, you must initiate transcoding through a workflow. This feature incurs transcoding fees based on the specifications and duration of the output video. For billing details, see Audio and video transcoding fees.
|
Feature |
Description |
Related API parameters |
Console operation |
|
HLS encryption |
Encrypts video content using HLS AES-128 with self-managed keys or KMS. Decrypts and plays on any standard HLS-compliant player. Provides strong security with broad device compatibility. |
Not applicable |
Supported |
|
Alibaba Cloud proprietary cryptography |
Encrypts video files using Alibaba Cloud's proprietary protocol into encrypted HLS format with KMS key management. Playback requires ApsaraVideo Player, preventing playback and redistribution even if files are downloaded. Ideal for online education and premium content. |
Not applicable |
Supported |
Video AI
Intelligent production
Using Alibaba Cloud's media AI technology, this feature processes and generates media content to improve production efficiency and quality.Try it online.
To use the following features, you must submit an intelligent production job. You are charged for intelligent production based on the duration of the successfully processed video. For billing details, see Intelligent production billing.
|
Type |
Feature |
Service name |
Description |
|
Audio processing |
Chorus detection |
MusicSegmentDetect |
Analyzes the structure of a song to identify and extract the timestamps of its chorus or highlight moments. This is useful for scenarios such as music recommendation previews and karaoke segments of popular clips. |
|
Beat detection |
AudioBeatDetection |
Analyzes and identifies multi-layered beat points in a piece of music and returns their timestamps. This can be used for scenarios such as audio and video production and music recommendations. |
|
|
Audio mixing |
AudioMixing |
Balances and mixes audio tracks from multiple input files, such as vocals and music, into a single audio output. |
|
|
Audio quality assessment |
AudioQualityAssessment |
Detects issues in the input audio, such as silence and stuttering. |
|
|
Noise reduction |
SpeechDenoise |
Filters background noise from vocal recordings while maintaining vocal fidelity, improving audio quality for real-time communication such as online meetings and virtual classrooms. |
|
|
Music source separation |
MusicDemix |
Separates vocals and instrumental accompaniment into independent audio files. Ideal for creating a cappella or instrumental tracks, isolating background music, asset acquisition, and audio post-production. |
|
|
Video processing |
Smart cover generation |
Cover |
Static cover: Selects the optimal image from a video as its static cover. Animated cover: Extracts multiple highlight frames from a video and combines them into an animated GIF cover. |
|
Video summary |
VideoClip |
Extracts and combines highlight clips from a video to generate a 5-second summary. |
|
|
Caption extraction |
CaptionExtraction |
Detects and extracts Chinese or English subtitles from a video, generating an SRT file with the text and timestamps. This feature is useful for video caption processing. |
|
|
Text removal |
VideoDetext |
Uses text detection and image inpainting to find and remove text and subtitles from a video or image, restoring the original background. |
|
|
Logo blurring |
VideoDelogo |
Detects the position of logos in a video and applies a blur effect. This helps obscure logos for post-production and content repurposing. |
|
|
Horizontal-to-vertical video conversion |
VideoH2V |
Identifies the main subject and performs adaptive cropping to convert a landscape video into portrait format for mobile viewing and multi-platform delivery. |
|
|
Chroma keying |
VideoGreenScreenMatting |
Detects a solid-color background, such as a green screen, in a video and replaces it with a specified background image. This feature is ideal for film production, short video creation, and post-editing. |
|
|
Video face beautification |
FaceBeauty |
Uses face detection to automatically apply configurable enhancements such as skin smoothing, whitening, and rosy cheeks. Ideal for short video production and post-processing. |
|
|
Image processing |
Logo blurring |
ImageDelogo |
Detects the position of logos in an image and applies a blur effect to restore the image to its original state. |
|
Horizontal-to-vertical image conversion |
ImageH2V |
Identifies the main subject in an image and performs adaptive cropping to convert a landscape image into a portrait format suitable for mobile viewing. |
|
|
Face stylization |
ImageCartoonize |
Transforms faces in an image into stylized versions, supporting styles such as anime and American comics. |
Smart tagging
Analyzes visual, text, audio, and behavioral information in media files. It uses multimodal information fusion and alignment to recognize content with high accuracy and automatically output multi-dimensional, structured tags. These tags are useful for applications like video search, personalized recommendations, and ad targeting.Try it online.
To use the following features, you must submit a smart tagging job. You are charged for smart tagging based on the duration of the successfully processed video. For billing details, see Smart tagging billing.
|
Feature |
Description |
|
Video classification |
Analyzes video content to determine its category. |
|
Label detection |
Analyzes the visual information in a video to identify content labels such as people, objects, and scenes. |
|
Text detection in video (OCR) |
Detects and extracts text from video frames. |
|
Speech-to-text (ASR) |
Analyzes the audio in a video and transcribes the speech into text. |
|
Custom labels |
Supports custom labels for faces, landmarks, logos, and products. |
The following table describes the input file limitations for smart tagging.
|
File parameter |
Video |
Audio |
Image |
|
Container format |
AVI, FLV, MKV, MPG, MP4, TS, MOV, MXF |
MP3, WAV |
JPG, JPEG, PNG |
|
Encoding format |
MPEG2, MPEG-4, H.264/AVC, H.265/HEVC |
Not applicable |
Not applicable |
|
Duration |
Up to 4 hours |
Up to 4 hours |
Not applicable |
|
Size |
Up to 4 GB |
Up to 400 MB |
Up to 4 MB |
|
Resolution |
240p to 2160p. For optimal analysis, a resolution of 720p or higher is recommended. |
Not applicable |
Up to 2160p |
Automated review
This feature reviews media content—including titles, descriptions, covers, videos, and audio—to detect policy violations. Common use cases include short video platforms, live streaming, and media compliance.Try it now.
To use the following features, you must submit a media review job. You are charged for automated review based on the duration of the successfully processed video. For billing details, see Automated review billing.
|
Feature |
Content |
Description |
|
Content moderation |
Pornography detection |
Detects pornographic and sexually suggestive content in audio, text, and visuals. |
|
Terroristic content detection |
Detects terroristic and politically sensitive content across more than 10 categories, including weapons, gore, specific attire, explosions, special symbols, crowds, and parades. |
|
|
Ad and QR code detection |
Detects various forms of ads in media files, including text, watermarks, QR codes, spammy ads, and mini program codes. |
|
|
Logo detection |
Detects various logos in videos and images, such as TV station logos, trademarks, and watermarks. This is often used for copyright protection. |
|
|
Undesirable scene detection |
Detects undesirable scenes that require moderation, such as picture-in-picture (PiP), smoking, in-car live streaming, and meaningless footage. |
|
|
Audio anti-spam |
Detects policy violations in audio, such as pornographic, terroristic, and abusive content. Supports both Chinese and English. |
Media fingerprinting
This feature uses Alibaba Cloud's technology to create a unique fingerprint for a media file. By extracting and comparing these fingerprints for videos, images, audio, and text, you can find duplicate content and trace the source of media clips.Try it now.
To use the following features, you must submit a media fingerprinting job. You are charged for media fingerprinting based on the duration of the successfully processed video or audio. For billing details, see Media fingerprinting billing.
|
Feature |
Description |
|
Video fingerprinting |
Extracts a visual fingerprint from a video. You can add the fingerprint to a library for analysis and comparison to find similar videos. |
|
Audio fingerprinting |
Extracts an audio fingerprint. You can add the fingerprint to a library for analysis and comparison to find similar audio tracks. |
|
Image fingerprinting |
Extracts a visual fingerprint from an image. You can add the fingerprint to a library for analysis and comparison to find similar images. |
|
Text fingerprinting |
Extracts a fingerprint from text content. You can add the fingerprint to a library for analysis and comparison to find similar text. |
Digital watermarking (invisible watermark)
This feature embeds invisible digital watermarks into videos. The watermark does not affect visual quality or file integrity and can be extracted for copyright protection, leak tracing, broadcast monitoring, information transmission, and file authentication. Digital Watermarking (Invisible Watermark).
To use the following features, you must submit a video copyright watermarking or traceable video watermarking job. You are charged for digital watermarking based on the duration of the successfully processed video. For billing details, see Digital watermarking billing.
|
Feature |
Description |
Related APIs |
Console actions |
|
Video copyright watermarking |
Embeds invisible text information into a video that is at least 3 minutes long. The watermark does not affect visual quality or file integrity and can be extracted even after the video is redistributed or modified. |
Submit a video copyright watermarking job, Submit a copyright watermark extraction job |
Not supported |
|
Traceable video watermarking |
Transcodes a video (3+ minutes) into two HLS streams (A and B), each with a different traceability watermark. During playback, segments from both streams are randomly combined into a unique M3U8 file tied to the user's identity. If piracy occurs, the watermark identifies the responsible user. |
Submit a traceable watermarking A/B stream job, Submit a traceable watermarking M3U8 job, Submit a traceable watermark extraction job |
Not supported |
Service management
|
Category |
Description |
API |
Console |
|
Media management |
Upload, manage, and publish media files. |
N/A |
N/A |
|
Workflow orchestration |
Automates cloud-based media processing by automatically starting jobs when a media file is uploaded. |
N/A |
Supported |
|
Transcoding template |
A transcoding template defines a reusable set of transcoding parameters, simplifying job creation and workflow configuration. Templates are custom, customized, or system-preset. |
|
Supported |
|
Watermark template |
A watermark template contains parameters that define a watermark's properties, such as its position and size. Using a template simplifies the process of adding watermarks. |
|
Supported |
|
Transcoding priority |
Sets the priority of a transcoding job within an MPS queue. You can define up to 10 priority levels. |
|
Not supported |
|
Conditional transcoding |
Compares source video properties (bitrate, resolution, audio bitrate) against output settings. If source values are lower, the job is skipped or the source is passed through unchanged. |
Examples: |
Supported |
|
MPS queue |
An MPS queue holds asynchronous jobs (transcoding, snapshotting, etc.) for scheduling and execution. You can create multiple queues with up to 10 priority levels per queue. |
|
Not supported |
|
Message notification |
Asynchronous jobs run in the background. Bind an MNS topic or queue to an MPS queue or workflow to receive automatic notifications when jobs complete or workflows start or finish. |
|
Supported |
|
Usage statistics |
Lets you query the usage duration for features like transcoding, snapshots, video review, and media fingerprinting within a specified time range. |
N/A |
N/A |