Smart tagging analyzes visual, text, audio, and behavioral information in videos. It uses multimodal information fusion and alignment for high-accuracy content recognition. This feature automatically generates multi-dimensional content tags for videos, converting unstructured information into a structured format. You can use smart tagging for media asset retrieval, personalized recommendations, and intelligent advertising.
Feature overview
Scenarios
You can retrieve media resources with accuracy and efficiency. Smart tagging uses a rich and accurate video tag system to catalog media files, such as videos, images, and text. You can quickly retrieve content from your media library using keywords or tags. This improves the efficiency and accuracy of video retrieval.
Personalized recommendations: Smart tagging provides accurate and personalized content recommendations. It matches content tags with user information, behavioral data, and user personas. This enables personalized video recommendations, solves the cold-start problem, and improves recommendation accuracy.
Intelligent advertising: You can deliver intelligent, scenario-based ads. Smart tagging analyzes multimodal content and video tags to automatically identify ad slots. It then matches the slots with content from your ad resource library. This enables intelligent ad delivery, reaches the target audience with precision, and increases ad conversion rates.
Features
For a list of features, see Smart tagging.
To try the smart tagging feature online, click Online Demo.
Limits
For information about service regions, see Service regions.
For more information about MPS queue concurrency, see Limits.
The following table describes the file formats that smart tagging can process.
Video
Audio
Image
Video formats: AVI, FLV, MKV, MPG, MP4, TS, MOV, and MXF
Audio formats: MP3 and WAV
Image formats: JPG, JPEG, and PNG
Encoding formats: MPEG-2, MPEG-4, H.264, and H.265/HEVC
N/A
N/A
Video duration: ≤ 4 hours
Audio duration: ≤ 4 hours
N/A
Video size: ≤ 4 GB
Audio size: ≤ 400 MB
Image size: ≤ 4 MB
Video resolution: 240p to 2160p. For best analysis results, a resolution of 720p or higher is recommended.
N/A
Image resolution: ≤ 2160p
Prerequisites
Before you start, perform the following configurations to ensure that the feature works correctly. If you have already configured the settings, you can skip this section.
Create an MPS queue. If no smart tagging MPS queue is available in the MPS queue list, you must first enable one.
Upload the video to be processed to OSS.
If you want to create a job, directly upload the video to OSS. For more information, see Upload videos.
If you want to create a workflow and automatically trigger the workflow, you must add media buckets and create a workflow before you upload the video. After you specify the input and output media buckets and upload a media file to the input media bucket, MPS receives a message about the upload and triggers the associated workflow. After the workflow is complete, the processed media file is uploaded to the output media bucket. For more information, see Add media buckets.
Optional. Enable the notification feature. If you want MPS to send notifications on jobs and workflows to MNS topics or queues, configure an MNS queue or topic as required. For more information, see Set up message notifications.
Submit a smart tagging job in the console
Log on to the MPS console.
In the top navigation bar, select a region from the drop-down list.
In the left navigation pane, choose Job Management.
Select Smart Tagging and click Create Job.
Set the parameters.
Parameter | Description |
Input File URL | Click Select. From the Bucket drop-down list, select a bucket name. The folders created in the bucket are displayed under Path. Select a file in a folder as the input path. |
Smart Tag Pipeline | Select the pipeline name from the drop-down list. Note If no MPS queue name is available in the drop-down list, enable a dedicated smart tagging MPS queue first. For more information, see Create an MPS queue. |
Title | Click the input box and enter a file title. The title is used in multimodal recognition for video classification analysis. |
Template | Select a template name from the drop-down list. Note If no template name is available in the drop-down list, create a smart tagging template first. For more information, see Smart tagging templates. |
Return Raw OCR Results | Click the switch to specify whether to return the text recognition results. If you turn on the switch, the smart tagging job can return the raw Optical Character Recognition (OCR) results. |
Return Raw ASR Results | Click the switch to specify whether to return the speech recognition results. If you turn on the switch, the smart tagging job can return the raw Automatic Speech Recognition (ASR) results. |
Click Submit Job to create the smart tagging job.
Submit a smart tagging job using an API
Configure the analysis type using a template.
Set the analysis type for a smart tagging job in a template. You must specify the template ID when you submit the job. To obtain a template ID, add a template.
Configure a custom face library.
Smart tagging jobs support analysis that uses public or custom face libraries. You must specify the face library parameters when you submit a job. For more information about how to create a custom face library, see Register a custom face.
Submit a smart tagging job.
You can submit a smart tagging job using an API or a software development kit (SDK) for videos, images, and audio files that are stored in OSS, available at third-party URLs, or located as media assets in ApsaraVideo VOD (VOD). For more information about the parameters, see Submit a smart tagging job and Smart tagging SDKs.
Receive callback messages.
After a smart tagging job is complete, a message is sent to the specified queue or topic in Simple Message Queue (formerly MNS) if you have configured MNS event notifications for the MPS queue. For more information, see Receive notifications.
Query the job result.
You can query the status and result of a smart tagging job by its job ID using an API or an SDK. For more information about the parameters, see Query a smart tagging job.