LVM-Caption-Video Mapper (DLC) generates text captions for MP4 videos in a Platform for AI (PAI) machine learning pipeline. It uses the VideoBLIP model to sample frames from a video and produce candidate captions based on those frames.
Prerequisites
Before you begin, ensure that you have:
A GPU instance — the VideoBLIP model requires a GPU to run
MP4 video files uploaded to Object Storage Service (OSS)
Access to Machine Learning Designer in PAI
How it works
The component reads MP4 videos from an OSS path.
It samples frames at evenly spaced intervals across the video duration.
VideoBLIP generates one or more candidate captions from the sampled frames.
The captions are written to a JSONL output file in the specified OSS directory.
On the first run, the component also generates a metadata index file (video_meta.jsonl) in the parent directory of your video data path. Pass this file as input on subsequent runs to skip re-scanning the video directory.
Supported computing resources
Input ports
Connect one of the following to the component's input port:
| Input | Description |
|---|---|
| Read File Data component | Reads the OSS path where your training data is stored |
| Any LVM Data Processing (DLC) component | Passes video data directly from an upstream processing step |
You can also set Video Data OSS Path directly in the component parameters to point to an OSS directory or an existing video_meta.jsonl file.
Output
The component writes results to the OSS path specified by Output File OSS Path:
<Output File OSS Path>/
├── <output-filename>.jsonl # Caption output (default: result.jsonl)
└── dj_run_yaml.yaml # Parameter configuration used during this runConfigure the component
Set the following parameters in Machine Learning Designer.
Field settings
| Parameter | Required | Default | Description |
|---|---|---|---|
| Video Data OSS Path | No | — | OSS directory containing your MP4 videos, or the path to an existing video_meta.jsonl file. Required if no upstream component is connected on the first run. The component generates video_meta.jsonl in the parent directory of this path on the first run. |
| Output File OSS Path | Yes | — | OSS directory where results are saved. |
| Output Filename | Yes | result.jsonl | File name for the caption output. |
Parameter settings
| Parameter | Required | Default | Description |
|---|---|---|---|
| Number of Candidate Captions | Yes | 1 | Number of caption candidates to generate per video. Increase this value to produce more diverse options for downstream selection. |
| Number of Sampled Frames | Yes | 3 | Number of frames to sample per video. Frames are evenly spaced across the video duration. Higher values capture more visual context but increase processing time. |
Execution tuning
| Parameter | Required | Default | Description |
|---|---|---|---|
| Select Resource Group > Public Resource Group | No | — | Select the GPU instance type and virtual private cloud (VPC). A GPU instance is required — the VideoBLIP model does not run on CPU-only instances. |
| Select Resource Group > Dedicated resource group | No | — | Specify the number of vCPUs, memory size, shared memory size, and number of GPUs to allocate. |
| Maximum Running Duration (seconds) | No | — | Time limit for the job. If the job exceeds this limit, it is terminated. |