LVM-Watermark Filter (DLC) is a Platform for AI (PAI) component that filters watermarked videos from training datasets. It runs as a Deep Learning Containers (DLC) job and processes only MP4 videos.
How it works
The component samples frames from each video at uniform intervals, runs a watermark detection model on each sampled frame to produce a probability score, and then compares that score against the configured maximum watermark probability threshold. Videos that exceed the threshold are filtered out. The results are written to the output path for use in downstream training pipelines.
The typical use case is cleaning datasets before training video generation models.
The watermark detection model requires a GPU instance. Select a GPU instance type when configuring the resource group.
Only MP4 videos are supported.
Supported computing resources
Inputs and outputs
Input
Connect one of the following as the upstream component:
Read File Data — reads the Object Storage Service (OSS) path where training data is stored. Configure the OSS Data Path parameter to point to the OSS directory containing the video files, or to a previously generated
video_meta.jsonlmetadata file.Any other component from LVM Data Processing (DLC).
Output
Filtering results written to the OSS directory specified by Output File OSS Path. The output directory contains the following files:
| File | Description |
|---|---|
{name}.jsonl | Output file. The filename is controlled by the Output Filename parameter. |
{name}_stats.jsonl | State file. |
dj_run_yaml.yaml | Parameter configuration used when the algorithm ran. |
Configure the component
Configure the following parameters in Machine Learning Designer.
Field settings
| Parameter | Required | Default | Description |
|---|---|---|---|
| Video Data OSS Path | No | — | The OSS directory containing the video files. If no upstream component is connected on the first run, set this parameter manually. Running the component generates video_meta.jsonl in the parent directory of the path you specify. On subsequent runs, point this parameter directly to video_meta.jsonl to skip re-scanning. |
| Output File OSS Path | Yes | — | The OSS directory where filtering results are written. |
| Output Filename | Yes | result.jsonl | The base filename for the output and statistics files. |
Parameter settings
| Parameter | Required | Default | Description |
|---|---|---|---|
| Watermark Probability | Yes | 0.8 | The maximum watermark probability. Videos whose watermark probability exceeds this threshold are filtered out. |
| Number of Sampled Frames | Yes | 1 | The number of frames sampled from each video for analysis. Frames are selected at uniform intervals across the video duration. |
Execution tuning
| Parameter | Required | Default | Description |
|---|---|---|---|
| Number of Processes | Yes | 4 | The number of parallel processes used during filtering. |
Resource group
| Parameter | Required | Default | Description |
|---|---|---|---|
| Public Resource Group | No | — | The instance type (CPU or GPU) and virtual private cloud (VPC) to use. Select a GPU instance type — the watermark detection algorithm requires a GPU. |
| Dedicated resource group | No | — | The number of vCPUs, memory, shared memory, and GPUs to allocate. |
| Maximum Running Duration (seconds) | No | — | The maximum time the component can run. If this limit is exceeded, the job is terminated. |