Custom event detection-Object Storage Service(OSS)-阿里云帮助中心

Traditional video retrieval relies on manual labeling or after-the-fact video review, making it difficult to quickly locate specific events like "a person falling," "casting a net," or "prohibited advertisements" from massive video archives. Custom event detection allows you to describe business-critical events in a single sentence. When a video is ingested, AI automatically recognizes these events, applies corresponding event labels, and writes them to the file metadata. You can then directly search for videos that contain these events by using metadata query APIs, eliminating the need to build your own video understanding pipeline.

Important

This feature is based on the new version of MetaQuery and is currently in invitation-only preview. It is supported only in the China (Hangzhou) and Singapore regions. Before you use this feature, ensure that your target bucket is in one of these regions and contact technical support to request access to the invitation-only preview of the new MetaQuery capabilities.

Use cases

Security monitoring: Automatically identify security incidents such as "a person falling" or "someone climbing a fence" from large volumes of surveillance footage. This facilitates post-event retrieval and alert integration.
Content moderation: Automatically flag business-defined violations in user-generated content (UGC) videos, such as "prohibited advertisements" or "inappropriate gestures," to assist manual review.
Industry inspection: Automatically label specific behaviors of interest in videos from sectors like manufacturing, fishery, and transportation. This creates searchable, structured labels.

How it works

Define event labels: Enable user-defined labels in the DatasetConfig of the target Dataset. For each event, configure a name (Name) and a trigger description (Description).
Detect events on ingest: After a video is uploaded and routed to the Dataset, OSS asynchronously performs video understanding to match events in the video against the label descriptions.
Write to metadata: OSS writes the matched event labels to the file's metadata (Labels) within the Dataset.
Retrieve by label: Use SimpleQuery to search for files that match an event by querying Labels.LabelName. You can also retrieve matched labels through MNS notifications.

Note

Video understanding runs asynchronously. It may take some time for event labels to be generated after a video is uploaded. During this period, the file may not be returned in queries.

Limitations

The caller must have permissions such as oss:MetaQuery and oss:PutObject.
If you need to receive notifications when file indexing is complete, create an MNS queue in advance and ensure that the RAM role used by OpenMetaQuery has permissions to send messages to this queue.
To route different objects to different Datasets, you need to configure a RouteRule when you enable MetaQuery.

Step 1: Enable MetaQuery

Enable metadata management for a bucket in semantic mode. The following examples use OSS Java SDK v2 (com.aliyun:alibabacloud-oss-v2) and OSS Go SDK v2 (github.com/aliyun/alibabacloud-oss-go-sdk-v2), and read access credentials from the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables.

Java SDK

import com.aliyun.sdk.service.oss2.OSSClient;
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.models.OpenMetaQueryRequest;
import com.aliyun.sdk.service.oss2.models.OpenMetaQueryResult;

public class OpenMetaQuerySample {
    public static void main(String[] args) throws Exception {
        try (OSSClient client = OSSClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(new EnvironmentVariableCredentialsProvider())
                .build()) {
            OpenMetaQueryResult result = client.openMetaQuery(OpenMetaQueryRequest.newBuilder()
                    .bucket("examplebucket")
                    .mode("semantic")
                    .build());
            System.out.println("open meta query status: " + result.statusCode());
        }
    }
}

Go SDK

package main

import (
	"context"
	"log"

	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
)

func main() {
	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion("cn-hangzhou")

	client := oss.NewClient(cfg)

	_, err := client.OpenMetaQuery(context.TODO(), &oss.OpenMetaQueryRequest{
		Bucket: oss.Ptr("examplebucket"),
		Mode:   oss.Ptr("semantic"),
	})
	if err != nil {
		log.Fatalf("open meta query failed: %v", err)
	}
	log.Println("open meta query success")
}

Note

To configure multi-Dataset routing (RouteRule), MNS notifications for indexing completion (NotificationAttributes), and a default bucket-level DatasetConfig, you must specify them together in the OpenMetaQuery request body. For field descriptions, see OpenMetaQuery. To return event labels in notification messages, add Labels to WithFields.

Step 2: Create a dataset and configure labels

Call CreateDataset to create a Dataset and configure custom event labels using DatasetConfig. Set Insights.Video.Label.UserDefined.Enable to true and specify the label list in Labels. The following describes each field.

Parameter	Description
Name	The label name. You use this name to query for matching files. Example: `PersonFalling`.
Description	The trigger description for the label, which serves as a semantic prompt for the AI. Provide a clear description of the action, scene, and key details. Example: `A person in the frame transitions from a standing or walking position to a fallen position on the ground`.

Java SDK

import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.dataprocess.OSSDataProcessClient;
import com.aliyun.sdk.service.oss2.dataprocess.models.CreateDatasetRequest;

public class CreateDatasetSample {
    public static void main(String[] args) throws Exception {
        String datasetConfig = "{\n" +
                "  \"Insights\": {\n" +
                "    \"Video\": {\n" +
                "      \"Label\": {\n" +
                "        \"UserDefined\": {\n" +
                "          \"Enable\": true,\n" +
                "          \"Labels\": [\n" +
                "            {\"Name\": \"PersonFalling\", \"Description\": \"A person in the frame transitions from a standing or walking position to a fallen position on the ground\"},\n" +
                "            {\"Name\": \"CastingNet\", \"Description\": \"A person by the water or on a boat casts a net to catch fish\"}\n" +
                "          ]\n" +
                "        }\n" +
                "      }\n" +
                "    }\n" +
                "  }\n" +
                "}";
        try (OSSDataProcessClient client = OSSDataProcessClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(new EnvironmentVariableCredentialsProvider())
                .build()) {
            client.createDataset(CreateDatasetRequest.newBuilder()
                    .bucket("examplebucket")
                    .datasetName("security-events")
                    .datasetConfig(datasetConfig)
                    .build());
            System.out.println("create dataset success");
        }
    }
}

Go SDK

package main

import (
	"context"
	"log"

	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/dataprocess"
)

func main() {
	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion("cn-hangzhou")

	client := dataprocess.NewClient(cfg)

	datasetConfig := `{
  "Insights": {
    "Video": {
      "Label": {
        "UserDefined": {
          "Enable": true,
          "Labels": [
            {"Name": "PersonFalling", "Description": "A person in the frame transitions from a standing or walking position to a fallen position on the ground"},
            {"Name": "CastingNet", "Description": "A person by the water or on a boat casts a net to catch fish"}
          ]
        }
      }
    }
  }
}`

	_, err := client.CreateDataset(context.TODO(), &dataprocess.CreateDatasetRequest{
		Bucket:        oss.Ptr("examplebucket"),
		DatasetName:   oss.Ptr("security-events"),
		DatasetConfig: oss.Ptr(datasetConfig),
	})
	if err != nil {
		log.Fatalf("create dataset failed: %v", err)
	}
	log.Println("create dataset success")
}

Note

For backward compatibility, you can also configure the basic custom label switch using WorkflowParameters. However, this method is intended only for existing configurations. Future features will not be supported through WorkflowParameters. For new integrations, use DatasetConfig.

Step 3: Upload and route a video

When you upload a video, set the OSS Object Tag to routing-dataset=security-events to route the file to the target Dataset. After file indexing is complete, the system matches events against the labels configured in the Dataset.

Java SDK

import com.aliyun.sdk.service.oss2.OSSClient;
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.models.PutObjectRequest;
import com.aliyun.sdk.service.oss2.transport.BinaryData;
import java.nio.file.Files;
import java.nio.file.Paths;

public class PutObjectSample {
    public static void main(String[] args) throws Exception {
        try (OSSClient client = OSSClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(new EnvironmentVariableCredentialsProvider())
                .build()) {
            client.putObject(PutObjectRequest.newBuilder()
                    .bucket("examplebucket")
                    .key("security/event-a.mp4")
                    .tagging("routing-dataset=security-events")
                    .body(BinaryData.fromStream(Files.newInputStream(Paths.get("/local/path/event-a.mp4"))))
                    .build());
            System.out.println("put object success");
        }
    }
}

Go SDK

package main

import (
	"context"
	"log"
	"os"

	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
)

func main() {
	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion("cn-hangzhou")

	client := oss.NewClient(cfg)

	file, err := os.Open("/local/path/event-a.mp4")
	if err != nil {
		log.Fatalf("open file failed: %v", err)
	}
	defer file.Close()

	_, err = client.PutObject(context.TODO(), &oss.PutObjectRequest{
		Bucket:  oss.Ptr("examplebucket"),
		Key:     oss.Ptr("security/event-a.mp4"),
		Tagging: oss.Ptr("routing-dataset=security-events"),
		Body:    file,
	})
	if err != nil {
		log.Fatalf("put object failed: %v", err)
	}
	log.Println("put object success")
}

(Optional) Attach temporary labels

To attach temporary labels not configured in the Dataset, you can provide a URL-encoded list of them in a custom metadata header during the upload. The HTTP header name is x-oss-meta-metaquery-user-defined-labels. When you use an SDK, set the metadata key to metaquery-user-defined-labels.

Java SDK

import com.aliyun.sdk.service.oss2.OSSClient;
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.models.PutObjectRequest;
import com.aliyun.sdk.service.oss2.transport.BinaryData;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Collections;

public class PutObjectWithLabelsSample {
    public static void main(String[] args) throws Exception {
        String labels = URLEncoder.encode(
                "[{\"Name\":\"PersonFalling\",\"Description\":\"A person in the frame transitions from a standing or walking position to a fallen position on the ground\"}]",
                StandardCharsets.UTF_8.name());
        try (OSSClient client = OSSClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(new EnvironmentVariableCredentialsProvider())
                .build()) {
            client.putObject(PutObjectRequest.newBuilder()
                    .bucket("examplebucket")
                    .key("security/event-x.mp4")
                    .tagging("routing-dataset=security-events")
                    .metadata(Collections.singletonMap("metaquery-user-defined-labels", labels))
                    .body(BinaryData.fromStream(Files.newInputStream(Paths.get("/local/path/event-x.mp4"))))
                    .build());
            System.out.println("put object success");
        }
    }
}

Go SDK

package main

import (
	"context"
	"log"
	"net/url"
	"os"

	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
)

func main() {
	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion("cn-hangzhou")

	client := oss.NewClient(cfg)

	file, err := os.Open("/local/path/event-x.mp4")
	if err != nil {
		log.Fatalf("open file failed: %v", err)
	}
	defer file.Close()

	labels := url.QueryEscape(`[{"Name":"PersonFalling","Description":"A person in the frame transitions from a standing or walking position to a fallen position on the ground"}]`)

	_, err = client.PutObject(context.TODO(), &oss.PutObjectRequest{
		Bucket:   oss.Ptr("examplebucket"),
		Key:      oss.Ptr("security/event-x.mp4"),
		Tagging:  oss.Ptr("routing-dataset=security-events"),
		Metadata: map[string]string{"metaquery-user-defined-labels": labels},
		Body:     file,
	})
	if err != nil {
		log.Fatalf("put object failed: %v", err)
	}
	log.Println("put object success")
}

Step 4: Query matched files

After file indexing is complete, call SimpleQuery to perform an exact match query for files within the specified Dataset by Labels.LabelName. If you configured MNS notifications when you enabled MetaQuery, you can also determine whether indexing is complete by consuming messages from an MNS queue and reading the matched Labels from the notification message.

Java SDK

import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.dataprocess.OSSDataProcessClient;
import com.aliyun.sdk.service.oss2.dataprocess.models.File;
import com.aliyun.sdk.service.oss2.dataprocess.models.Label;
import com.aliyun.sdk.service.oss2.dataprocess.models.SimpleQuery;
import com.aliyun.sdk.service.oss2.dataprocess.models.SimpleQueryRequest;
import com.aliyun.sdk.service.oss2.dataprocess.models.SimpleQueryResult;
import java.util.Arrays;

public class SimpleQueryLabelSample {
    public static void main(String[] args) throws Exception {
        try (OSSDataProcessClient client = OSSDataProcessClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(new EnvironmentVariableCredentialsProvider())
                .build()) {
            SimpleQueryResult result = client.simpleQuery(SimpleQueryRequest.newBuilder()
                    .bucket("examplebucket")
                    .datasetName("security-events")
                    .query(SimpleQuery.newBuilder()
                            .field("Labels.LabelName")
                            .value("PersonFalling")
                            .operation("eq")
                            .build())
                    .withFields(Arrays.asList("Filename", "URI", "Labels"))
                    .maxResults(20)
                    .build());
            for (File file : result.files()) {
                StringBuilder names = new StringBuilder();
                if (file.labels() != null) {
                    for (Label label : file.labels()) {
                        names.append(label.labelName()).append(" ");
                    }
                }
                System.out.println(file.filename() + " -> " + names);
            }
        }
    }
}

Go SDK

package main

import (
	"context"
	"log"

	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/dataprocess"
)

func main() {
	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion("cn-hangzhou")

	client := dataprocess.NewClient(cfg)

	result, err := client.SimpleQuery(context.TODO(), &dataprocess.SimpleQueryRequest{
		Bucket:      oss.Ptr("examplebucket"),
		DatasetName: oss.Ptr("security-events"),
		Query:       oss.Ptr(`{"Field":"Labels.LabelName","Value":"PersonFalling","Operation":"eq"}`),
		WithFields:  oss.Ptr(`["Filename","URI","Labels"]`),
		MaxResults:  oss.Ptr(int32(20)),
	})
	if err != nil {
		log.Fatalf("simple query failed: %v", err)
	}
	for _, file := range result.Files {
		log.Printf("%s %v", oss.ToString(file.Filename), file.Labels)
	}
}

In the response, the Labels.Label.LabelName field of each matched file contains the matched business label. For example:

<File>
  <Filename>security/event-a.mp4</Filename>
  <Labels>
    <Label>
      <LabelName>PersonFalling</LabelName>
    </Label>
  </Labels>
</File>

Related APIs

Enable metadata management: OpenMetaQuery
Create a Dataset and configure event labels: CreateDataset
Query for matched files by label: SimpleQuery
Data structure for event label configuration: DatasetConfig