Content Moderation

Background

The content moderation feature is powered by the AI algorithms and domain-specific knowledge base of the Alibaba Cloud Security team. These assets are continuously updated. Compared with manual review and labeling, this feature offers faster content analysis, higher task concurrency, and lower costs. This makes it ideal for managing large volumes of user-generated content (UGC) and AI-generated content (AIGC).

OSS Content Moderation supports the analysis and classification of over 90 content risk categories. You can use it to scan image files in a bucket to identify content or elements that violate web content regulations, disrupt platform order, or negatively affect user experience.

You can adjust the detection rules based on your industry's standards or your platform's content governance policies. You can also develop further review or governance measures for specific content based on the risk labels and confidence scores provided by the system.

Detection scope

Prerequisites

A RAM user must have the oss:ActivateProduct permission. For more information, see Grant custom permissions to a RAM user.

Billing

• One scan of an object can detect multiple risk categories, such as pornography, politically sensitive content, terrorism, and violence. The service is priced at CNY 5 per 10,000 scans. The total scan count depends on how many detection services you select per image.

  For example, if you select only the General Baseline Detection service for an object, it counts as one scan. If you select both General Baseline Detection and AIGC Risk Detection, it counts as two scans.

• This service supports only the pay-as-you-go billing method and is billed hourly. Bills are generated after each billing cycle, though the exact billing time is subject to system processing.

Notes

The OSS Content Moderation feature scans images in a bucket, analyzes them for risks, and applies corresponding labels. The service does not modify or move the images.

Limitations

• Region restrictions

  Content Moderation is available in the following regions: China (Hangzhou), China (Shanghai), China (Nanjing), China (Fuzhou), China (Wuhan), China (Qingdao), China (Beijing), China (Zhangjiakou), China (Hohhot), China (Ulanqab), China (Shenzhen), China (Heyuan), China (Guangzhou), and China (Chengdu).

• Bucket storage class restrictions

  Only buckets of the standard storage class support content moderation. Buckets of the infrequent access, archive, cold archive, and deep cold archive storage classes are not supported.

• Object storage class restrictions

  • Objects of the standard and infrequent access storage classes can be scanned directly.

  • Archive objects must be restored before they can be scanned, unless Archive Instant Access is enabled. If Archive Instant Access is enabled, you can scan archive objects directly. For more information, see Archive Instant Access and Restore objects.

  • Cold archive and deep cold archive objects must be restored before they can be scanned.

• Image format restrictions

  Only PNG, JPG, JPEG, BMP, WEBP, TIFF, SVG, ICO, and HEIC formats are supported.

• Image size restrictions

  For the supported formats, images other than HEIC should be no larger than 10 MB. We recommend a minimum resolution of 200x200 pixels. Low-resolution images may affect the accuracy of the content moderation algorithm. For HEIC images, the longest side must be less than 8,192 pixels.

Use the OSS console

1. To use Content Moderation for the first time, you must activate the service and grant the required role permissions.

   1. Log on to the OSS console.

   2. In the left-side navigation pane, click Buckets. On the Buckets page, find and click the desired bucket.

   3. In the left-side navigation pane, choose Content Detection > Content Security.

   4. On the Content Security page, click the Detection Task tab. Click Activate and follow the on-screen instructions to start your free trial. After the free trial is used up, the service is billed on a pay-as-you-go basis.

   5. After you activate the service, on the Content Security page, click the Detection Task tab, and then click Authorize OSS and Log Service. Follow the on-screen instructions to grant the AliyunOSSCIPScanOSSRole role.

      Authorization takes 1 to 2 minutes. By default, the granted role has permission to access objects in your bucket and create the Log Service Project and Logstore needed to store detection results.

2. Configure detection rules.

   1. On the Content Security page, click the Detection Rule tab.

   2. On the Detection Rule tab, select a detection service and click Set Rule in the Actions column.

   3. On the Rule Configuration page, click Edit, and then enable or disable detection items based on your business needs.

   4. Click Save. The rule configuration takes effect in about 2 to 3 minutes.

3. Create a detection task.

   1. Select the task type.

      1. On the Content Security page, click the Detection Task tab.

      2. On the Detection Task tab, click Create Batch Task.

      3. In the Create Batch Task dialog box, modify the task name, and select a task type and detection service.

         The smart format detection option is enabled by default. When enabled, the detection task automatically identifies image files based on their Content-Type.

      4. Click Next.

   2. (Optional) Specify the detection scope and click Next.

      By default, the task scans all objects in the bucket. You can filter the objects to be scanned by time range or file prefix.

   3. (Optional) Select the specifications and click Submit.

      By default, all objects are scanned asynchronously in a queue. You can also set a maximum number of objects to scan.

      Note

      An asynchronous task with millions of files typically completes within 24 hours. This may extend to 48 hours for tens of millions of files, or 72 hours for hundreds of millions.

4. After the task is complete, view the detection results.

   Up to 50,000 results

   If you have fewer than 50,000 result entries, view them directly in the OSS Management Console.

   1. On the Detection Task tab, find the target task and click View Results in the Actions column.

   2. On the Query Result page, you can filter and search by time range, file name (Object), or text information.

      You can query up to 50,000 entries from the last 180 days. The results are saved to a log bucket. The results table includes the File Name, File MD5, Detection Service, Detection Result, and Operation columns. The detection result might be a label such as nonLabel (normal) or political_flag (politically sensitive flag). Click View to see detailed results.

   Over 50,000 results

   All detection results are automatically sent to a dedicated Logstore within a Project in Log Service. The Project is named in the format aliyun-cip-<Alibaba Cloud account ID>-cn-shanghai, and the dedicated Logstore is named oss-cip-osscheck-log. If you have more than 50,000 result entries, view them in Log Service.

   1. Log on to the Simple Log Service console.

   2. In the Project list, click the target Project.

   3. On the Log Storage > Logstores tab, click oss-cip-osscheck-log.

   4. View the logs delivered from OSS content moderation to Log Service.

      The following table describes the log fields.

      Field	Type	Indexed	Description
      bucket	String	Yes	The name of the OSS bucket.
      code	Integer	Yes	The error code of the detection request.
      contentType	String	Yes	The file type.
      jobId	String	Yes	The ID of the detection task.
      jobName	String	Yes	The name of the detection task.
      labels	String	Yes	The labels returned by the content moderation service. Multiple labels are separated by commas (,).
      labels2	String	Yes	Additional labels returned by other detection types, if any. Multiple labels are separated by commas (,).
      md5	String	Yes	The MD5 hash of the object.
      msg	String	Yes	The error message of the detection request.
      object	String	Yes	The name of the object.
      parentJobId	String	Yes	The ID of the parent task, used for scheduled detection tasks.
      regionId	String	Yes	The region of the OSS bucket.
      requestId	String	Yes	The ID of the detection request.
      scanResult	String	Yes	The detailed detection results in a JSON string.
      serviceCode	String	Yes	The configured detection service.
      subUid	String	Yes	The RAM user ID.
      uid	String	Yes	The Alibaba Cloud account ID.

   5. After Log Service collects the Content Moderation results, you can perform operations such as query and analysis, download, delivery, data transformation, and alert creation to understand content risk trends and perform real-time monitoring. For more information, see Common operations for cloud service logs.

   For a description of the labels that correspond to different detection services, see Risk label definitions.

5. Process images whose results are not nonLabel.

   1. If you want to keep the image, apply a watermark or a blur effect. For more information, see Image watermarks and Blur effects.

   2. If you do not want to keep the image, we recommend deleting it. For more information, see Delete objects.

Related documents

For advanced Content Moderation requirements, such as scanning audio, video, text, and document files in OSS or on-premises, see Content Moderation Pro.