Customize moderation policies

Procedure

1. Log on to the Content Moderation console.

2. In the left-side navigation pane, choose Machine Audit V1.0 > Settings.

3. Create a business scenario.

   If you have already created a business scenario, skip this step.

   1. On the BizType Manage tab of the Machine Moderation page, click Create BizType.

   2. In the Create BizType dialog box, configure the parameters as described in the following table.

      Parameter	Description
      BizType name	The name of the business scenario. It can contain digits, letters, and underscores (_) and must be no more than 32 characters long.
      Industry template configuration	A policy configuration that follows industry standards. If you use an industry template, some features of the moderation policy cannot be customized. If you do not use a template, you can configure all settings yourself.
      Industry Category	The industry to which your business belongs. Specifying an industry helps improve the accuracy of the policy configuration.
      Copy from	If you have already created a business scenario, you can import its configuration.
      Description	A description of the business scenario. It can contain Chinese characters, English letters, digits, and underscores (_) and must be no more than 32 characters long.

   3. Click OK to create the business scenario.

4. Configure the business scenario.

   You can configure the Moderation Policy, Audit Data, and Evidence Storage.

   • Moderation Policy

     The moderation policies vary based on the detection scenarios. The options displayed in the console are definitive.

     1. On the Machine Moderation page, find the target business scenario and click Edit in the Actions column.

     2. Select the moderation policy that you want to set based on your requirements.

        You can set moderation policies for images, videos, text, and audio. For images and videos, the policies cover pornography, terrorism-related content, undesirable scenes, and text embedded in images. For text and audio, moderation policies cover content related to pornography, politics, abuse, advertising, and prohibited content.

        If a specific detection scenario is not relevant to your business, you do not need to configure a policy for it. For example, if you do not need to scan for advertisements, do not configure ad-related settings in the moderation policy.

        Note

        When you set the moderation policy for images, you can use the Apply Libraries and Associated Text Library options on the right to link configurations from your risk libraries.

     3. Click Save to finish customizing the moderation policy.

        When you call an API, you must pass the corresponding business scenario for the custom moderation policy to take effect. For example, when you scan images for pornographic content (for more information, see synchronous scan), you must set the request parameter scene to porn.

   • Audit Data

     1. Click Audit Data.

     2. On the Audit Data tab, select which data types to send for human review.

        For more information about human review, see human review.

   • Evidence Storage

     When you use the Content Moderation API, you can enable evidence storage to save the results from machine-assisted moderation. This feature uses Alibaba Cloud Object Storage Service (OSS) to store video, audio, and image content identified as non-compliant, suspicious, or compliant in a bucket that you specify. The service then returns an OSS URL for each stored file. This section describes how to enable and configure evidence storage.

     Evidence storage is supported only for video, audio, and image scans. This applies to detected content for which the suggestion from machine-assisted moderation is block (non-compliant), review (suspicious), or pass (compliant).

     Evidence storage is disabled by default. To use this feature, you must enable and configure storage for video, audio, or image evidence.

     • After you enable video evidence storage, video files (including video streams) and video snapshots flagged as non-compliant, suspicious, or compliant are stored in your configured bucket.

     • After you enable audio evidence storage, audio files (including audio streams) and audio fragments flagged as non-compliant, suspicious, or compliant are stored in your configured bucket.

     • After you enable image evidence storage, image files flagged as non-compliant, suspicious, or compliant are stored in your configured bucket.

     1. Click Evidence Storage.

     2. On the Evidence Storage tab, enable Video Evidence Storage, Audio Evidence Storage, or Image evidence transfer as needed, and complete the related configurations.

        Note

        If this is your first time using this feature, follow the on-screen prompts to grant Content Moderation read and write permissions on Object Storage Service (OSS). This action reuses the existing authorization for OSS violation detection. If authorization is already granted, the configuration page is displayed directly.

        Parameter	Description
        Storage bucket	The Object Storage Service (OSS) bucket where evidence files are stored.
        Storage Directory	The user-defined directory where evidence files are stored. All evidence files are stored in this directory according to the storage rules. For more information, see storage rules. Note If the specified directory does not exist in the bucket, it is automatically created.
        访问链接有效期	The validity period of the access URL generated for each stored evidence file. Valid values: 300 to 3,600. Unit: seconds.
        Transfer range	The following storage scopes are supported: Machine identification violation: Stores content with a moderation result of block. Machine identification is suspected: Stores content with a moderation result of review. The machine identification is normal.: Stores content with a moderation result of pass.

     3. Click Save.

        After you successfully configure evidence storage, the next time you call the video moderation, audio moderation, or image moderation service and it detects non-compliant, suspicious, or normal content, you will find the OSS URL of the stored file in the response parameters (data.extras.newUrl for video moderation, data.new_url and data.result.details.url for audio moderation, and data.storedUrl for image moderation). You can also go to the OSS console to view the stored files in the bucket.

        Storage rules

        Table 1. Video storage rules

        File type	Storage path	Naming convention
        Video file	${bucket}/${user_directory}/video/${suggestion}/${taskId}/${video_name.suffix}	Stored video files are named after the original video files.
        Video snapshot	${bucket}/${user_directory}/video/${suggestion}/${taskId}/frames/${snapshot_name.suffix}	The saved snapshot images are named after the time they are captured. For example, 00_01_02 represents a snapshot captured at 00 hours, 01 minute, and 02 seconds.
        Video stream	${bucket}/${user_directory}/video/${suggestion}/${taskId}/${stream_name.suffix}	Saved video streams are named based on the time when detection starts. For example, the filename 20190102_12_02_03.wav indicates that detection for the video stream started at 12:02:03 on January 2, 2019.

        Table 2. Audio storage rules

        File type	Storage path	Naming convention
        Audio file	${bucket}/${user_directory}/audio/${suggestion}/${taskId}/${audio_name.suffix}	Stored audio files are named after the original audio files.
        Audio fragment	${bucket}/${user_directory}/audio/${suggestion}/${taskId}/slice/${fragment_name.suffix}	Saved audio fragments are named based on their start and end times. For example, 00_01_02-00_10_13.mp3 represents the audio segment from 00 hours, 01 minute, and 02 seconds to 00 hours, 10 minutes, and 13 seconds.

        Table 3. Image storage rules

        File type	Storage path	Naming convention
        Image file	${bucket}/${user_directory}/image/${suggestion}/${taskId}/${image_name.suffix}	Stored image files are named after the original image files.