Multimodal async moderation API integration guide-AI Guardrails(AI Guardrails)-阿里云帮助中心

The AI Safety Guardrail multimodal async moderation API moderates video, audio, and image content through an asynchronous submit-and-poll workflow. You submit a moderation task by calling MultiModalGuardAsync, receive a task ID, and then poll for results by calling MultiModalGuardAsyncResult at a recommended interval of 30 seconds. Results are available for up to 24 hours after task submission.

Before you begin

Step 1: Activate the service

Go to the AI Safety Guardrail service activation page to activate AI Safety Guardrail.

Step 2: Authorize RAM users

Before you integrate the SDK or API, authorize your RAM users. Create an AccessKey pair for your Alibaba Cloud account or RAM user. You need the AccessKey pair to complete identity verification when you call Alibaba Cloud APIs. For how to obtain an AccessKey pair, see Obtain an AccessKey pair.

Grant permissions to a RAM user

Log on to the RAM console using your Alibaba Cloud account.
Create a RAM user. For details, see Create a RAM user.
Grant the AliyunYundunGreenWebFullAccess system policy to the RAM user. This policy grants full access to Content Moderation. For details, see Manage RAM user permissions.
The RAM user can now call the Content Moderation API.

Step 3: Install and integrate the SDK

For the AI Safety Guardrail SDK, see Multimodal SDK integration.

API reference

Submit a moderation task

Call this API operation to submit an async content moderation task.

API operation: MultiModalGuardAsync
Supported regions and endpoints:

Region	Public endpoint	VPC endpoint
China (Shanghai)	https://green-cip.cn-shanghai.aliyuncs.com	https://green-cip-vpc.cn-shanghai.aliyuncs.com
China (Beijing)	https://green-cip.cn-beijing.aliyuncs.com	https://green-cip-vpc.cn-beijing.aliyuncs.com
China (Hangzhou)	https://green-cip.cn-hangzhou.aliyuncs.com	https://green-cip-vpc.cn-hangzhou.aliyuncs.com
China (Shenzhen)	https://green-cip.cn-shenzhen.aliyuncs.com	https://green-cip-vpc.cn-shenzhen.aliyuncs.com
China (Chengdu)	https://green-cip.cn-chengdu.aliyuncs.com	N/A
Singapore	https://green-cip.ap-southeast-1.aliyuncs.com	https://green-cip-vpc.ap-southeast-1.aliyuncs.com

Billing: This is a paid API. Billing is based on your configured video frame detection policy and video audio detection policy. You can select multiple services for video frame detection. Fees are accumulated by multiplying the number of frame captures by the unit price of each service. If you also detect audio content violations in the video, an additional fee is calculated as the video duration multiplied by the unit price of the audio violation detection feature. For billing details, see Activation and billing overview.
QPS limit: The per-user QPS limit for this API operation is 100 requests per second, and the concurrent moderation task limit is 50 tasks (a maximum of 50 tasks can be processed at the same time. To increase the concurrency limit, contact your sales representative). Requests that exceed the limit are throttled, which may affect your business. Call this API at a reasonable rate.

Request parameters

Name	Type	Required	Example	Description
Service	String	Yes	video_security_check	AIGC Video Content Moderation (video_security_check), AIGC Audio Content Moderation (audio_security_check), Multi-Image AI Input Content Moderation - Overseas Edition (query_security_check_cb), Multi-Image AI-Generated Content Moderation - Overseas Edition (response_security_check_cb), Multi-Image AIGC Input Image Moderation (img_query_security_check), Multi-Image AIGC Output Image Moderation (img_response_security_check)
ServiceParameters	JSONString	Yes		The set of parameters required by the moderation service. This is a JSON string. For a description of each parameter, see ServiceParameters below.

Table 1. ServiceParameters

Name	Type	Required	Example	Description
url	String	Yes. Enhanced audio and video moderation supports two methods for submitting audio and video files. Select one of the following methods: submit the video URL for detection by passing in url, or use OSS authorization for detection by passing in ossBucketName, ossObjectName, and ossRegionId simultaneously. Multi-image detection supports only image URL.	http://www.aliyundoc.com/a.flv	The URL of the object to be moderated. Make sure that the URL is publicly accessible, or pass in an OSS internal endpoint URL in the same region. Note The URL must not contain Chinese characters, must be no more than 2,048 characters in length, and only one URL can be passed in per request.
ossBucketName	String		bucket_01	The name of the authorized OSS bucket. Note To use an OSS internal video URL, you must first use your Alibaba Cloud account (the primary account) to visit the cloud resource authorization page to grant the required permissions.
ossObjectName	String		20240307/07/28/test.flv	The file name in the authorized OSS bucket.
ossRegionId	String		cn-shanghai	The region where the OSS bucket is located.
ImageUrls	List\<String>		["http://www.aliyundoc.com/1.jpg","http://www.aliyundoc.com/2.jpg"]	The URLs of the images to be moderated. Make sure that the URLs are publicly accessible.
callback	String	No	http://www.aliyundoc.com	The URL to receive callback notifications with moderation results. Both HTTP and HTTPS addresses are supported. If this field is empty, you must periodically poll for moderation results. The callback endpoint must support the POST method, UTF-8 encoded data, and the form parameters checksum and content. Content Security sets checksum and content according to the following rules and calls your callback endpoint to return moderation results. checksum: A string generated by concatenating `user UID + seed + content` and computing the SHA256 hash. The user UID is your Alibaba Cloud account ID, which you can find in the Alibaba Cloud console. To prevent tampering, you can generate the string using the same algorithm when you receive a push result and verify it against the checksum. Note: The user UID must be the Alibaba Cloud account UID, not the RAM user UID. content: A JSON string. Parse and deserialize it into a JSON object. For examples of content results, see the response examples of the query moderation result API. When your server callback endpoint receives a moderation result pushed by Content Security, an HTTP 200 status code indicates successful receipt. Any other HTTP status code is treated as a failure. On failure, Content Security retries up to 16 times until receipt is successful. If receipt still fails after 16 retries, no further pushes are sent. Check the status of your callback endpoint.
seed	String	No	abc****	A random string used for signing callback notification requests. Consists of letters, digits, and underscores (_), with a maximum length of 64 characters. You define this value to verify that callback notifications are sent by Alibaba Cloud Content Security. Note This field is required when callback is used.
cryptType	String	No	SHA256	The signing algorithm for callback notification content. Content Security computes a signature by applying the algorithm you specify to the string formed by concatenating user UID + seed + content, then sends the result to your callback URL. Valid values: SHA256 (default): Uses the SHA256 encryption algorithm. SM3: Uses the Chinese national standard HMAC-SM3 encryption algorithm, which returns a hexadecimal string of lowercase letters and digits. For example, after SM3 encryption, abc returns 66c7f0f462eeedd9d1f2d46bdc10e4e24167c4875cf2f7a2297da02b8f4ba8e0.
dataId	String	No	dataId****	The data ID corresponding to the moderation object. Consists of uppercase and lowercase letters, digits, underscores (_), hyphens (-), and periods (.), with a maximum length of 128 characters. Use this value to uniquely identify your business data.

Response parameters

Name	Type	Example	Description
Code	Integer	200	The status code. For more information, see Code reference.
Data	JSONObject	{"TaskId":""}	The task submission result data. For details, see Data.
Message	String	OK	The response message.
RequestId	String	AAAAAA-BBBB-CCCCC-DDDD-EEEEEEEE****	The request ID.

Table 2. Data

Name	Type	Example	Description
DataId	String	dataId****	The data ID.
TaskId	String	AAAAA-BBBBB	The moderation task ID.

Examples

Request example:

{
    "Service": "XXX",
    "ServiceParameters": {
        "url": "http://www.aliyundoc.com/a.flv",
        "dataId": "videoId****"
    }
}

Success response example:

{
    "Message": "OK",
    "Code": 200,
    "Data": {
        "TaskId": "AAAAA-BBBBB",
        "DataId": "videoId****"
    },
    "RequestId": "ABCD1234-1234-1234-1234-123****"
}

Get moderation task results

Call this API operation to get async task results.

API operation: MultiModalGuardAsyncResult
Billing: This API operation is free of charge.
Query timeout: Set the query interval to 30 seconds (query results 30 seconds after submitting the async moderation task). The maximum timeout is 24 hours. Results are automatically deleted after this period.
QPS limit: The per-user QPS limit for this API operation is 100 requests per second. Requests that exceed the limit are throttled, which may affect your business. Call this API at a reasonable rate.

Request parameters

Name	Type	Required	Example	Description
Service	String	Yes	video_security_check	AIGC Video Content Moderation (video_security_check), AIGC Audio Content Moderation (audio_security_check)
ServiceParameters	JSONString	Yes		The set of parameters required by the moderation service. This is a JSON string. For a description of each parameter, see ServiceParameters below.

Table 1. ServiceParameters

Name	Type	Required	Example	Description
taskId	String	Yes	abcd****	The taskId of the moderation task to query. Only one taskId can be passed in per request. You can obtain the taskId from the response data after submitting a moderation task.

Response data

Name	Type	Example	Description
Code	Integer	200	The status code. For more information, see Code reference.
Data	JSONObject	{"TaskId":""}	The query task result data. For details, see Data.
Message	String	OK	The response message.
RequestId	String	AAAAAA-BBBB-CCCCC-DDDD-EEEEEEEE****	The request ID.

Table 2. Data

Name	Type	Example	Description
Suggestion	String	Pass	The moderation suggestion. block: Recommend blocking. pass: Recommend passing. watch: Recommend monitoring. mask: Recommend masking. Note Currently, only sensitive content detection supports monitoring and masking. Other detection dimensions support only blocking or passing. When you detect multiple dimensions, the Suggestion results of each dimension are merged. The merge priority from highest to lowest is: block, mask, watch, pass.
DataId	String	dataId****	The data ID corresponding to the moderation object. Consists of uppercase and lowercase letters, digits, underscores (_), hyphens (-), and periods (.), with a maximum length of 128 characters. Use this value to uniquely identify your business data.
TaskId	String		The moderation task ID.
AudioResult	object		The video frame detection result. When the call is successful (code=200), the response contains a structure. For the structure details, see AudioResult.
FrameResult	object		The audio detection result in audio/video. When the call is successful (code=200), the response contains a structure. For the structure details, see FrameResult.

Table 3. AudioResult

Name	Type	Example	Description
Suggestion	String	Pass	The moderation suggestion. block: Recommend blocking. pass: Recommend passing. watch: Recommend monitoring. mask: Recommend masking. Note Currently, only sensitive content detection supports monitoring and masking. Other detection dimensions support only blocking or passing. When you detect multiple dimensions, the Suggestion results of each dimension are merged. The merge priority from highest to lowest is: block, mask, watch, pass.
SliceNum	Integer	5	The number of audio slices returned.
SliceDetails	JSONArray		The text details of each audio segment (one element per segment). Contains one or more elements. For the structure details, see SliceDetail.

Table 4. SliceDetail

Name	Type	Example	Description
StartTime	Integer	0	The start time of the sentence, in seconds.
EndTime	Integer	20	The end time of the sentence, in seconds.
Url	String	https://aliyundoc.com/test.wav	If the moderated content is audio, this is the temporary access URL of the audio slice corresponding to the text segment. This URL is valid for 30 minutes. Save the content promptly.
Text	String		The text transcribed from the audio.
Suggestion	String	Pass	The moderation suggestion. block: Recommend blocking. pass: Recommend passing. watch: Recommend monitoring. mask: Recommend masking. Note Currently, only sensitive content detection supports monitoring and masking. Other detection dimensions support only blocking or passing. When you detect multiple dimensions, the Suggestion results of each dimension are merged. The merge priority from highest to lowest is: block, mask, watch, pass.
Detail	JSONArray		The detection details of the audio slice. Contains one or more elements. For the structure details, see Detail.

Table 5. FrameResult

Name	Type	Example	Description
Suggestion	String	Pass	The moderation suggestion. block: Recommend blocking. pass: Recommend passing. watch: Recommend monitoring. mask: Recommend masking. Note Currently, only sensitive content detection supports monitoring and masking. Other detection dimensions support only blocking or passing. When you detect multiple dimensions, the Suggestion results of each dimension are merged. The merge priority from highest to lowest is: block, mask, watch, pass.
SliceNum	Integer	5	The number of video frame captures returned.
Frames	JSONArray		The frame capture details of the video (one element per frame). Contains one or more elements. For the structure details, see Frame.

Table 6. Frame

Name	Type	Example	Description
Offset	Float	50.5	The timestamp of the video frame capture from the beginning of the video, in seconds.
Url	String	http://www.aliyundoc.com/test.jpg	The temporary URL of the video frame capture. Valid for 30 minutes.
Suggestion	String	Pass	The moderation suggestion. block: Recommend blocking. pass: Recommend passing. watch: Recommend monitoring. mask: Recommend masking. Note Currently, only sensitive content detection supports monitoring and masking. Other detection dimensions support only blocking or passing. When you detect multiple dimensions, the Suggestion results of each dimension are merged. The merge priority from highest to lowest is: block, mask, watch, pass.
Detail	JSONArray		The detection details of the video frame capture. Contains one or more elements. For the structure details, see Detail.
ErrorMsg	String	success	The detection message for the video frame capture.
ErrorCode	Integer	200	The detection code for the video frame capture.

Table 7. Detail

Name	Type	Example	Description
Suggestion	String	pass	The moderation suggestion. block: Recommend blocking. pass: Recommend passing. watch: Recommend monitoring. mask: Recommend masking. Note Currently, only sensitive content detection supports monitoring and masking. Other detection dimensions support only blocking or passing. When you detect multiple dimensions, the Suggestion results of each dimension are merged. The merge priority from highest to lowest is: block, mask, watch, pass.
Type	String	contentSecurity	The protection dimension. contentModeration: Content compliance detection. promptAttack: Prompt attack detection. sensitiveData: Sensitive content detection. modelHallucination: Model hallucination detection. maliciousFile: Malicious file detection. maliciousUrl: Malicious URL detection. waterMark: Digital watermark. customLabel: Custom detection agent.
Level	String	high	The risk level, returned based on the configured high and low risk thresholds. Valid values: high: High risk (if a custom vocabulary is matched, the risk level defaults to high risk). medium: Medium risk. low: Low risk. none: No risk detected. Note High-risk content should be handled directly. Medium-risk content should be reviewed manually. Low-risk content should only be processed for high-recall requirements. In normal cases, handle low-risk content the same as no-risk content. Risk score thresholds can be configured in the AI Safety Guardrail console. Sensitivity level (for SensitiveData). Valid values: S0, S1, S2, S3. S0 indicates no sensitive content detected. Higher numbers indicate higher sensitivity.
Result	JSONArray		The content compliance risk labels, confidence scores, and other results. For details, see Result.

Table 8. Result

Name	Type	Example	Description
Description	String	Suspected political entity	The description of the Label field. Important This field provides an explanation of the Label value and may be adjusted. When processing results, use the Label field rather than basing your handling on this field.
Confidence	Float	81.22	The confidence score, ranging from 0 to 100 with up to 2 decimal places. Some labels do not have a confidence score.
Label	String	political_xxx	The label returned after text content detection. Multiple labels and scores may be detected.
Level	String	high	The risk level, returned based on the configured high and low risk thresholds. Valid values: high: High risk (if a custom vocabulary is matched, the risk level defaults to high risk). medium: Medium risk. low: Low risk. none: No risk detected. Note High-risk content should be handled directly. Medium-risk content should be reviewed manually. Low-risk content should only be processed for high-recall requirements. In normal cases, handle low-risk content the same as no-risk content. Risk score thresholds can be configured in the AI Safety Guardrail console. Sensitivity level (for SensitiveData). Valid values: S0, S1, S2, S3. S0 indicates no sensitive content detected. Higher numbers indicate higher sensitivity.
Ext	JSONObject		Some protection dimensions return corresponding extended information. For details, see Ext.

Table 9. Ext

Name	Type	Example	Description
Riskwords	String	AA,BB,CC	Applicable protection dimension: contentModeration (content compliance detection). The detected sensitive words, separated by commas. Some labels do not return sensitive words.
CustomizedHit	JSONArray	[{"LibName":"...","Keywords":"..."}]	Applicable protection dimension: contentModeration (content compliance detection). When a custom library is matched, the Label is customized, and the custom library name and custom words are returned.
SensitiveData	JSONArray	["6201112223455"]	Applicable protection dimension: sensitiveData (sensitive content detection). The detected sensitive data samples.
Desensitization	String	... [Mobile phone number] is my contact ...	Applicable protection dimension: sensitiveData (sensitive content detection). The desensitized content.
FileUrl	String	https://sase-public-server-files.oss-cn-hangzhou.aliyuncs.com/saas-XXX	Applicable protection dimension: waterMark (digital watermark). The download link for the watermarked file.
OutFileSize	String	152357	Applicable protection dimension: waterMark (digital watermark). The file size.
FileUrlExp	String	1754135551	Applicable protection dimension: waterMark (digital watermark). The expiration time of the watermarked file download link.
Filename	String	B7VKehJ4gZR.png	Applicable protection dimension: waterMark (digital watermark). The file name.
OutFileHashMd5	String	8b96ff73e8d8060016bb41b16d337871	Applicable protection dimension: waterMark (digital watermark). The MD5 hash of the file.

Table 10. CustomizedHit

Name	Type	Example	Description
LibName	String	Custom library 1	The name of the custom library.
Keywords	String	Custom word 1, custom word 2	The custom words, separated by commas.

Examples

Request example:

{
    "Service": "video_security_check",
    "ServiceParameters": {
        "taskId": "abcd****"
    }
}

Response example (detecting video_security_check, matching system policy):

{
  "code": 200,
  "data": {
    "AudioResult": {
      "SliceNum": 1,
      "Suggestion": "block",
      "sliceDetails": [{
        "Detail":[
          {
            "level":"none",
            "result":[
              {
                "confidence":0,
                "description":"No risk detected",
                "label":"nonLabel",
                "level":"none"
              }
            ],
            "suggestion":"pass",
            "type":"maliciousUrl"
          },
          {
            "level":"S0",
            "result":[
              {
                "description":"No risk",
                "label":"0",
                "level":"S0"
              }
            ],
            "suggestion":"pass",
            "type":"sensitiveData"
          },
          {
            "level":"none",
            "result":[
              {
                "confidence":0,
                "description":"No risk detected",
                "label":"nonLabel",
                "level":"none"
              }
            ],
            "suggestion":"pass",
            "type":"promptAttack"
          },
          {
            "level":"none",
            "result":[
              {
                "description":"No risk detected",
                "label":"nonLabelp",
                "level":"none"
              }
            ],
            "suggestion":"pass",
            "type":"contentModeration"
          }
        ],
        "EndTime":20,
        "StartTime":0,
        "Suggestion":"block",
        "Text":"Hello",
        "Url":"https://oss-cip-testing.oss-cn-shanghai.aliyuncs.com/xxx/xxx/xxx/xxx.wav"
      }]
    },
    "FrameResult": {
      "SliceNum": 1,
      "Suggestion": "block",
      "sliceDetails": [
        {
          "Detail": [
            {
              "level": "high",
              "result": [
                {
                  "description": "Hit well-known IP detection agent",
                  "label": "Well-known film, animation, and anime IP",
                  "level": "high"
                }
              ],
              "suggestion": "block",
              "type": "wellKnownIPsAgent"
            },
            {
              "level": "S0",
              "result": [
                {
                  "description": "No risk",
                  "label": "0",
                  "level": "S0"
                }
              ],
              "suggestion": "pass",
              "type": "sensitiveData"
            },
            {
              "level": "none",
              "result": [
                {
                  "confidence": 0,
                  "description": "No risk detected",
                  "label": "nonLabel",
                  "level": "none"
                }
              ],
              "suggestion": "pass",
              "type": "promptAttack"
            },
            {
              "level": "none",
              "result": [
                {
                  "description": "No risk detected",
                  "label": "nonLabel",
                  "level": "none"
                }
              ],
              "suggestion": "pass",
              "type": "customLabel"
            },
            {
              "ext": {
                "textInImage": {
                  "ocrResult": [{"text":"what do you want from me"},{"text":"What do you want me to do"}]
                }
              },
              "level": "none",
              "result": [
                {
                  "description": "No risk detected",
                  "label": "nonLabel",
                  "level": "none"
                }
              ],
              "suggestion": "pass",
              "type": "contentModeration"
            }
          ],
          "Offset": 10.5,
          "Suggestion": "block",
          "Url": "https://oss-cip-testing.oss-cn-shanghai.aliyuncs.com/xxx/xxx/xxx/xxx.jpg",
          "subTaskId": "vi_f_********",
          "success": true
        }
      ]
    },
    "Suggestion": "block",
    "TaskId": "vi_f_********",
    "labels": "Well-known film, animation, and anime IP"
  },
  "msg": "SUCCESS",
  "requestId": "5D8770A8-1234-1234-1234-AAABBBCCCDDD"
}

Code reference

Code	Status code	Description
200	OK	The request was successful.
400	BAD_REQUEST	The request is invalid. This may be caused by incorrect request parameters. Check your request parameters.
408	PERMISSION_DENY	The account may be unauthorized, overdue, not activated, or banned.
500	GENERAL_ERROR	An error occurred. A temporary server error may have occurred. Retry the request. If this error persists, please contact us through Online Service.
581	TIMEOUT	A timeout occurred. Retry the request. If this error persists, please contact us through Online Service.
588	EXCEED_QUOTA	The request rate exceeds the quota.