DocsICP FilingConsole
官方文档
首页

Content moderation best practices for AI scenarios

更新时间：
复制 MD 格式
产品详情
我的收藏

The rapid development of Artificial Intelligence Generated Content (AIGC) technology has led to its widespread use in social and commercial applications. While this technology generates a large amount of engaging content, it also introduces challenges related to illegal, non-compliant, and harmful content. Alibaba Cloud AI Guardrails offers solutions to detect and manage these threats in AI applications. This document explains how to integrate AI Guardrails into your AI applications to reject harmful content, provide safe responses, and implement content administration.

Important

For content moderation in AI scenarios, use the AI Guardrails product. AI Guardrails is a protection system for AI applications. It provides real-time and accurate detection to ensure the compliance, security, and stability of your AI applications.

Prerequisites

  1. Activate AI Guardrails Pro.

  2. Integrate the AI Guardrails Pro - Text Moderation API operation and the AI Guardrails Pro - Image Moderation API operation.

Solution flow

  • Text-to-text AI

  1. Content moderation flowchart:

image

  1. APIs to integrate and configure:

Step

API Operations

API service configuration

Step 2

TextModerationPlus

llm_query_moderation

Step 7

TextModerationPlus

llm_response_moderation

  • Text-to-image AI

  1. Content moderation flowchart:

image

  1. APIs to integrate and configure:

Step

API operation

API service configuration

Step 2

TextModerationPlus

llm_query_moderation

Step 7

ImageModeration

aigcCheck

  • Image-to-image AI

  1. Content moderation flowchart:

image

  1. APIs to integrate and configure:

Step

API Operations

API service configuration

Step 2

ImageModeration

baselineCheck

Step 7

ImageModeration

aigcCheck

  • Image understanding AI

  1. Content moderation flowchart:

image

  1. APIs to integrate and configure:

Step

API operation

API service configuration

Step 2

ImageModeration

baselineCheck

Step 7

TextModerationPlus

llm_response_moderation

Adjust detection rules

You can adjust detection rules in the Alibaba Cloud AI Guardrails console. For more information, see the AI Guardrails console. The following example uses the TextModerationPlus API operation:

  1. In the navigation pane on the left, choose Text Moderation Pro > Rule Configuration.

  2. On the Rule Management tab, find the large language model input text detection (llm_query_moderation) scenario. Then, click Manage Detection Rules in the Actions column.

    1. Select the detection type to adjust. This example uses harmful content detection.

    2. Click Edit to modify the detection status.

    3. Click Save. The new configuration takes about 2 to 5 minutes to take effect.

Previous：Use LLM moderationNext：Best practices for detecting and labeling AIGC