A dedicated Content Moderation service for Model Studio users-AI Guardrails(AI Guardrails)-阿里云帮助中心

Authorize content moderation in Model Studio

Grant the required SLR authorization in Model Studio and pass a specific identifier in the header of your API calls. For details, see the following document:

Content moderation in Model Studio

Activate content moderation

Click Activate Service to use the Content Moderation service via Model Studio.

Model matching strategy

After you grant SLR authorization for Content Moderation in the Model Studio console and pass the cip identifier in the Model Studio API header, the service automatically matches the appropriate Content Moderation service to your model version. The matching logic is as follows:

Service name	Service identifier	Applicable models	Features
Model Studio text input detection_pro	`bailian_query_check_pro`	Qwen-Max series models	Detects baseline violations (such as pornographic, political, and violent content), malicious prompts, and certain types of prompts for sensitive topics. In some scenarios, this service uses a large model for moderation to increase detection accuracy.
Model Studio text output detection_pro	`bailian_response_check_pro`	Qwen-Max series models	Detects baseline violations (such as pornographic, political, and violent content) and malicious prompts. It also detects abusive language, bias, and harmful values that may be generated by the AI. In some scenarios, this service uses a large model for moderation to increase detection accuracy.
Model Studio text input detection	`bailian_query_check`	Other model series (non-Qwen-Max)	Detects baseline violations (such as pornographic, political, and violent content), malicious prompts, and certain types of prompts for sensitive topics.
Model Studio text output detection	`bailian_response_check`	Other model series (non-Qwen-Max)	Detects baseline violations (such as pornographic, political, and violent content) and malicious prompts. It also detects abusive language, bias, and harmful values that may be generated by the AI.

Billing

Token-based pay-as-you-go

When you grant SLR authorization for Content Moderation in the Model Studio console and use its policies, you are charged on a pay-as-you-go basis according to the number of tokens processed. You are billed daily based on your actual usage. If you do not use the service, you are not charged.

Moderation type	Supported services	Price
Standard text moderation by token (text_token_standard)	Model Studio text input detection: `bailian_query_check` Model Studio text output detection: `bailian_response_check`	CNY 0.0004 per 1,000 tokens
Advanced text moderation by token (text_token_advanced)	Model Studio text input detection_pro: `bailian_query_check_pro` Model Studio text output detection_pro: `bailian_response_check_pro`	CNY 0.003 per 1,000 tokens

Important

For each query/response moderation request in Model Studio, if the text contains fewer than 1,000 tokens, the request is billed as 1,000 tokens. If the text contains more than 1,000 tokens, you are charged based on the actual number of tokens.

Risk labels

Label definitions

To get information about the supported labels for each service and their detailed detection scopes, click View Labels on the rule managementContent Moderation (Enhanced Edition) > text moderation > rule configuration page. The following table describes the risk labels, their confidence score ranges, and their meanings:

Label	Confidence score range	Description
pornographic_adult	0–100. A higher score indicates greater confidence.	Suspected pornographic content
sexual_terms	0–100. A higher score indicates greater confidence.	Suspected sexual health content
sexual_prompts	0–100. A higher score indicates greater confidence.	Suspected prompt for generating pornographic content
sexual_suggestive	0–100. A higher score indicates greater confidence.	Suspected vulgar content
political_figure	0–100. A higher score indicates greater confidence.	Suspected political figure
political_entity	0–100. A higher score indicates greater confidence.	Suspected political entity
political_n	0–100. A higher score indicates greater confidence.	Suspected sensitive political content
political_p	0–100. A higher score indicates greater confidence.	Suspected politically banned figure
political_prompts	0–100. A higher score indicates greater confidence.	Suspected prompt for generating political content
political_a	0–100. A higher score indicates greater confidence.	Enhanced protection for sensitive political content
violent_extremist	0–100. A higher score indicates greater confidence.	Suspected extremist group
violent_incidents	0–100. A higher score indicates greater confidence.	Suspected extremist content
violent_weapons	0–100. A higher score indicates greater confidence.	Suspected weapons and ammunition
violent_prompts	0–100. A higher score indicates greater confidence.	Suspected prompt for generating violent content
contraband_drug	0–100. A higher score indicates greater confidence.	Suspected drug-related content
contraband_gambling	0–100. A higher score indicates greater confidence.	Suspected gambling-related content
contraband_act	0–100. A higher score indicates greater confidence.	Suspected prohibited activities
contraband_entity	0–100. A higher score indicates greater confidence.	Suspected prohibited items
inappropriate_discrimination	0–100. A higher score indicates greater confidence.	Suspected discriminatory or biased content
inappropriate_ethics	0–100. A higher score indicates greater confidence.	Suspected content with harmful values
inappropriate_profanity	0–100. A higher score indicates greater confidence.	Suspected abusive or insulting content
inappropriate_oral	0–100. A higher score indicates greater confidence.	Suspected vulgar slang
inappropriate_superstition	0–100. A higher score indicates greater confidence.	Suspected superstitious content
inappropriate_nonsense	0–100. A higher score indicates greater confidence.	Suspected spam or meaningless content
pt_to_sites	0–100. A higher score indicates greater confidence.	Suspected traffic diversion to external sites
pt_by_recruitment	0–100. A higher score indicates greater confidence.	Suspected ads for online earning or part-time jobs
pt_to_contact	0–100. A higher score indicates greater confidence.	Suspected promotion account for traffic diversion
religion_b	0–100. A higher score indicates greater confidence.	Suspected content related to Buddhism
religion_t	0–100. A higher score indicates greater confidence.	Suspected content related to Taoism
religion_c	0–100. A higher score indicates greater confidence.	Suspected content related to Christianity
religion_i	0–100. A higher score indicates greater confidence.	Suspected content related to Islam
religion_h	0–100. A higher score indicates greater confidence.	Suspected content related to Hinduism
customized	0–100. A higher score indicates greater confidence.	hit in custom lexicon

Manage labels

With the exception of some baseline security labels, you can enable or disable all other risk labels in the console. Some risk labels also provide more granular configuration options for their detection scope. For more information, see the Content Moderation console.

In the left-side navigation pane, choose Content Moderation - (Enhanced Edition) > text moderation > rule configuration.
On the rule management tab, take the bailian_query_check service as an example. In the Actions column, click Manage detection rules.
1. Select the detection type to adjust, for example, Inappropriate content detection.
2. Click Edit to modify the detection status.
3. Click Save. The new settings take about 2 to 5 minutes to take effect in the production environment.

Step 1: Activate the service

Before you can use Content Moderation (Enhanced Edition), you must activate the service.

Go to the Content Moderation (Enhanced Edition) page. Read the terms of service and select the checkbox to agree.
Click Activate Now.

Step 2: Customize rules (Optional)

Content Moderation (Enhanced Edition) provides a built-in set of labels for text moderation that meets most requirements.

If you need to create custom moderation rules, you can create a lexicon, define risky keywords or keywords to ignore, and then configure detection rules that use your lexicon.

Log on to the Content Moderation console.
On the lexicon managementContent Moderation (Enhanced Edition) > text moderation, follow these steps to configure a lexicon.
1. On the Keyword Library Management tab, click Create Library.
2. In the Create Library panel, enter the lexicon information as prompted.
  
  1. You can combine multiple keywords into a single entry by using logical operators. For example, WeChat&part-time is a match only if both keywords appear. The & operator represents AND, and the ~ operator represents NOT (exclude). The & operator must be placed before the ~ operator.
  
  2. Separate each keyword with a line break. A single keyword cannot exceed 50 characters.
  
  3. You can add up to 1,000 keywords at a time. To add more than 1,000, import them from a file.
  
  4. Each account can have a total of 100,000 keywords across a maximum of 20 lexicons.
3. Click Create Library.
  
  If lexicon creation fails, a detailed error message is displayed.
Configure rules.
1. On the rule managementContent Moderation (Enhanced Edition) > text moderation > rule configuration page, select the target service and click Operation in the Set Thesaurus column.
2. Select the lexicon to use for ignoring keywords. Then, click Next.
  
  If the text contains a keyword from the ignore lexicon, the service ignores that keyword and moderates the remaining text. This is useful for adding keywords to an allowlist.
  
  For example, if the ignore lexicon contains the keywords convenient and fast, and the text to be moderated is Student small loans: safe, fast, convenient, no collateral, borrow and repay anytime, same-day funding, in-person service, the words convenient and fast are ignored. Only the remaining text, Student small loans: safe, no collateral, borrow and repay anytime, same-day funding, in-person service, is moderated.
3. Select the lexicon to use for matching keywords. Then, click OK.
  
  When you call the enhanced text moderation API, if any keyword from a keyword library is matched in the text to be moderated, the labels field returns C_customized (match in a user-defined library). This scenario is primarily used to detect potential violations in the text.
  
  For example, suppose the keywords in your keyword library are small loans and door-to-door service. If the text to be moderated is Our school's small loans, safe, fast, convenient, no collateral, borrow as you go, same-day disbursement, door-to-door service, the system matches the keywords small loans and door-to-door service. When you call the API for the enhanced version of Text Moderation, the value of the labels parameter in the response includes C_customized in addition to any matched built-in labels.
After you configure the rules, the new rules take effect in about 2 to 5 minutes.

Step 3: View results (Optional)

You can view the moderation results to analyze the most frequent types of violations in your text.

On the Result queryContent Moderation (Enhanced Edition) > text moderation, view the moderated text, the labels that were hit, and the request time.
You can filter the results by time range, Alibaba Cloud request ID, text content, and label. You can query data from the last 30 days. The Result query page can store up to 50,000 records. If you have higher storage needs, you must save the API response data yourself.

When you filter by label, the following options are available:
- Include: The results must contain the specified label.
- Exclude: The results must not contain the specified label.
- Empty: The results did not match any labels.
- Not empty: The results matched one or more labels. You do not need to specify a label value.
The system does not display non-risky content segments.
Find a specific text entry and click Operation in the View column to see detailed moderation information.

If you disagree with a moderation result, you can submit feedback. In the Operation column for the entry, click the Feedback drop-down list and select No violation false alarm or Violation missed.