Build custom detection with Moderation Agent

Add an agent

• The system provides a default workflow that supports parallel calls to agent groups. In the agent component on the canvas, add an agent configuration.

Configure the agent

• Click the Edit icon to open the agent configuration page. Here, you can select a large model and configure custom prompts. This process includes selecting a preset scenario template and defining detection labels.

• Select a large model. Select a large model that matches your moderation requirements. During detection, the system calls this model. The following large models are available:

  Modality	Model name	Model description
  Text	Text Moderation Large Model	A text moderation large model fine-tuned from the base Qwen model for content moderation. It accurately identifies specific risk content related to compliance and governance.
  Text	Qwen3.6-Plus	A Plus model in the Qwen 3.6 series. This model offers significantly improved performance over previous versions. It is suitable for complex scenarios where high effectiveness is required and some latency is acceptable.
  Text	Qwen3-Plus	A Plus model in the Qwen 3 series. It balances effectiveness, speed, and cost, and is suitable for complex scenarios that require high effectiveness and can tolerate some latency.
  Text	Qwen3-Flash	A Flash model in the Qwen 3 series. It offers high speed and low cost, making it ideal for simple tasks.
  Image, image-text multi-modality	Qwen3-VL-Plus	The most powerful Plus model in the Qwen 3 VL series. It is suitable for complex scenarios where high effectiveness is required and some latency is acceptable.
  Image, image-text multi-modality	Qwen3-VL-Flash	A Flash model in the Qwen 3 VL series. It offers higher speed and lower cost, making it a cost-effective choice for scenarios sensitive to response time.

• Configure custom prompts

  • Select a preset scenario template. The system provides different preset scenario templates for various scenarios. Each template supports different task objectives and detection labels. The available scenario template is:

    • Custom label template: Supports the configuration of custom detection labels for general-purpose scenarios.

  • Configure detection labels. Configure detection labels and their corresponding prompts based on your business requirements. For each label, you must define a detection label and a detection prompt. Configuring multiple detection labels means you are setting up a multi-classification task for the large model. Therefore, describe each detection task with a precise and concise detection label and detection prompt.

    • Configuration instructions:

      • Detection Label: Specifies the name of the category to be detected. It is usually a noun phrase. A single label can be up to 15 characters long and can contain only Chinese characters, letters, digits, and underscores (_).

      • Detection Prompt: Specifies the criteria and rules for detection. It provides a detailed description of the detection label and can include one to three examples. The prompt for a single label can be up to 300 characters long and can contain only Chinese characters, letters, digits, underscores (_), and common punctuation marks.

    • For example, to detect text content, you can use the following configuration:

      Moderation label	Moderation criteria
      Off-platform diversion	Behavior that directs users to other platforms or channels, either directly or implicitly (including variations, metaphors, and other obfuscations). This includes explicitly mentioning competitor platform names or their variations (e.g., common competitors are xx), mentioning other off-platform channels or their variations (e.g., common platforms are xx), or including explicit contact information.
      Malicious negative reviews for Brand XX	Unsubstantiated malicious comparisons, false negative reviews, or fabricated defamation targeting Brand XX, or false slander and rumors against the brand's founder that intentionally damage the brand or founder's image. For example: "XX is all false advertising, much worse than Brand YY."

    Important

    • For each agent, the character length of the custom prompts (the total length of all detection labels and detection prompts) affects billing. For more information, see Billing.

    • In addition, longer prompts increase detection latency. Therefore, you can configure a maximum of 50 custom detection labels.

    • The model's output format is preset and does not require configuration. For more information, see response data.

    During actual detection, the system combines the selected preset scenario template, your custom detection label configurations, and the preset output format to create a complete prompt. This prompt is then sent to the selected large model to obtain the moderation result. The resulting prompt is similar to the following example:

    You are a senior ****** moderation expert, particularly skilled in ******. The business problem you are facing is ******, and the task objective is ******. The labels to be moderated are as follows: 1. Off-platform diversion: Behavior that directs users to other platforms or channels, either directly or implicitly (including variations, metaphors, and other obfuscations). This includes explicitly mentioning competitor platform names or their variations (e.g., common competitors are xx), mentioning other off-platform channels or their variations (e.g., common platforms are xx), or including explicit contact information. 2. Malicious negative reviews for Brand XX: Unsubstantiated malicious comparisons, false negative reviews, or fabricated defamation targeting Brand XX, or false slander and rumors against the brand's founder that intentionally damage the brand or founder's image. For example: "XX is all false advertising, much worse than Brand YY." 3. ******. ******. You will be given a sample to review. Determine if the sample text falls within the scope of the labels described above. Strictly adhere to the following output format: ******.

AI-powered prompt optimization

• On the detection label configuration page, you can use a large model to generate and optimize prompts. Provide an optimization direction (required), moderation rules (optional), and violating/normal samples (optional). The system then calls a large model to generate detection labels, criteria, and definitions based on your input. Providing more details generally yields better results.

  • Prompt generation: The initial prompt can be empty. Provide the required details to generate an initial prompt.

  • Prompt optimization: The initial prompt can be non-empty. Optimize based on the initial prompt.