AI Red Teaming

Overview

AI Red Teaming can be used to detect security risks in connected AI models and AI agents. By adding detection objects, you can perform security assessments on AI services, identify potential attack surfaces and security vulnerabilities. It also provides compliance test reports that can be used for pre-filing self-testing, helping enterprises strengthen AI defenses through offensive testing.

• Supported detection objects:

  • AI Model: Detects AI model services that are compatible with OpenAI or Anthropic APIs. Checks for security risks such as prompt injection and data leakage in large language models accessed through APIs.

  • AI Agent: Evaluates the security of agent applications deployed on platforms such as Alibaba Cloud Model Studio, Dify, and PAI.

• Detection results description: Currently, only report download is supported. Automatic processing is not supported, and results are not synchronized to the Agent Threat list.

Prerequisites

A paid instance of Agent Security Center is activated. For more information, see Purchase and unsubscribe from Agent Security Center.

Add a detection object

1. Log on to Security Center console.

2. In the left-side navigation pane, choose Agent Security > AI Red Teaming .

3. Above the AI Red Teaming list, click Add Detection Target.

4. On the Target Information page, configure the following parameters. After the configuration is complete, click Next to proceed to connection parameter configuration.

   Parameter	Description
   Target Type	AI Model: For model services that are compatible with OpenAI or Anthropic APIs. AI Agent: For agent applications deployed on platforms such as Alibaba Cloud Model Studio and Dify. Note The parameters in subsequent steps vary depending on the selected type.
   Deployment Location	Direct Internet Connection: For targets deployed on the public network. Corporate Intranet: For targets deployed in enterprise intranets where cloud platforms cannot directly connect. You need to prepare an internal test machine that can access the public network, run a relay script on it, and call the target Agent locally to complete the test and return results to Agent Security Center.
   Access Method/Quick Access	AI Model: Direct Internet Connection: OpenAI Compatible: For model services that are compatible with the OpenAI Chat Completions API format, such as Alibaba Cloud Model Studio, Azure OpenAI, and various open-source model deployments. Anthropic Compatible: For model services that are compatible with the Anthropic Messages API format. Corporate Intranet: Only OpenAI Compatible is supported. AI Agent: Direct Internet Connection: Supports quick access from platforms such as Alibaba Cloud Model Studio, Dify, Agent Run, PAI, and AgentKit. Corporate Intranet: Only OpenAI Compatible is supported.
   Target Name	A custom name used to identify and manage the detection object.
   Description	Optional. Enter a description for the detection object.

5. Configure connection parameters: On the Connection Parameters tab, fill in the corresponding connection parameters based on the target type and deployment location. After the configuration is complete, click Next to proceed to connection verification.

   Direct Internet Connection

   • AI model parameters

     Parameter	Description
     Streaming Output	Whether to enable streaming output. Enabled by default.
     API Address	The API endpoint of the AI model. For example, the OpenAI-compatible API endpoint for Alibaba Cloud Model Studio is https://dashscope.aliyuncs.com/compatible-mode/v1.
     API Key	The access key used for authentication. For example, Alibaba Cloud Model Studio users can create an API key on the API Key Management page in the Model Studio console.
     Model Name	The name of the model to test. Examples for Alibaba Cloud Model Studio: qwen-plus, qwen-max.
     Platform	Select the platform to which the AI model belongs from the drop-down list, such as Alibaba Cloud Model Studio.

   • AI agent parameters

     • Basic parameters

       Parameter	Description
       Streaming Output	Whether to enable streaming output. Important PAI enables streaming output by default and does not support disabling it.
       API Address	The API endpoint of the agent. PAI: Format: http://{uid}.{region}.pai-eas.aliyuncs.com/api/predict/{serviceName}/run. Replace {uid}, {region}, and {serviceName} with the actual PAI-EAS service configurations. Alibaba Cloud Model Studio: https://dashscope.aliyuncs.com/api/v1/apps/{appId}/completion, where {appId} is the Model Studio application ID. Dify: Format: http://{host}/v1/chat-messages. Replace {host} with the actual Dify service endpoint. Agent Run: Format: https://{workspaceId}.agentrun-data.{region}.aliyuncs.com/agent-runtimes/{agentName}/endpoints/Default/invocations/openai/v1/chat/completions. Replace {workspaceId}, {region}, and {agentName} with the actual Agent Run configurations. AgentKit: Format: https://{id}.apigateway-{region}.volceapi.com/invoke. Replace {id} and {region} with the actual Agent Kit gateway configurations.
       API Key	The access key used for authentication. For example, Alibaba Cloud Model Studio users can create an API key on the API Key Management page in the Model Studio console.
       Model Name	Displayed only for the PAI platform. Enter the model name.

     • Advanced settings (optional)

       Parameter	Description
       HTTP Method	The request method: GET, POST, or PUT. Default value: POST.
       Authentication Method	Select an authentication method: None, Bearer Token, or Custom Header.
       Authentication Header Name	Displayed only when Authentication is set to Custom Header. Enter a custom authentication header name.
       Timeout Period	The request timeout period, in milliseconds. Default value: 30000.
       Request Template	The request body template, preset by platform.
       JSONPath	The response parsing path, preset by platform.
       Request Header	Custom request headers in JSON format.

   Corporate Intranet

   1. Test Machine Architecture (Linux): Select the CPU architecture of the internal test machine. The script delivers a binary file of the corresponding architecture. Currently, only Linux x86_64 (amd64) and Linux ARM64 (aarch64) are supported. You can run uname -m on the internal machine to check the architecture (x86_64=amd64, aarch64/arm64=arm64).

      Important

      The internal test machine must be able to access the public network to upload scan results to Agent Security Center.

   2. Obtain Configuration Policy and AK/SK:

      1. Create a custom permission policy:

         1. Log on to the RAM console. On the Permissions > Policies page, click Create Policy.

         2. On the Create Policy page, select Script Editor, enter the following script, and then click OK.

            {
              "Version": "1",
              "Statement": [
                {
                  "Effect": "Allow",
                  "Action": [
                    "yundun-aisc:PollRelayTurn",
                    "yundun-aisc:ReportRelayTurnResult"
                  ],
                  "Resource": "*"
                }
              ]
            }

      2. Create a RAM user and grant permissions:

         1. On the Identities > Users page, click Create User and fill in the basic information.

         2. In the Access Configuration section, select Use Permanent AccessKey for Access, and then click OK.

         3. After the user is created, the system automatically generates an AccessKey ID and an AccessKey Secret for the RAM user.

            Important

            The AccessKey Secret of the RAM user is displayed only at creation and cannot be viewed later. Make sure to download or copy and save the AccessKey Secret immediately. A leaked AccessKey poses a serious threat to the security of all resources under your account. For more information, see Create an AccessKey.

         4. Return to the Users page. In the Actions column of the new user, click Add Permissions.

         5. In the Permissions section, select the custom policy created in the previous step, and then click Confirm.

   3. Test Agent Connectivity on an Internal Machine (Optional): Before running the access script, we recommend that you run the following command on the internal test machine to verify that the target Agent is reachable. Replace <IP>, <PORT>, and <KEY> with the API endpoint of the target Agent in the internal network. A response of 200 with choices content indicates that the Agent is connected.

      curl -X POST http://<IP>:<PORT>/v1/chat/completions \
        -H "Authorization: Bearer <KEY>" \
        -H "Content-Type: application/json" \
        -d '{"model":"hermes-agent","messages":[{"role":"user","content":"hello"}],"stream":false}'

   4. Run the access script:

      1. Click Generate Access Script. The system generates an executable script based on the selected internal test machine architecture.

      2. Run the script on the internal test machine and enter the following information as prompted:

         1. Target Agent API endpoint: The API endpoint of the target Agent in the internal network, for example, http://127.0.0.1:8080/v1/chat/completions.

         2. Target Agent API Key: The authentication key of the target Agent. If the target Agent does not require authentication, leave it empty and press Enter.

         3. Alibaba Cloud OpenAPI authentication: Used by the relay script to report detection results to Agent Security Center. Enter the AccessKey ID and AccessKey Secret created in the previous step.

6. On the Test Connection tab, confirm the configured target details and click Connectivity Test to verify whether the configured parameters can connect to the AI service.

   Important

   • You can click Save only after the connectivity test passes. If the test fails, check whether the API endpoint, API Key, and model name are correct.

   • The connectivity test does not actually run a detection task. It only verifies network connectivity, API Key, and API Address.

   • Testing: The button displays a loading animation.

   • Test passed: The button changes to a success state, and the Save button becomes available.

   • Test failed: You can view the failure cause in the Connection test failed. prompt area. If the failure is caused by network issues, click Retry to try again.

Configure Detection Task

1. In the Actions column of the target detection object, click the icon and select Configure Detection Task.

2. In the Configure Detection Task panel, select a scan mode:

   Scan Mode	Description	Applicable Scenario
   Security Detection (default)	AI Red Teaming intelligent agent security adversarial testing based on red team experience.	For security adversarial testing.
   Compliance	Based on TC260 keywords and question banks to meet filing compliance assessment requirements.	For filing compliance self-testing.

3. Click OK to save the configuration.

Run detection

1. Select detection objects:

   • Single detection: In the Actions column of the target detection object, click Check.

   • Batch detection: Select multiple detection objects and click Check below the AI Red Teaming list.

2. In the confirmation dialog box, click OK.

3. After the detection task is created, the list automatically refreshes and the detection status is updated.

View detection tasks and reports

1. In the Actions column of the target detection object, click Test Records, or click the number in the Scan Count column.

2. In the Detection Records panel, you can view detection details and filter by Scan Mode and Scan Progress.

   Name	Description
   Scan Task ID	The unique identifier of the detection task.
   Scan Mode	Security Detection or Compliance.
   Scan Start Time	The start time of the detection task.
   Scan End Time	The completion time of the detection task.
   Scan Task Progress	The task execution progress. Displays the completion percentage while running.
   Risk Level	The risk level detected in this scan.

3. View Report: After detection completes, in the Actions column, click View Report to view the security assessment report for the agent or model. The report includes an overview, security score, attack distribution, risk matrix, attack technique analysis, attack intent analysis, attack case details, and security recommendations.

4. Download Report: After detection completes, in the Actions column, click the icon and then click Download Report to download the security report to your local device.

5. Download Hit Data: After detection completes, in the Actions column, click the icon and then click Download Hit Data to download the hit sample data in CSV format.

Stop or delete detection tasks

1. In the Actions column of the target detection object, click the icon and select Test Records, or click the number in the Scan Count column.

2. In the Detection Records panel, in the Actions column of the target task, click Stop or Delete.

   Action	Applicable To	Effect
   Stop	Incomplete tasks only	Terminates the scan and retains the detection record.
   Delete	Tasks in all states	Terminates the scan and deletes the detection record.

Manage detection objects

Billing

• Billing unit: Billed by scan count. Each detection counts as 1 instance.

• Accumulation method: If the same object is detected multiple times, billing is accumulated based on the actual number of executions.

  Example: If you perform detection 10 times on an AI agent in Model Studio, it will be counted as 10 instances.

• Billing condition: Charges are incurred only when the detection task is successfully completed. If the detection task fails or is aborted, no charge is incurred.

• Pricing: ¥1,000/instance .

FAQ