What is AI Guardrails

When developing and operating AI applications and AI Agents, developers and enterprises often face security threats, including content compliance risks, data leakage risks, prompt injection attacks, hallucinations, and jailbreaks. These AI risks not only threaten normal business operations but also expose enterprises to substantial compliance and social risks.

AI Guardrails is designed to protect the compliance, security, and stability of AI workloads. It provides an end-to-end protection system for different business forms, including pre-trained foundation models, AI services, and AI Agents. In generative AI input and output scenarios, AI Guardrails provides risk detection and proactive defense capabilities.

Product features

• Risk detection: AI Guardrails provides comprehensive detection for content compliance, sensitive content, prompt injection attacks, and more.

• Content compliance detection: Performs multidimensional compliance reviews of text input to and output from generative AI. It covers risk categories such as politically sensitive content, pornography and vulgarity, bias and discrimination, and harmful values. This detection helps ensure that AI-generated content (AIGC) complies with laws, regulations, and platform rules. Typical use cases include chatbots, AI education, intelligent customer service, and AIGC creation platforms.

• Sensitive content detection: Detects private data and sensitive information that may be disclosed during AI interactions. It identifies sensitive content involving personal privacy and enterprise privacy to prevent training data leakage and conversational information exposure. Typical use cases include AI healthcare, AI financial services, and enterprise knowledge base Q&A.

• Prompt injection detection: Defends against injection attacks targeting generative AI. It identifies adversarial behavior such as jailbreak instructions, role assumption inducements, and system instruction tampering. Typical use cases include securing AI Agent instruction interactions, defending open-domain dialogue systems against adversarial attacks, and managing permissions for third-party plug-in calls.

• Malicious file detection: Parses common document formats uploaded by users, such as PDF, PPT, and DOC, to identify hidden executable scripts, macro viruses, nested attack code, and other malicious content. This detection prevents attackers from using file injection to gain unauthorized control of AI systems or exfiltrate data. Typical use cases include AI scenarios that support document uploads, such as intelligent resume parsing, contract Q&A, and enterprise knowledge base construction.

• Malicious URL detection: Performs real-time analysis of the links that AI receives or generates during interactions. It identifies high-risk URLs such as phishing websites, malicious redirects, and hidden attack payloads to prevent foundation models from being induced to access illegal resources or becoming vectors for cyberattacks. Typical use cases include AI search, web page summarization, retrieval-augmented generation (RAG) knowledge retrieval, and automated outreach operations.

• Digital watermarking: Automatically embeds visible or invisible watermarks in AI-generated images in accordance with the Measures for Labeling Artificial Intelligence-Generated Synthetic Content. Watermarks make the source of AIGC content traceable and responsibilities attributable, which helps prevent the spread of false information and copyright disputes. Typical use cases include compliance-sensitive scenarios such as AIGC creation platforms, news media, government communications, and educational content generation.

• Custom mitigation settings: You can modify granular risk detection items in Mitigation Settings. Log on to the AI Guardrails console to enable or disable risk detection content at any time and create a risk detection template that suits your business needs.

• Custom detection items: Configure granular tags for content compliance detection.

• Custom risk thresholds: Configure the hit threshold for granular tags. For model confidence scores from 0 to 100, the minimum configurable increment is 1.

• Custom filter words: Configure sensitive words to detect and block, such as competitor names. You can add, delete, and modify entries in the word library.

For detailed rules and agreements for sensitive word libraries, see the AI Guardrails Third-party Sensitive Word Agreement.

Scenarios

AI Guardrails is recommended for risk detection in the following scenarios:

• Detecting risks in user prompts submitted to generative AI for processing.

• Detecting risks in multi-modal content generated by generative AI, including text, images, and videos.

• Scanning and cleaning the corpora used for generative AI training.

• Detecting risks in the inputs and outputs of AI Agent user instructions.

FAQ