The enhanced text moderation service can be integrated with Decision Engine. This integration allows you to pass Content Moderation detection results through Decision Engine as a pass-through service. This simplifies development and reduces integration costs. This topic describes how to use the text moderation service in Decision Engine.
Supported business scenarios
Decision Engine currently integrates two commonly used Content Moderation text moderation services: nickname_detection and comment_detection.
Service category | Business scenario | Scenario description | Decision Engine variable | Key value |
nickname_detection | Applies to user nicknames, official account names, and live stream titles. | Focuses on detecting risks such as baseline violations, impersonation of official accounts, and prohibited traffic diversion to help govern fraudulent accounts. | enhanced text moderation - User nickname - Label | labels |
enhanced text moderation - User nickname - fine-grained label | riskTips | |||
enhanced text moderation - User nickname - Ad number matches | adNums | |||
enhanced text moderation - User nickname - Matched risk words | riskWords | |||
comment_detection | Applies to comments, bullet chats, public chat messages, and reposts. | Typically used for content with high risk rates, diverse risk types, and frequent variations. Detects risks such as baseline violations, ad-based traffic diversion, and prohibited content. | enhanced text moderation - Public chat and comments - Label | labels |
enhanced text moderation - Public chat and comments - fine-grained label | riskTips | |||
enhanced text moderation - Public chat and comments - Ad number matches | adNums | |||
enhanced text moderation - Public chat and comments - Matched risk words | riskWords |
Use cases
You can configure Content Moderation variables in Decision Engine to adjust the detection scope and the granularity of risk detection to meet your business requirements. You can also convert the labels returned by the Content Moderation service into entity-level (account, device, IP, etc.) risk profiles to manage entity-based risks more accurately and efficiently.
Use case | Description |
Use the Content Moderation text moderation service in Decision Engine | If your application is already integrated with Decision Engine, you can perform content moderation directly through the engine, eliminating the need for a separate Content Moderation API integration. |
Use the Content Moderation text moderation service in Decision Engine | You can adjust the scope and granularity of risk detection based on common industry practices or your specific business needs. For example, if your business experiences frequent issues with ad-based traffic diversion, especially involving mobile numbers, you can configure a policy to flag text that matches these specific fine-grained labels. |
You can convert text-level risks into entity-level risks for accounts, devices, or IPs. This allows you to handle moderation results more accurately and efficiently. For example, you can set a tolerance for the number of ad-related messages an account can send before a penalty is applied. |
Related documents
For detailed instructions on the three scenarios, see Use the Content Moderation text moderation service in Decision Engine.