Use this solution to dynamically moderate and control content that large models generate as a stream.
How it works
When a user submits a prompt, the application calls a large model, which returns generated text as a continuous stream. A guardrail system intercepts this stream and checks each segment for harmful content before the text is rendered to the user.
Segmentation methods
Stream-based content moderation triggers a check after content accumulates to a certain size. Two segmentation methods are commonly used:
Method 1: Segment by character count
When N characters accumulate, send those N characters as a batch for moderation.
Method 2: Segment using a sliding window
For every X new characters generated, send the most recently generated N characters for the check. Because the window retains surrounding context, semantic accuracy improves and perceived user wait time decreases.
High-performance moderation service
If you want to keep the moderation time for each segment under 100 ms, use our high-performance service.
response_security_check_hp
This service is suitable for Method 1, which segments content by character count.
When you use the stream-based moderation solution, you must pass the following input parameters to associate moderation checks across multiple segments:
chatId: Uniquely identifies a single round of interaction, which includes the user input and the large model output.
done: Indicates that this segment is the final segment of the current conversation.
For more information, see Multi-modal API integration guide.
Handle moderation results
After each segment is checked, take action based on the result:
| Result | Action |
|---|---|
| No threat detected | Progressively render the corresponding text to the user. |
| Threat detected | Immediately stop generating subsequent content. Revoke any harmful content already rendered, or replace it with a preset compliant response. |
Stopping generation and revoking displayed content as soon as a threat is detected protects users and maintains content safety.