Qwen-LiveTranslate-Realtime client events
This topic describes client events for the Qwen real-time speech and audiovisual translation API, including event types, parameters, and examples.
Reference: Real-time audio and video translation - Qwen.
NoteThis page is the event reference for Qwen-LiveTranslate-Realtime. Event definitions are based on WebSocket access. AOQ and WebRTC use their respective data channels to send and receive model events, and media tracks to transmit media. First read WebSocket access, WebRTC access, or AOQ access.
For WebSocket SDK usage, see Python SDK and Java SDK. To select a model and protocol, see Overview.
session.update
After establishing a WebSocket connection, the client must first send this event to update the default session configuration.
After receiving the session.update event, the server validates the parameters. If any parameter is invalid, the server returns an error; if all parameters are valid, the server updates the configuration and returns the complete configuration.
typestring(Required)Event type. Must be set to session.update.
sessionobject(Optional)The session configuration.
Properties
input_modalitiesarray(Optional)For qwen3.8-livetranslate-flash-realtime over WebSocket. Set to ["audio"] (audio only, the default) or ["audio", "video"] (audio and images). The model processes images sent through input_image_buffer.append only when video is enabled.
output_modalitiesarray(Optional)Applies to qwen3.8-livetranslate-flash-realtime. Output modalities: ["text"] for text only or ["text", "audio"] for text and audio (default).
audioobject(Optional)Applies to qwen3.8-livetranslate-flash-realtime. Audio configuration.
Properties
inputobjectInput audio configuration.
Properties
formatobject(Optional)Input audio format.
Properties
typestring(Optional)Audio format. Valid values: pcm (default) and opus.
sample_rateinteger(Optional)Input audio sample rate in Hz. The default is 16000.
turn_detectionobject(Optional)Speech detection configuration.
Properties
typestring(Optional)Valid values: speaker_detection (default, enables real-time speaker diarization) and server_vad (voice activity detection).
thresholdnumber(Optional)The detection threshold. In speaker_detection mode, it distinguishes silence from valid speech. Sounds below the threshold are filtered out and are not passed to the model. The value is fixed at 0.5.
outputobject(Optional)Translated audio configuration.
Properties
voicestring(Optional)Voice for translated speech. When voice cloning is disabled, the default is Tina. For real-time voice cloning with voice_clone_options.frequency set to once or always, set the voice to default.
modalitiesarray(Optional)Applies to qwen3.5-livetranslate-flash-realtime.
Output modalities. Valid values:
-
["text"]
Outputs text only.
-
["text","audio"] (Default)
Outputs text and audio.
voicestring(Optional)Voice for generated audio. When voice cloning is disabled, set this to a system preset voice. Valid values: Supported voices. The default voice is Tina for qwen3.8-livetranslate-flash-realtime and qwen3.5-livetranslate-flash-realtime.
In 3.8, you can also set the voice in the output object under audio.
When
enable_voice_cloneistrue, the value ofvoicedepends on thevoice_clone_options.frequencysetting. Whenfrequencyisonceoralways, set it todefault. Whenfrequencyisnever, set it to the voice ID you previously cloned. Using a system preset voice in this case causes a server error.
enable_voice_cloneboolean(Optional)Whether to enable voice cloning. The default is false. When enabled, you can clone the input voice in real time during translation or use a voice cloned in advance. In this case, voice no longer uses a system preset voice. Set it to default or the ID of a voice you cloned in advance through the Voice cloning API.
voice_clone_optionsobject(Optional)Voice clone control options. Takes effect only when enable_voice_clone is true.
Properties
frequencystring(Optional)Voice cloning frequency. Valid values:
-
neverDisables server-side voice cloning and uses a voice you previously cloned. Set
voiceto your cloned voice ID. -
onceClones the voice once at the start of the session and reuses it for all subsequent output. Suitable for single-speaker scenarios. Set
voicetodefault. -
alwaysRe-clones the voice in real time before each output, dynamically adapting to the input audio. Suitable for multi-speaker scenarios. Set
voicetodefault.
sample_rateinteger(Optional)Applies to qwen3.5-livetranslate-flash-realtime.
Input audio sample rate, in Hz. Valid values:
- 8000
- 16000 (Default)
input_audio_transcriptionobject(Optional)Applies to qwen3.5-livetranslate-flash-realtime.
For qwen3.8-livetranslate-flash-realtime, ASR is always enabled and cannot be disabled. Transcripts are free of charge.
Configuration for input audio.
Properties
modelstring|null(Optional)Speech recognition model. Defaults to qwen3-asr-flash-realtime; ASR is enabled by default. The server returns the input audio recognition result (original source language text) through the conversation.item.input_audio_transcription.text and conversation.item.input_audio_transcription.completed events. To disable ASR, explicitly set this parameter to null.
Valid values: qwen3-asr-flash-realtime (default, enables ASR) and null (disables ASR).
languagestring(Optional)Source language for translation. Valid values: Supported languages. If not specified, the model automatically detects the source language.
input_audio_formatstring(Optional)Applies to qwen3.5-livetranslate-flash-realtime.
Input audio format. Valid values:
-
pcm(Default)Uncompressed raw audio data.
-
opusLossy audio codec with low-latency support, suitable for VoIP scenarios.
output_audio_formatstring(Optional)Applies to qwen3.5-livetranslate-flash-realtime.
Output audio format. Currently, this parameter can only be set to pcm.
turn_detectionobject|null(Optional)Applies to qwen3.5-livetranslate-flash-realtime.
Voice Activity Detection (VAD) configuration, used to control how speech start and end are detected:
- Set to a configuration object (default): Enables VAD mode. The server automatically detects speech start and end, commits the audio buffer, and triggers translation responses. The client does not need to send
input_audio_buffer.commitevents. - Set to
null: Enables Manual mode. The client manually commits the audio buffer by sendinginput_audio_buffer.commitevents. The server automatically starts generating translation responses upon receiving the commit.
Properties
typestring(Optional)VAD type. Must be set to server_vad.
thresholdfloat(Optional)VAD detection sensitivity. A lower value makes it easier to recognize faint sounds (including background noise) as speech; a higher value requires clearer, louder speech to trigger detection.
Value range: [-1.0, 1.0]. Default value: 0.2.
silence_duration_msinteger(Optional)Minimum duration of silence (in milliseconds) required after speech ends. Once this duration is exceeded, the server determines that speech has ended, automatically commits the audio buffer, and triggers a translation response.
Value range: [200, 6000]. Default value: 1000.
translationobject(Optional)Translation configuration.
Properties
languagestring(Optional)Target language for translation. Valid values: Supported languages. Default value: en.
same_language_skip_optionsobject(Optional)qwen3.8-livetranslate-flash-realtime does not support this parameter.
Same-language output configuration. When the source and target languages are the same, the service can skip text output, audio output, or both. This parameter takes effect only when the target language is set to zh or en.
Properties
skip_textboolean(Optional)Specifies whether to skip text output when the source and target languages are the same.
skip_audioboolean(Optional)Specifies whether to skip audio output when the source and target languages are the same.
corpusobject(Optional)Hot-word configuration to improve translation accuracy for specific terms.
Properties
phrasesobject(Optional)Hot-word mapping. The key is the term in the source language, and the value is its target-language translation.
Example: {"人工智能": "Artificial Intelligence"}
qwen3.8-livetranslate-flash-realtime
{
"type": "session.update",
"session": {
"input_modalities": [
"audio",
"video"
],
"output_modalities": [
"text",
"audio"
],
"audio": {
"input": {
"format": {
"type": "pcm",
"sample_rate": 16000
},
"turn_detection": {
"type": "speaker_detection",
"threshold": 0.5
}
},
"output": {
"voice": "default"
}
},
"enable_voice_clone": true,
"voice_clone_options": {
"frequency": "once"
},
"translation": {
"language": "en",
"corpus": {
"phrases": {
"人工智能": "Artificial Intelligence",
"机器学习": "Machine Learning"
}
}
}
}
}
qwen3.5-livetranslate-flash-realtime
{
"event_id": "event_ToPZqeobitzUJnt3QqtWg",
"type": "session.update",
"session": {
"modalities": [
"text",
"audio"
],
"voice": "default",
"enable_voice_clone": true,
"voice_clone_options": {
"frequency": "once"
},
"sample_rate": 16000,
"input_audio_format": "pcm",
"output_audio_format": "pcm",
"input_audio_transcription": {
"model": "qwen3-asr-flash-realtime",
"language": "zh"
},
"turn_detection": {
"type": "server_vad",
"threshold": 0.2,
"silence_duration_ms": 1000
},
"translation": {
"language": "en",
"same_language_skip_options": {
"skip_text": false,
"skip_audio": false
},
"corpus": {
"phrases": {
"人工智能": "Artificial Intelligence",
"机器学习": "Machine Learning"
}
}
}
}
}
input_audio_buffer.append
For WebSocket access, appends audio bytes to the input audio buffer. The server uses this buffer to detect speech and determine when to commit it.
typestring(Required)Event type. Must be set to input_audio_buffer.append.
audiostring(Required)Base64-encoded audio data.
{
"event_id": "event_xxx",
"type": "input_audio_buffer.append",
"audio": "xxx"
}
input_audio_buffer.commit
Commits the input audio buffer. This event only needs to be sent in Manual mode (turn_detection set to null). In VAD mode, the server commits automatically and the client does not need to send this event.
Upon receiving this event, the server returns an input_audio_buffer.committed event as confirmation and automatically starts generating a translation response (no additional event is needed to trigger the response). If the audio buffer is empty, the server returns an error event.
typestring(Required)Event type. Must be set to input_audio_buffer.commit.
{
"event_id": "event_xxx",
"type": "input_audio_buffer.commit"
}
input_audio_buffer.clear
Clears uncommitted audio data from the input audio buffer.
typestring(Required)Event type. Must be set to input_audio_buffer.clear.
{
"event_id": "event_xxx",
"type": "input_audio_buffer.clear"
}
input_image_buffer.append
For WebSocket access, adds image data to the image buffer. Images can come from a local file or be captured in real time from a video stream.
For qwen3.8-livetranslate-flash-realtime, set session.input_modalities to ["audio", "video"] in session.update before sending images. Existing requests that omit this field can still upload images, but the model ignores them.
Image input limits:
- Image format: JPG or JPEG. Recommended resolution for optimal performance: 480p or 720p (maximum: 1080p).
- Maximum image size: 500 KB (before Base64 encoding).
- Image data must be Base64-encoded.
- Maximum frequency: 2 images per second.
- Send at least one input_audio_buffer.append event before sending input_image_buffer.append.
typestring(Required)Event type. Must be set to input_image_buffer.append.
imagestring(Required)Base64-encoded image data.
{
"event_id": "event_xxx",
"type": "input_image_buffer.append",
"image": "xxx"
}
session.finish
Send this event to end the current session. Server responses:
- If speech is detected: The server completes the remaining translation. If input audio transcription is enabled, it returns the complete source-language transcript through conversation.item.input_audio_transcription.completed. After all audio translation is complete, it sends session.finished to indicate session end.
- If no speech is detected: The server sends
session.finisheddirectly.
After receiving the session.finished event, the client must finish local playback of any remaining translated audio before actively disconnecting, if playback is needed.
typestring(Required)Event type. Must be set to session.finish.
{
"event_id": "event_xxx",
"type": "session.finish"
}