Client event reference for the Qwen-Audio Realtime API.
User guide: Realtime Audio Chat (Qwen-Audio-Realtime). For event interaction sequences, see WebSocket API.
session.update
Description: After a connection is established, send this event to update the default session configuration. Include only the fields you want to change; omitted fields retain their current values. If any parameter is invalid, the server returns an error. If all parameters are valid, the server applies the changes and returns the full configuration.
Noteturn_detection can only be modified before the first audio is sent (IDLE state).
typestring(required) Event type. Fixed value: session.update. sessionobject (optional) Session configuration. Properties modalitiesarray (optional) Output modalities for the model. Valid values: voicestring (optional) TTS voice name. Default: longanqian. Two types are supported. Can only be set in the first session.update; ignored in subsequent calls.
- System voices: Available values:
longanqian, longanlingxin, longanlingxi, longanxiaoxin, longanlufeng.
- Cloned voices: created via the Voice Cloning API. Pass the returned
voice_id as this parameter's value. For details, see Voice configuration.
enable_speech_emotionboolean (optional) Specifies whether to enable speech emotion enhancement. When enabled, the response voice exhibits more pronounced emotional variations. Default: true. Valid values: true and false. instructionsstring (optional) System instructions that define the model's role, response style, and behavioral preferences. Applies to the entire session. input_audio_formatstring (optional) Input audio format. Currently only pcm (16 kHz, 16-bit, mono) is supported and is the default. Can only be modified before the first audio is sent (IDLE state). output_audio_formatstring (optional) Output audio format. Currently only pcm (24 kHz, 16-bit, mono) is supported and is the default. max_history_turnsinteger (optional) Maximum number of conversation turns (question-answer pairs) included in a single request. Valid values: 1 to 50. Default: 20. toolsarray (optional) Tool definitions for Function Calling. Once configured, the model decides whether to call a tool based on user input. Properties typestring(required) Fixed value: function. function.namestring(required) Name of the tool function. function.descriptionstring (optional) Description of the tool function. The model uses this to determine whether to call the tool. function.parametersobject (optional) Description of the tool function's input parameters. The model uses this to extract the required parameters. Omit this field if the function takes no parameters. Properties typestring(required) Fixed value: object. propertiesobject (optional) Describes each parameter's name, data type, and description. requiredarray (optional) Specifies which parameters are required. turn_detectionobject|null (optional) Turn detection configuration. To switch to push-to-talk mode, set this field to null. In push-to-talk mode, audio must be committed manually and inference triggered manually. If this field isn't provided, VAD is enabled with its default parameters. Properties typestring (optional) VAD type. Valid values:
server_vad (default): Detects speech onset and offset based on acoustic features and automatically triggers inference.smart_turn: An intelligent turn detection mode that combines acoustic perception and semantic understanding to determine turn boundaries. Filler sounds such as "um" or "uh" don't trigger a new turn or interrupt model playback. thresholdfloat (optional) VAD sensitivity. Only effective in server_vad mode (ignored in smart_turn mode). Lower values increase VAD sensitivity, making it easier to detect faint sounds (including background noise) as speech. Higher values decrease sensitivity, requiring clearer and louder speech to trigger detection. Range: [-1.0, 1.0]. Default: 0.5. silence_duration_msinteger (optional) Minimum silence duration (in milliseconds) after speech ends before triggering a model response. Only effective in server_vad mode (ignored in smart_turn mode). Lower values produce faster responses but may cause false triggers during brief pauses. Range: [200, 6000]. Default: 800. Recommended range for conversations: 400-800. voiceprint_audio_urlsarray (optional) Only effective in smart_turn mode. A list of publicly accessible URLs pointing to pre-recorded audio samples from the target user, used for speaker enhancement. Once registered, the model locks onto the target speaker during duplex conversations, effectively ignoring other voices and background noise. Maximum 5 URLs. Audio format requirements: 16kHz PCM or WAV. ImportantThis parameter can only be configured in the first session.update event. It is ignored in subsequent session.update events. | {
"type": "session.update",
"session": {
"modalities": [
"text",
"audio"
],
"voice": "longanqian",
"turn_detection": {
"type": "server_vad",
"threshold": 0.5,
"silence_duration_ms": 800
}
}
}
Function Calling: {
"type": "session.update",
"session": {
"modalities": [
"text",
"audio"
],
"voice": "longanqian",
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get weather for a specified city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"title": "City"
}
},
"required": ["city"]
}
}
}
],
"turn_detection": {
"type": "server_vad",
"threshold": 0.5,
"silence_duration_ms": 800
}
}
}
Voiceprint registration (voiceprint_audio_urls): {
"type": "session.update",
"session": {
"turn_detection": {
"type": "smart_turn",
"voiceprint_audio_urls": [
"https://example.com/speaker1.pcm",
"https://example.com/speaker2.wav"
]
}
}
}
|
Description: Appends audio data to the input buffer. Send this event continuously at a high frequency — for example, one chunk every 20–40 ms. The server sends no acknowledgment for this event.
typestring(required) Event type. Fixed value: input_audio_buffer.append. audiostring(required) Base64-encoded audio data. | {
"type": "input_audio_buffer.append",
"audio": "<Base64-encoded audio data>"
}
|
Description: Push-to-talk mode only. Commits the buffered audio as a user message. This doesn't automatically trigger inference. Send response.create to trigger inference manually.
This event is ignored in server_vad and smart_turn modes.
typestring(required) Event type. Fixed value: input_audio_buffer.commit. | {
"type": "input_audio_buffer.commit"
}
|
Description: Push-to-talk mode only. Clears uncommitted audio from the buffer. This event is ignored in server_vad and smart_turn modes. The server responds with an input_audio_buffer.cleared event.
typestring(required) Event type. Fixed value: input_audio_buffer.clear. | {
"type": "input_audio_buffer.clear"
}
|
conversation.item.create
Description: Inserts a conversation item into the conversation context. Use this event to inject historical context or add text content, or to return Function Calling results.
NoteIf item.id already exists in the conversation, an error is returned and the item isn't created.
typestring(required) Event type. Fixed value: conversation.item.create. previous_item_idstring (optional) Specifies the conversation item after which the new item is inserted. If not provided, the item is appended to the end of the conversation. itemobject(required) The conversation item to create. Properties idstring (optional) Unique identifier for the conversation item. If not provided, the server generates one automatically. An error is returned if the specified ID already exists in the conversation. typestring(required) Conversation item type. Valid values:
message: A regular conversation message.function_call: A function call request. Typically generated by the server, but clients can also use it to inject historical context.function_call_output: A tool execution result. After receiving a function_call, the client executes the tool and returns the result using this type. rolestring (required for message type) Message role. Valid values: system, user, assistant. contentarray (required for message type) List of message content elements. Each element contains a type field and corresponding data fields. Supported content types by role systeminput_text: System message. Required field: text.
user
input_text: User text input. Required field: text.input_audio: User audio input. Required field: audio (Base64-encoded). assistantoutput_text: Assistant text output. Required field: text.
call_idstring (required for function_call / function_call_output type) Unique identifier for the function call, used to correlate requests and results. namestring (required for function_call type) Name of the function to call. argumentsstring (required for function_call type) Function call parameters in JSON string format. outputstring (required for function_call_output type) Tool execution result in JSON string format. | Inject a user text message: {
"type": "conversation.item.create",
"previous_item_id": "item_xxx",
"item": {
"id": "my_item_001",
"type": "message",
"role": "user",
"content": [
{
"type": "input_text",
"text": "Please summarize our last conversation"
}
]
}
}
Return a Function Calling result: {
"type": "conversation.item.create",
"item": {
"type": "function_call_output",
"call_id": "call_xxx",
"output": "{\"temperature\":18,\"condition\":\"sunny\"}"
}
}
|
conversation.item.retrieve
Description: Retrieves a conversation item stored on the server. Audio-type content in the response contains only the transcript (transcript), not the original audio data.
typestring(required) Event type. Fixed value: conversation.item.retrieve. item_idstring(required) ID of the conversation item to retrieve. The server returns the result in a conversation.item.retrieved event. | {
"type": "conversation.item.retrieve",
"item_id": "item_xxx"
}
|
conversation.item.delete
Description: Deletes a conversation item from the conversation context. The server confirms the deletion with a conversation.item.deleted event.
typestring(required) Event type. Fixed value: conversation.item.delete. item_idstring(required) ID of the conversation item to delete. | {
"type": "conversation.item.delete",
"item_id": "item_xxx"
}
|
response.create
Description: Triggers model inference. Behavior varies by mode:
- Push-to-talk mode: Must be called manually. Commit the buffered audio with
input_audio_buffer.commit first, or return a function_call_output result before triggering. Can't be called while a response is being generated.
- server_vad mode: Typically triggered automatically by the server. Clients can also call it manually when no response is being generated. Can't be called while a response is being generated.
- smart_turn mode: Can be called while waiting for the next user turn. Can't be called during an active turn (between
input_audio_buffer.speech_started and response.done).
The optional response field overrides the session defaults for the current inference round. In Function Calling scenarios, after the client returns a function_call_output, this event triggers the second inference round.
NoteIn server_vad and smart_turn modes, manually triggered inferences can still be interrupted by new speech.
typestring(required) Event type. Fixed value: response.create. responseobject (optional) Overrides the session defaults for the current inference round. If not provided, the current session configuration is used. Properties modalitiesarray (optional) Overrides the output modalities for the current round. Valid values are the same as session.update modalities. voicestring (optional) Overrides the TTS voice for the current round. | {
"type": "response.create",
"response": {
"modalities": ["audio", "text"]
}
}
|
response.cancel
Description: Cancels the current inference. Any text generated so far is saved to the item list. The server then returns a response.done event with status=cancelled.
An error is returned if no inference is in progress.
typestring(required) Event type. Fixed value: response.cancel. | {
"type": "response.cancel"
}
|