Qwen-Audio-Realtime server events
Reference for server events of Qwen-Audio-Realtime API. All server events include the common fields event_id (automatically generated by the server) and type (event type).
User guide: Real-time voice conversation (Qwen-Audio-Realtime). For the event interaction sequence, see WebSocket access.
NoteThis page is the event reference for Qwen-Audio-Realtime. Event definitions are based on WebSocket access. AOQ and WebRTC use their respective data channels to send and receive model events, and media tracks to transmit media. First read WebSocket access, WebRTC access, or AOQ access.
For WebSocket SDK usage, see Android SDK, iOS SDK, and HarmonyOS SDK. To select a model and protocol, see Overview.
error
Description: Returned when a request fails or a service error occurs. Client errors (invalid_request_error) keep the connection open. Server errors (server_error) terminate the connection.
event_idstringUnique identifier for this event.
typestringEvent type. Always error.
errorobjectError details.
Properties
typestringError type, such as invalid_request_error (client error) or server_error (server error).
codestringError code.
messagestringError message.
paramstringThe parameter associated with the error.
{
"event_id": "event_xxx",
"type": "error",
"error": {
"type": "invalid_request_error",
"code": "invalid_value",
"message": "Cannot create response while another response is in progress.",
"param": "response.create"
}
}
session.created
Description: The first event sent by the server after the connection is established. Contains the default session configuration.
event_idstringUnique identifier for this event.
typestringEvent type. Always session.created.
sessionobjectSession configuration.
Properties
objectstringAlways realtime.session.
modelstringModel name.
modalitiesarrayOutput modalities of the model.
voicestringVoice used for audio generation. Either a system voice name or the voice_id of a cloned voice.
input_audio_transcriptionobjectAudio transcription configuration.
Properties
modelstringTranscription model, such as fun-asr.
turn_detectionobjectTurn detection (VAD) configuration.
idstringUnique session identifier.
{
"event_id": "event_KiKZC2zrhNsKFPZ5cTpyA",
"type": "session.created",
"session": {
"object": "realtime.session",
"model": "qwen-audio-3.1-realtime-plus",
"modalities": ["text", "audio"],
"voice": "longanqian_v3.1",
"input_audio_transcription": {
"model": "fun-asr"
},
"turn_detection": {
"type": "server_vad",
"threshold": 0.5,
"silence_duration_ms": 800
},
"id": "sess_A1LbG2D63WELBSawRbpq8"
}
}
session.updated
Description: Returned after a session.update request is processed successfully. Contains the full updated session configuration. If the request fails, an error event is returned instead.
event_idstringUnique identifier for this event.
typestringEvent type. Always session.updated.
sessionobjectFull session configuration after the update. Shares the same structure as the session object in session.created.
{
"event_id": "event_FMG6kiHbILCGiqXFPA98e",
"type": "session.updated",
"session": {
"id": "sess_A1LbG2D63WELBSawRbpq8",
"object": "realtime.session",
"model": "pre-fun-realtime-audiochat-beta-3",
"modalities": ["text", "audio"],
"voice": "longanqian",
"input_audio_transcription": {
"model": "fun-asr"
},
"turn_detection": {
"type": "smart_turn",
"threshold": 0.1,
"silence_duration_ms": 900
}
}
}
input_audio_buffer.speech_started
Description: Sent when Voice Activity Detection (VAD) detects the start of speech in server_vad / smart_turn / smart_turn_v2 modes.
event_idstringUnique identifier for this event.
typestringEvent type. Always input_audio_buffer.speech_started.
audio_start_msintegerTimestamp in milliseconds when speech started.
item_idstringID of the item that will be created when this speech segment is committed.
{
"event_id": "event_xxx",
"type": "input_audio_buffer.speech_started",
"audio_start_ms": 1200,
"item_id": "item_xxx"
}
input_audio_buffer.speech_stopped
Description: Sent when VAD detects the end of speech in server_vad / smart_turn / smart_turn_v2 modes.
event_idstringUnique identifier for this event.
typestringEvent type. Always input_audio_buffer.speech_stopped.
audio_end_msintegerTimestamp in milliseconds when speech ended.
item_idstringID of the user message item to be created.
reasonstringReturned only in smart_turn / smart_turn_v2 modes. When set to turn_invalid, indicates the current turn was classified as invalid (no semantic content) and inference will not be triggered. This field is absent for valid turns.
{
"event_id": "event_xxx",
"type": "input_audio_buffer.speech_stopped",
"audio_end_ms": 3400,
"item_id": "item_xxx",
"reason": "turn_invalid"
}
input_audio_buffer.committed
Description: The audio buffer has been committed as a user message, either through a push-to-talk commit or automatic VAD commit.
event_idstringUnique identifier for this event.
typestringEvent type. Always input_audio_buffer.committed.
previous_item_idstringID of the previous conversation item.
item_idstringID of the created user message item.
{
"event_id": "event_xxx",
"type": "input_audio_buffer.committed",
"previous_item_id": "item_xxx",
"item_id": "item_xxx"
}
input_audio_buffer.cleared
Description: The audio buffer has been cleared (push-to-talk mode only).
event_idstringUnique identifier for this event.
typestringEvent type. Always input_audio_buffer.cleared.
{
"event_id": "event_xxx",
"type": "input_audio_buffer.cleared"
}
conversation.item.created
Description: A new conversation item was created. Triggered when user audio is committed, the client manually creates an item, or an assistant response begins.
event_idstringUnique identifier for this event.
typestringEvent type. Always conversation.item.created.
previous_item_idstringID of the previous conversation item.
itemobjectThe created conversation item.
Properties
idstringUnique identifier for the conversation item.
objectstringAlways realtime.item.
typestringItem type: message (standard message) or function_call (function call).
statusstringItem status, such as in_progress or completed.
rolestringMessage role, such as user or assistant. Only present for message type items.
contentarrayMessage content list. Only present for message type items.
{
"event_id": "event_xxx",
"type": "conversation.item.created",
"previous_item_id": "item_xxx",
"item": {
"id": "item_xxx",
"object": "realtime.item",
"type": "message",
"status": "in_progress",
"role": "assistant",
"content": []
}
}
conversation.item.deleted
Description: A conversation item was deleted. Returned as a confirmation after the client sends a conversation.item.delete event.
event_idstringUnique identifier for this event.
typestringEvent type. Always conversation.item.deleted.
item_idstringID of the deleted conversation item.
{
"event_id": "event_xxx",
"type": "conversation.item.deleted",
"item_id": "item_xxx"
}
conversation.item.retrieved
Description: A conversation item was retrieved successfully. Returned after the client sends a conversation.item.retrieve event. For audio-type content, only the transcript text is included; raw audio data isn't returned.
event_idstringUnique identifier for this event.
typestringEvent type. Always conversation.item.retrieved.
itemobjectThe retrieved conversation item.
Properties
idstringUnique identifier for the conversation item.
objectstringAlways realtime.item.
typestringItem type: message (standard message) or function_call (function call).
rolestringMessage role, such as user or assistant. Only present for message type items.
contentarrayMessage content list. For audio-type content, only the transcript text is included; raw audio data isn't returned.
{
"event_id": "event_xxx",
"type": "conversation.item.retrieved",
"item": {
"id": "item_xxx",
"object": "realtime.item",
"type": "message",
"role": "user",
"content": [
{
"type": "input_audio",
"transcript": "Hello"
}
]
}
}
conversation.item.input_audio_transcription.delta
Description: Incremental Automatic Speech Recognition (ASR) transcription result, streamed during speech recognition. Includes emotion and language detection information.
event_idstringUnique identifier for this event.
typestringEvent type. Always conversation.item.input_audio_transcription.delta.
item_idstringID of the associated conversation item.
content_indexintegerIndex of the content part.
textstringFinalized transcription text.
stashstringTentative text that hasn't been finalized yet.
{
"event_id": "event_xxx",
"type": "conversation.item.input_audio_transcription.delta",
"item_id": "item_xxx",
"content_index": 0,
"text": "Hello",
"stash": "world"
}
conversation.item.input_audio_transcription.completed
Description: Final ASR transcription result. The transcribed text is written to the transcript field of the corresponding item.
event_idstringUnique identifier for this event.
typestringEvent type. Always conversation.item.input_audio_transcription.completed.
item_idstringID of the associated conversation item.
content_indexintegerIndex of the content part.
transcriptstringThe complete transcription text.
{
"event_id": "event_xxx",
"type": "conversation.item.input_audio_transcription.completed",
"item_id": "item_xxx",
"content_index": 0,
"transcript": "Hello world"
}
conversation.item.input_audio_transcription.failed
Description: ASR transcription failed.
event_idstringUnique identifier for this event.
typestringEvent type. Always conversation.item.input_audio_transcription.failed.
item_idstringID of the associated conversation item.
content_indexintegerIndex of the content part.
errorobjectError details.
Properties
typestringError type, such as transcription_error.
codestringError code, such as transcription_failed.
messagestringError message.
{
"event_id": "event_xxx",
"type": "conversation.item.input_audio_transcription.failed",
"item_id": "item_xxx",
"content_index": 0,
"error": {
"type": "transcription_error",
"code": "transcription_failed",
"message": "ASR transcription failed"
}
}
conversation.item.ambient_audio_transcription.delta
Description: smart_turn / smart_turn_v2 modes only. Incremental transcription result for ambient audio. When VAD detects voice activity but semantic analysis classifies the turn as invalid (for example, background noise or filler words like "hmm" or "uh"), the ASR result is forwarded to the client as an ambient event. This event isn't linked to any conversation item. The item_id is a standalone temporary ID.
event_idstringUnique identifier for this event.
typestringEvent type. Always conversation.item.ambient_audio_transcription.delta.
item_idstringA standalone temporary ID, not associated with any conversation item.
content_indexintegerIndex of the content part.
textstringFinalized transcription text.
stashstringTentative text that hasn't been finalized yet.
{
"event_id": "event_xxx",
"type": "conversation.item.ambient_audio_transcription.delta",
"item_id": "item_xxx",
"content_index": 0,
"text": "Hmm",
"stash": ""
}
conversation.item.ambient_audio_transcription.completed
Description: smart_turn / smart_turn_v2 modes only. Final transcription result for ambient audio. Paired with the corresponding delta event to mark the end of an ambient audio transcription segment. This result isn't written to the conversation context.
event_idstringUnique identifier for this event.
typestringEvent type. Always conversation.item.ambient_audio_transcription.completed.
item_idstringA standalone temporary ID, not associated with the conversation context.
content_indexintegerIndex of the content part.
transcriptstringThe complete transcription text.
{
"event_id": "event_xxx",
"type": "conversation.item.ambient_audio_transcription.completed",
"item_id": "item_xxx",
"content_index": 0,
"transcript": "Hmm",
}
response.created
Description: Sent when a model inference round starts.
event_idstringUnique identifier for this event.
typestringEvent type. Always response.created.
responseobjectThe response object.
Properties
idstringUnique identifier for the response.
objectstringAlways realtime.response.
statusstringResponse status, such as in_progress.
modalitiesarrayOutput modalities of the model.
voicestringVoice used for audio generation. Either a system voice name or the voice_id of a cloned voice.
outputarrayOutput items of the response. Initially an empty array.
{
"event_id": "event_xxx",
"type": "response.created",
"response": {
"id": "resp_xxx",
"object": "realtime.response",
"status": "in_progress",
"modalities": ["text", "audio"],
"voice": "longanqian",
"output": []
}
}
response.output_item.added
Description: A new output item was added to the response. The output item type is message for standard replies, or function_call for Function Calling.
event_idstringUnique identifier for this event.
typestringEvent type. Always response.output_item.added.
response_idstringAssociated response ID.
output_indexintegerIndex of the output item within the response.
itemobjectThe added output item.
Properties
idstringUnique identifier for the output item.
objectstringAlways realtime.item.
typestringOutput item type: message (standard message) or function_call (function call).
statusstringOutput item status, such as in_progress.
rolestringMessage role. Always assistant. Only present for message type items.
contentarrayMessage content list. Only present for message type items.
Message event
{
"event_id": "event_xxx",
"type": "response.output_item.added",
"response_id": "resp_xxx",
"output_index": 0,
"item": {
"id": "item_xxx",
"object": "realtime.item",
"type": "message",
"status": "in_progress",
"role": "assistant",
"content": []
}
}
Function Call item fragment
Function Calling output item example:
When the output item is a function call, the response.output_item.added / conversation.item.created / response.output_item.done events contain the following item structure:
{
"id": "item_xxx",
"object": "realtime.item",
"type": "function_call",
"status": "completed",
"call_id": "call_xxx",
"name": "get_weather",
"arguments": "{\"city\":\"Hangzhou\"}"
}
NoteA single response can include multiple function_call items and may also contain standard message output alongside function_call output. The Function Calling portion isn't sent to TTS for audio playback.
response.content_part.added
Description: A new content part was added to an output item.
event_idstringUnique identifier for this event.
typestringEvent type. Always response.content_part.added.
response_idstringAssociated response ID.
item_idstringAssociated output item ID.
output_indexintegerIndex of the output item within the response.
content_indexintegerIndex of the content part within the output item.
partobjectThe added content part.
Properties
typestringContent type, such as audio or text.
textstringText content. Initially an empty string.
{
"event_id": "event_xxx",
"type": "response.content_part.added",
"response_id": "resp_xxx",
"item_id": "item_xxx",
"output_index": 0,
"content_index": 0,
"part": {
"type": "audio",
"text": ""
}
}
response.text.delta
Description: Text delta event in text-only mode. Streams text fragments incrementally.
event_idstringUnique identifier for this event.
typestringEvent type. Always response.text.delta.
response_idstringAssociated response ID.
item_idstringAssociated output item ID.
output_indexintegerIndex of the output item within the response.
content_indexintegerIndex of the content part within the output item.
deltastringIncremental text fragment.
{
"event_id": "event_xxx",
"type": "response.text.delta",
"response_id": "resp_xxx",
"item_id": "item_xxx",
"output_index": 0,
"content_index": 0,
"delta": "Hello"
}
response.text.done
Description: Text output complete event in text-only mode.
event_idstringUnique identifier for this event.
typestringEvent type. Always response.text.done.
response_idstringAssociated response ID.
item_idstringAssociated output item ID.
output_indexintegerIndex of the output item within the response.
content_indexintegerIndex of the content part within the output item.
textstringThe complete text output.
{
"event_id": "event_xxx",
"type": "response.text.done",
"response_id": "resp_xxx",
"item_id": "item_xxx",
"output_index": 0,
"content_index": 0,
"text": "Hello, how can I help you?"
}
response.audio_transcript.delta
Description: Transcript delta event in audio mode. Streams transcript fragments incrementally.
event_idstringUnique identifier for this event.
typestringEvent type. Always response.audio_transcript.delta.
response_idstringAssociated response ID.
item_idstringAssociated output item ID.
output_indexintegerIndex of the output item within the response.
content_indexintegerIndex of the content part within the output item.
deltastringIncremental transcript fragment.
{
"event_id": "event_xxx",
"type": "response.audio_transcript.delta",
"response_id": "resp_xxx",
"item_id": "item_xxx",
"output_index": 0,
"content_index": 0,
"delta": "Hello"
}
response.audio_transcript.done
Description: Transcript output complete event in audio mode.
event_idstringUnique identifier for this event.
typestringEvent type. Always response.audio_transcript.done.
response_idstringAssociated response ID.
item_idstringAssociated output item ID.
output_indexintegerIndex of the output item within the response.
content_indexintegerIndex of the content part within the output item.
transcriptstringThe complete transcript text.
{
"event_id": "event_xxx",
"type": "response.audio_transcript.done",
"response_id": "resp_xxx",
"item_id": "item_xxx",
"output_index": 0,
"content_index": 0,
"transcript": "Hello, how can I help you?"
}
response.audio.delta
Description: For WebSocket access, this is an incremental audio data event in audio mode. The delta field contains Base64-encoded PCM audio data.
event_idstringUnique identifier for this event.
typestringEvent type. Always response.audio.delta.
response_idstringAssociated response ID.
item_idstringAssociated output item ID.
output_indexintegerIndex of the output item within the response.
content_indexintegerIndex of the content part within the output item.
deltastringBase64-encoded PCM audio data fragment.
{
"event_id": "event_xxx",
"type": "response.audio.delta",
"response_id": "resp_xxx",
"item_id": "item_xxx",
"output_index": 0,
"content_index": 0,
"delta": "<base64-encoded audio data>"
}
response.audio.done
Description: Audio output complete event in audio mode. Contains no audio data.
event_idstringUnique identifier for this event.
typestringEvent type. Always response.audio.done.
response_idstringAssociated response ID.
item_idstringAssociated output item ID.
output_indexintegerIndex of the output item within the response.
content_indexintegerIndex of the content part within the output item.
{
"event_id": "event_xxx",
"type": "response.audio.done",
"response_id": "resp_xxx",
"item_id": "item_xxx",
"output_index": 0,
"content_index": 0
}
response.content_part.done
Description: A content part within an output item has completed.
event_idstringUnique identifier for this event.
typestringEvent type. Always response.content_part.done.
response_idstringAssociated response ID.
item_idstringAssociated output item ID.
output_indexintegerIndex of the output item within the response.
content_indexintegerIndex of the content part within the output item.
partobjectThe completed content part.
Properties
typestringContent type, such as audio or text.
textstringText content or audio transcript text.
{
"event_id": "event_xxx",
"type": "response.content_part.done",
"response_id": "resp_xxx",
"item_id": "item_xxx",
"output_index": 0,
"content_index": 0,
"part": {
"type": "audio",
"text": "Hello, how can I help you?"
}
}
response.output_item.done
Description: An output item within the response has completed.
event_idstringUnique identifier for this event.
typestringEvent type. Always response.output_item.done.
response_idstringAssociated response ID.
output_indexintegerIndex of the output item within the response.
itemobjectThe completed output item with full details.
Properties
idstringUnique identifier for the output item.
objectstringAlways realtime.item.
typestringOutput item type: message (standard message) or function_call (function call).
statusstringOutput item status, such as completed.
rolestringMessage role. Always assistant. Only present for message type items.
contentarrayMessage content list. Only present for message type items.
{
"event_id": "event_xxx",
"type": "response.output_item.done",
"response_id": "resp_xxx",
"output_index": 0,
"item": {
"id": "item_xxx",
"object": "realtime.item",
"type": "message",
"status": "completed",
"role": "assistant",
"content": [
{
"type": "text",
"text": "Hello, how can I help you?"
}
]
}
}
response.function_call_arguments.delta
Description: Function Calling argument delta. When the model decides to invoke a tool, the server first sends response.output_item.added (with item.type=function_call) and conversation.item.created, then streams argument fragments through this event.
event_idstringUnique identifier for this event.
typestringEvent type. Always response.function_call_arguments.delta.
response_idstringAssociated response ID.
item_idstringAssociated output item ID.
output_indexintegerIndex of the output item within the response.
call_idstringUnique identifier for the function call.
deltastringIncremental fragment of the function call arguments (JSON string fragment).
{
"event_id": "event_xxx",
"type": "response.function_call_arguments.delta",
"response_id": "resp_xxx",
"item_id": "item_xxx",
"output_index": 0,
"call_id": "call_xxx",
"delta": "{\"city"
}
response.function_call_arguments.done
Description: Sent when Function Calling argument output is complete. After receiving this event, the client should: execute the corresponding tool, write the result to the conversation via conversation.item.create with a function_call_output item, and then send response.create to trigger a follow-up inference round.
event_idstringUnique identifier for this event.
typestringEvent type. Always response.function_call_arguments.done.
response_idstringAssociated response ID.
item_idstringAssociated output item ID.
output_indexintegerIndex of the output item within the response.
call_idstringUnique identifier for the function call.
namestringName of the invoked function.
argumentsstringComplete function call arguments (JSON string).
{
"event_id": "event_xxx",
"type": "response.function_call_arguments.done",
"response_id": "resp_xxx",
"item_id": "item_xxx",
"output_index": 0,
"call_id": "call_xxx",
"name": "get_weather",
"arguments": "{\"city\":\"Hangzhou\"}"
}
response.done
Description: An inference round is complete. status indicates the reason for completion.
event_idstringUnique identifier for this event.
typestringEvent type. Always response.done.
responseobjectThe complete response object.
Properties
idstringUnique identifier of the response.
objectstringAlways realtime.response.
statusstringCompletion status of the response. Valid values:
completed: Completed normally.cancelled: Interrupted and cancelled.status_details.reasonisturn_detected(VAD interruption) orclient_cancelled(client-initiated cancellation).failed: LLM or TTS error.
status_detailsobjectStatus details. Present only when the status is cancelled or failed.
Properties
typestringStatus type, such as cancelled.
reasonstringCancellation reason: turn_detected (VAD interruption) or client_cancelled (client-initiated cancellation).
modalitiesarrayOutput modalities of the model.
voicestringVoice used for audio generation. Either a system voice name or the voice_id of a cloned voice.
outputarrayList of output items in the response, containing complete item objects.
usageobjectToken consumption for this response. Returned only when status is completed.
Properties
total_tokensintegerTotal number of tokens consumed by this response.
input_tokensintegerNumber of input tokens.
output_tokensintegerNumber of output tokens.
input_tokens_detailsobjectBreakdown of input tokens, including text_tokens (text token count) and audio_tokens (audio token count, present only when the audio modality is used).
output_tokens_detailsobjectBreakdown of output tokens, including text_tokens (text token count) and audio_tokens (audio token count, present only when the audio modality is used).
pluginsobject(optional)Plug-in usage metering information. Returned when web search (enable_search) is enabled.
Properties
searchobjectWeb search metering information.
Properties
countintegerNumber of searches.
strategystringSearch strategy.
Completed normally
{
"event_id": "event_xxx",
"type": "response.done",
"response": {
"id": "resp_xxx",
"object": "realtime.response",
"status": "completed",
"modalities": ["text", "audio"],
"voice": "longanqian",
"output": [
{
"id": "item_xxx",
"object": "realtime.item",
"type": "message",
"status": "completed",
"role": "assistant",
"content": [
{
"type": "audio",
"transcript": "Hello, how can I help you?"
}
]
}
],
"usage": {
"total_tokens": 377,
"input_tokens": 336,
"output_tokens": 41,
"input_tokens_details": {
"text_tokens": 228,
"audio_tokens": 108
},
"output_tokens_details": {
"text_tokens": 9,
"audio_tokens": 32
}
}
}
}
Interrupted and cancelled
{
"event_id": "event_xxx",
"type": "response.done",
"response": {
"id": "resp_xxx",
"status": "cancelled",
"status_details": {
"type": "cancelled",
"reason": "turn_detected"
}
}
}
voiceprint_audio_list.in_progress
Description: The voiceprint registration process is in progress asynchronously.
event_idstringUnique identifier for this event.
typestringEvent type. Always voiceprint_audio_list.in_progress.
item_idstringUnique identifier for the voiceprint registration task.
{
"event_id": "event_PaEcN7CCrlhE8q4MM2yND",
"type": "voiceprint_audio_list.in_progress",
"item_id": "vp_Y12cA986j1KZ9O9YmAXOA"
}
voiceprint_audio_list.completed
Description: The voiceprint registration process has completed successfully.
event_idstringUnique identifier for this event.
typestringEvent type. Always voiceprint_audio_list.completed.
item_idstringUnique identifier for the voiceprint registration task.
{
"event_id": "event_PaEcN7CCrlhE8q4MM2yND",
"type": "voiceprint_audio_list.completed",
"item_id": "vp_Y12cA986j1KZ9O9YmAXOA"
}
voiceprint_audio_list.failed
Description: The voiceprint registration failed. Normal conversation calls are not blocked.
event_idstringUnique identifier for this event.
typestringEvent type. Always voiceprint_audio_list.failed.
item_idstringUnique identifier for the voiceprint registration task.
reasonstringDescription of the failure reason.
{
"event_id": "event_PaEcN7CCrlhE8q4MM2yND",
"type": "voiceprint_audio_list.failed",
"item_id": "vp_Y12cA986j1KZ9O9YmAXOA",
"reason": ""
}