DocsICP FilingConsole
Documentation
Home Page Alibaba Cloud Model Studio API Reference (Models) Audio Voice Conversation API Reference Real-time voice conversation API reference Server events

Server events for Qwen-Audio Realtime API

Updated at:
Copy as MD
Product
My Favorites

Server event reference for the Qwen-Audio Realtime API. All server events include the event_id (auto-generated by the server) and type (event type) fields.

User guide: Realtime Audio Chat (Qwen-Audio-Realtime). For event interaction sequences, see WebSocket API.

error

Description: Returned when a request fails or a service error occurs. Client errors (invalid_request_error) keep the connection open. Server errors (server_error) terminate the connection.

event_idstring

Unique identifier for this event.

typestring

Event type. Always error.

errorobject

Error details.

Properties

typestring

Error type, such as invalid_request_error (client error) or server_error (server error).

codestring

Error code.

messagestring

Error message.

paramstring

The parameter associated with the error.

{
    "event_id": "event_xxx",
    "type": "error",
    "error": {
        "type": "invalid_request_error",
        "code": "invalid_value",
        "message": "Cannot create response while another response is in progress.",
        "param": "response.create"
    }
}

session.created

Description: The first event sent by the server after the connection is established. Contains the default session configuration.

event_idstring

Unique identifier for this event.

typestring

Event type. Always session.created.

sessionobject

Session configuration.

Properties

objectstring

Always realtime.session.

modelstring

Model name.

modalitiesarray

Output modalities of the model.

voicestring

Voice used for audio generation. Either a system voice name or the voice_id of a cloned voice.

input_audio_transcriptionobject

Audio transcription configuration.

Properties

modelstring

Transcription model, such as fun-asr.

turn_detectionobject

Turn detection (VAD) configuration.

idstring

Unique session identifier.

{
    "event_id": "event_KiKZC2zrhNsKFPZ5cTpyA",
    "type": "session.created",
    "session": {
        "object": "realtime.session",
        "model": "qwen-audio-3.0-realtime-plus",
        "modalities": ["text", "audio"],
        "voice": "longanqian",
        "input_audio_transcription": {
            "model": "fun-asr"
        },
        "turn_detection": {
            "type": "server_vad",
            "threshold": 0.5,
            "silence_duration_ms": 800
        },
        "id": "sess_A1LbG2D63WELBSawRbpq8"
    }
}

session.updated

Description: Returned after a session.update request is processed successfully. Contains the full updated session configuration. If the request fails, an error event is returned instead.

event_idstring

Unique identifier for this event.

typestring

Event type. Always session.updated.

sessionobject

Full session configuration after the update. Shares the same structure as the session object in session.created.

{
    "event_id": "event_FMG6kiHbILCGiqXFPA98e",
    "type": "session.updated",
    "session": {
        "id": "sess_A1LbG2D63WELBSawRbpq8",
        "object": "realtime.session",
        "model": "pre-fun-realtime-audiochat-beta-3",
        "modalities": ["text", "audio"],
        "voice": "longanqian",
        "input_audio_transcription": {
            "model": "fun-asr"
        },
        "turn_detection": {
            "type": "smart_turn",
            "threshold": 0.1,
            "silence_duration_ms": 900
        }
    }
}

input_audio_buffer.speech_started

Description: Sent when Voice Activity Detection (VAD) detects the start of speech in server_vad / smart_turn mode.

event_idstring

Unique identifier for this event.

typestring

Event type. Always input_audio_buffer.speech_started.

audio_start_msinteger

Timestamp in milliseconds when speech started.

item_idstring

ID of the item that will be created when this speech segment is committed.

{
    "event_id": "event_xxx",
    "type": "input_audio_buffer.speech_started",
    "audio_start_ms": 1200,
    "item_id": "item_xxx"
}

input_audio_buffer.speech_stopped

Description: Sent when VAD detects the end of speech in server_vad / smart_turn mode.

event_idstring

Unique identifier for this event.

typestring

Event type. Always input_audio_buffer.speech_stopped.

audio_end_msinteger

Timestamp in milliseconds when speech ended.

item_idstring

ID of the user message item to be created.

reasonstring

Returned only in smart_turn mode. When set to turn_invalid, indicates the current turn was classified as invalid (no semantic content) and inference will not be triggered. This field is absent for valid turns.

{
    "event_id": "event_xxx",
    "type": "input_audio_buffer.speech_stopped",
    "audio_end_ms": 3400,
    "item_id": "item_xxx",
    "reason": "turn_invalid"
}

input_audio_buffer.committed

Description: The audio buffer has been committed as a user message, either through a push-to-talk commit or automatic VAD commit.

event_idstring

Unique identifier for this event.

typestring

Event type. Always input_audio_buffer.committed.

previous_item_idstring

ID of the previous conversation item.

item_idstring

ID of the created user message item.

{
    "event_id": "event_xxx",
    "type": "input_audio_buffer.committed",
    "previous_item_id": "item_xxx",
    "item_id": "item_xxx"
}

input_audio_buffer.cleared

Description: The audio buffer has been cleared (push-to-talk mode only).

event_idstring

Unique identifier for this event.

typestring

Event type. Always input_audio_buffer.cleared.

{
    "event_id": "event_xxx",
    "type": "input_audio_buffer.cleared"
}

conversation.item.created

Description: A new conversation item was created. Triggered when user audio is committed, the client manually creates an item, or an assistant response begins.

event_idstring

Unique identifier for this event.

typestring

Event type. Always conversation.item.created.

previous_item_idstring

ID of the previous conversation item.

itemobject

The created conversation item.

Properties

idstring

Unique identifier for the conversation item.

objectstring

Always realtime.item.

typestring

Item type: message (standard message) or function_call (function call).

statusstring

Item status, such as in_progress or completed.

rolestring

Message role, such as user or assistant. Only present for message type items.

contentarray

Message content list. Only present for message type items.

{
    "event_id": "event_xxx",
    "type": "conversation.item.created",
    "previous_item_id": "item_xxx",
    "item": {
        "id": "item_xxx",
        "object": "realtime.item",
        "type": "message",
        "status": "in_progress",
        "role": "assistant",
        "content": []
    }
}

conversation.item.deleted

Description: A conversation item was deleted. Returned as a confirmation after the client sends a conversation.item.delete event.

event_idstring

Unique identifier for this event.

typestring

Event type. Always conversation.item.deleted.

item_idstring

ID of the deleted conversation item.

{
    "event_id": "event_xxx",
    "type": "conversation.item.deleted",
    "item_id": "item_xxx"
}

conversation.item.retrieved

Description: A conversation item was retrieved successfully. Returned after the client sends a conversation.item.retrieve event. For audio-type content, only the transcript text is included; raw audio data isn't returned.

event_idstring

Unique identifier for this event.

typestring

Event type. Always conversation.item.retrieved.

itemobject

The retrieved conversation item.

Properties

idstring

Unique identifier for the conversation item.

objectstring

Always realtime.item.

typestring

Item type: message (standard message) or function_call (function call).

rolestring

Message role, such as user or assistant. Only present for message type items.

contentarray

Message content list. For audio-type content, only the transcript text is included; raw audio data isn't returned.

{
    "event_id": "event_xxx",
    "type": "conversation.item.retrieved",
    "item": {
        "id": "item_xxx",
        "object": "realtime.item",
        "type": "message",
        "role": "user",
        "content": [
            {
                "type": "input_audio",
                "transcript": "Hello"
            }
        ]
    }
}

conversation.item.input_audio_transcription.delta

Description: Incremental Automatic Speech Recognition (ASR) transcription result, streamed during speech recognition. Includes emotion and language detection information.

event_idstring

Unique identifier for this event.

typestring

Event type. Always conversation.item.input_audio_transcription.delta.

item_idstring

ID of the associated conversation item.

content_indexinteger

Index of the content part.

textstring

Finalized transcription text.

stashstring

Tentative text that hasn't been finalized yet.

{
    "event_id": "event_xxx",
    "type": "conversation.item.input_audio_transcription.delta",
    "item_id": "item_xxx",
    "content_index": 0,
    "text": "Hello",
    "stash": "world"
}

conversation.item.input_audio_transcription.completed

Description: Final ASR transcription result. The transcribed text is written to the transcript field of the corresponding item.

event_idstring

Unique identifier for this event.

typestring

Event type. Always conversation.item.input_audio_transcription.completed.

item_idstring

ID of the associated conversation item.

content_indexinteger

Index of the content part.

transcriptstring

The complete transcription text.

{
    "event_id": "event_xxx",
    "type": "conversation.item.input_audio_transcription.completed",
    "item_id": "item_xxx",
    "content_index": 0,
    "transcript": "Hello world"
}

conversation.item.input_audio_transcription.failed

Description: ASR transcription failed.

event_idstring

Unique identifier for this event.

typestring

Event type. Always conversation.item.input_audio_transcription.failed.

item_idstring

ID of the associated conversation item.

content_indexinteger

Index of the content part.

errorobject

Error details.

Properties

typestring

Error type, such as transcription_error.

codestring

Error code, such as transcription_failed.

messagestring

Error message.

{
    "event_id": "event_xxx",
    "type": "conversation.item.input_audio_transcription.failed",
    "item_id": "item_xxx",
    "content_index": 0,
    "error": {
        "type": "transcription_error",
        "code": "transcription_failed",
        "message": "ASR transcription failed"
    }
}

conversation.item.ambient_audio_transcription.delta

Description: smart_turn mode only. Incremental transcription result for ambient audio. When VAD detects voice activity but semantic analysis classifies the turn as invalid (for example, background noise or filler words like "hmm" or "uh"), the ASR result is forwarded to the client as an ambient event. This event isn't linked to any conversation item. The item_id is a standalone temporary ID.

event_idstring

Unique identifier for this event.

typestring

Event type. Always conversation.item.ambient_audio_transcription.delta.

item_idstring

A standalone temporary ID, not associated with any conversation item.

content_indexinteger

Index of the content part.

textstring

Finalized transcription text.

stashstring

Tentative text that hasn't been finalized yet.

{
    "event_id": "event_xxx",
    "type": "conversation.item.ambient_audio_transcription.delta",
    "item_id": "item_xxx",
    "content_index": 0,
    "text": "Hmm",
    "stash": ""
}

conversation.item.ambient_audio_transcription.completed

Description: smart_turn mode only. Final transcription result for ambient audio. Paired with the corresponding delta event to mark the end of an ambient audio transcription segment. This result isn't written to the conversation context.

event_idstring

Unique identifier for this event.

typestring

Event type. Always conversation.item.ambient_audio_transcription.completed.

item_idstring

A standalone temporary ID, not associated with the conversation context.

content_indexinteger

Index of the content part.

transcriptstring

The complete transcription text.

{
    "event_id": "event_xxx",
    "type": "conversation.item.ambient_audio_transcription.completed",
    "item_id": "item_xxx",
    "content_index": 0,
    "transcript": "Hmm",
}

response.created

Description: Sent when a model inference round starts.

event_idstring

Unique identifier for this event.

typestring

Event type. Always response.created.

responseobject

The response object.

Properties

idstring

Unique identifier for the response.

objectstring

Always realtime.response.

statusstring

Response status, such as in_progress.

modalitiesarray

Output modalities of the model.

voicestring

Voice used for audio generation. Either a system voice name or the voice_id of a cloned voice.

outputarray

Output items of the response. Initially an empty array.

{
    "event_id": "event_xxx",
    "type": "response.created",
    "response": {
        "id": "resp_xxx",
        "object": "realtime.response",
        "status": "in_progress",
        "modalities": ["text", "audio"],
        "voice": "longanqian",
        "output": []
    }
}

response.output_item.added

Description: A new output item was added to the response. The output item type is message for standard replies, or function_call for Function Calling.

event_idstring

Unique identifier for this event.

typestring

Event type. Always response.output_item.added.

response_idstring

Associated response ID.

output_indexinteger

Index of the output item within the response.

itemobject

The added output item.

Properties

idstring

Unique identifier for the output item.

objectstring

Always realtime.item.

typestring

Output item type: message (standard message) or function_call (function call).

statusstring

Output item status, such as in_progress.

rolestring

Message role. Always assistant. Only present for message type items.

contentarray

Message content list. Only present for message type items.

{
    "event_id": "event_xxx",
    "type": "response.output_item.added",
    "response_id": "resp_xxx",
    "output_index": 0,
    "item": {
        "id": "item_xxx",
        "object": "realtime.item",
        "type": "message",
        "status": "in_progress",
        "role": "assistant",
        "content": []
    }
}

Function Calling output item example:

When the output item is a function call, the response.output_item.added / conversation.item.created / response.output_item.done events contain the following item structure:

{
          "id": "item_xxx",
          "object": "realtime.item",
          "type": "function_call",
          "status": "completed",
          "call_id": "call_xxx",
          "name": "get_weather",
          "arguments": "{\"city\":\"Hangzhou\"}"
      }

NoteA single response can include multiple function_call items and may also contain standard message output alongside function_call output. The Function Calling portion isn't sent to TTS for audio playback.

response.content_part.added

Description: A new content part was added to an output item.

event_idstring

Unique identifier for this event.

typestring

Event type. Always response.content_part.added.

response_idstring

Associated response ID.

item_idstring

Associated output item ID.

output_indexinteger

Index of the output item within the response.

content_indexinteger

Index of the content part within the output item.

partobject

The added content part.

Properties

typestring

Content type, such as audio or text.

textstring

Text content. Initially an empty string.

{
    "event_id": "event_xxx",
    "type": "response.content_part.added",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0,
    "part": {
        "type": "audio",
        "text": ""
    }
}

response.text.delta

Description: Text delta event in text-only mode. Streams text fragments incrementally.

event_idstring

Unique identifier for this event.

typestring

Event type. Always response.text.delta.

response_idstring

Associated response ID.

item_idstring

Associated output item ID.

output_indexinteger

Index of the output item within the response.

content_indexinteger

Index of the content part within the output item.

deltastring

Incremental text fragment.

{
    "event_id": "event_xxx",
    "type": "response.text.delta",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0,
    "delta": "Hello"
}

response.text.done

Description: Text output complete event in text-only mode.

event_idstring

Unique identifier for this event.

typestring

Event type. Always response.text.done.

response_idstring

Associated response ID.

item_idstring

Associated output item ID.

output_indexinteger

Index of the output item within the response.

content_indexinteger

Index of the content part within the output item.

textstring

The complete text output.

{
    "event_id": "event_xxx",
    "type": "response.text.done",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0,
    "text": "Hello, how can I help you?"
}

response.audio_transcript.delta

Description: Transcript delta event in audio mode. Streams transcript fragments incrementally.

event_idstring

Unique identifier for this event.

typestring

Event type. Always response.audio_transcript.delta.

response_idstring

Associated response ID.

item_idstring

Associated output item ID.

output_indexinteger

Index of the output item within the response.

content_indexinteger

Index of the content part within the output item.

deltastring

Incremental transcript fragment.

{
    "event_id": "event_xxx",
    "type": "response.audio_transcript.delta",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0,
    "delta": "Hello"
}

response.audio_transcript.done

Description: Transcript output complete event in audio mode.

event_idstring

Unique identifier for this event.

typestring

Event type. Always response.audio_transcript.done.

response_idstring

Associated response ID.

item_idstring

Associated output item ID.

output_indexinteger

Index of the output item within the response.

content_indexinteger

Index of the content part within the output item.

transcriptstring

The complete transcript text.

{
    "event_id": "event_xxx",
    "type": "response.audio_transcript.done",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0,
    "transcript": "Hello, how can I help you?"
}

response.audio.delta

Description: Audio data delta event in audio mode. The delta field contains Base64-encoded PCM audio data.

event_idstring

Unique identifier for this event.

typestring

Event type. Always response.audio.delta.

response_idstring

Associated response ID.

item_idstring

Associated output item ID.

output_indexinteger

Index of the output item within the response.

content_indexinteger

Index of the content part within the output item.

deltastring

Base64-encoded PCM audio data fragment.

{
    "event_id": "event_xxx",
    "type": "response.audio.delta",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0,
    "delta": "<base64-encoded audio data>"
}

response.audio.done

Description: Audio output complete event in audio mode. Contains no audio data.

event_idstring

Unique identifier for this event.

typestring

Event type. Always response.audio.done.

response_idstring

Associated response ID.

item_idstring

Associated output item ID.

output_indexinteger

Index of the output item within the response.

content_indexinteger

Index of the content part within the output item.

{
    "event_id": "event_xxx",
    "type": "response.audio.done",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0
}

response.content_part.done

Description: A content part within an output item has completed.

event_idstring

Unique identifier for this event.

typestring

Event type. Always response.content_part.done.

response_idstring

Associated response ID.

item_idstring

Associated output item ID.

output_indexinteger

Index of the output item within the response.

content_indexinteger

Index of the content part within the output item.

partobject

The completed content part.

Properties

typestring

Content type, such as audio or text.

textstring

Text content or audio transcript text.

{
    "event_id": "event_xxx",
    "type": "response.content_part.done",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0,
    "part": {
        "type": "audio",
        "text": "Hello, how can I help you?"
    }
}

response.output_item.done

Description: An output item within the response has completed.

event_idstring

Unique identifier for this event.

typestring

Event type. Always response.output_item.done.

response_idstring

Associated response ID.

output_indexinteger

Index of the output item within the response.

itemobject

The completed output item with full details.

Properties

idstring

Unique identifier for the output item.

objectstring

Always realtime.item.

typestring

Output item type: message (standard message) or function_call (function call).

statusstring

Output item status, such as completed.

rolestring

Message role. Always assistant. Only present for message type items.

contentarray

Message content list. Only present for message type items.

{
    "event_id": "event_xxx",
    "type": "response.output_item.done",
    "response_id": "resp_xxx",
    "output_index": 0,
    "item": {
        "id": "item_xxx",
        "object": "realtime.item",
        "type": "message",
        "status": "completed",
        "role": "assistant",
        "content": [
            {
                "type": "text",
                "text": "Hello, how can I help you?"
            }
        ]
    }
}

response.function_call_arguments.delta

Description: Function Calling argument delta. When the model decides to invoke a tool, the server first sends response.output_item.added (with item.type=function_call) and conversation.item.created, then streams argument fragments through this event.

event_idstring

Unique identifier for this event.

typestring

Event type. Always response.function_call_arguments.delta.

response_idstring

Associated response ID.

item_idstring

Associated output item ID.

output_indexinteger

Index of the output item within the response.

call_idstring

Unique identifier for the function call.

deltastring

Incremental fragment of the function call arguments (JSON string fragment).

{
    "event_id": "event_xxx",
    "type": "response.function_call_arguments.delta",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "call_id": "call_xxx",
    "delta": "{\"city"
}

response.function_call_arguments.done

Description: Sent when Function Calling argument output is complete. After receiving this event, the client should: execute the corresponding tool, write the result to the conversation via conversation.item.create with a function_call_output item, and then send response.create to trigger a follow-up inference round.

event_idstring

Unique identifier for this event.

typestring

Event type. Always response.function_call_arguments.done.

response_idstring

Associated response ID.

item_idstring

Associated output item ID.

output_indexinteger

Index of the output item within the response.

call_idstring

Unique identifier for the function call.

namestring

Name of the invoked function.

argumentsstring

Complete function call arguments (JSON string).

{
    "event_id": "event_xxx",
    "type": "response.function_call_arguments.done",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "call_id": "call_xxx",
    "name": "get_weather",
    "arguments": "{\"city\":\"Hangzhou\"}"
}

response.done

Description: An inference round is complete. status indicates the reason for completion.

event_idstring

Unique identifier for this event.

typestring

Event type. Always response.done.

responseobject

The complete response object.

Properties

idstring

Unique identifier of the response.

objectstring

Always realtime.response.

statusstring

Completion status of the response. Valid values:

  • completed: Completed normally.
  • cancelled: Interrupted and cancelled. status_details.reason is turn_detected (VAD interruption) or client_cancelled (client-initiated cancellation).
  • failed: LLM or TTS error.

status_detailsobject

Status details. Present only when the status is cancelled or failed.

Properties

typestring

Status type, such as cancelled.

reasonstring

Cancellation reason: turn_detected (VAD interruption) or client_cancelled (client-initiated cancellation).

modalitiesarray

Output modalities of the model.

voicestring

Voice used for audio generation. Either a system voice name or the voice_id of a cloned voice.

outputarray

List of output items in the response, containing complete item objects.

{
    "event_id": "event_xxx",
    "type": "response.done",
    "response": {
        "id": "resp_xxx",
        "object": "realtime.response",
        "status": "completed",
        "modalities": ["text", "audio"],
        "voice": "longanqian",
        "output": [
            {
                "id": "item_xxx",
                "object": "realtime.item",
                "type": "message",
                "status": "completed",
                "role": "assistant",
                "content": [
                    {
                        "type": "audio",
                        "transcript": "Hello, how can I help you?"
                    }
                ]
            }
        ]
    }
}

{
    "event_id": "event_xxx",
    "type": "response.done",
    "response": {
        "id": "resp_xxx",
        "status": "cancelled",
        "status_details": {
            "type": "cancelled",
            "reason": "turn_detected"
        }
    }
}

voiceprint_audio_list.in_progress

Description: The voiceprint registration process is in progress asynchronously.

event_idstring

Unique identifier for this event.

typestring

Event type. Always voiceprint_audio_list.in_progress.

item_idstring

Unique identifier for the voiceprint registration task.

{
    "event_id": "event_PaEcN7CCrlhE8q4MM2yND",
    "type": "voiceprint_audio_list.in_progress",
    "item_id": "vp_Y12cA986j1KZ9O9YmAXOA"
}

voiceprint_audio_list.completed

Description: The voiceprint registration process has completed successfully.

event_idstring

Unique identifier for this event.

typestring

Event type. Always voiceprint_audio_list.completed.

item_idstring

Unique identifier for the voiceprint registration task.

{
    "event_id": "event_PaEcN7CCrlhE8q4MM2yND",
    "type": "voiceprint_audio_list.completed",
    "item_id": "vp_Y12cA986j1KZ9O9YmAXOA"
}

voiceprint_audio_list.failed

Description: The voiceprint registration failed. Normal conversation calls are not blocked.

event_idstring

Unique identifier for this event.

typestring

Event type. Always voiceprint_audio_list.failed.

item_idstring

Unique identifier for the voiceprint registration task.

reasonstring

Description of the failure reason.

{
    "event_id": "event_PaEcN7CCrlhE8q4MM2yND",
    "type": "voiceprint_audio_list.failed",
    "item_id": "vp_Y12cA986j1KZ9O9YmAXOA",
    "reason": ""
}
Previous：Client eventsNext：Real-time Multimodal