Qwen-Audio Realtime API server events-阿里云帮助中心

error

Description: Returned when a request fails or a service error occurs. Client errors (invalid_request_error) keep the connection open. Server errors (server_error) terminate the connection.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "error",
    "error": {
        "type": "invalid_request_error",
        "code": "invalid_value",
        "message": "Cannot create response while another response is in progress.",
        "param": "response.create"
    }
}

type string

Event type. Always error.

error object

Error details.

Properties

type string

Error type, such as invalid_request_error (client error) or server_error (server error).

code string

Error code.

message string

Error message.

param string

The parameter associated with the error.

session.created

Description: The first event sent by the server after the connection is established. Contains the default session configuration.

event_id string

Unique identifier for this event.

{
    "event_id": "event_KiKZC2zrhNsKFPZ5cTpyA",
    "type": "session.created",
    "session": {
        "object": "realtime.session",
        "model": "qwen-audio-3.0-realtime-plus",
        "modalities": ["text", "audio"],
        "voice": "longanqian",
        "input_audio_transcription": {
            "model": "fun-asr"
        },
        "turn_detection": {
            "type": "server_vad",
            "threshold": 0.5,
            "silence_duration_ms": 800
        },
        "id": "sess_A1LbG2D63WELBSawRbpq8"
    }
}

type string

Event type. Always session.created.

session object

Session configuration.

Properties

object string

Always realtime.session.

model string

Model name.

modalities array

Output modalities of the model.

voice string

Voice used for audio generation. Either a system voice name or the voice_id of a cloned voice.

input_audio_transcription object

Audio transcription configuration.

Properties

model string

Transcription model, such as fun-asr.

turn_detection object

Turn detection (VAD) configuration.

id string

Unique session identifier.

session.updated

Description: Returned after a session.update request is processed successfully. Contains the full updated session configuration. If the request fails, an error event is returned instead.

event_id string

Unique identifier for this event.

{
    "event_id": "event_FMG6kiHbILCGiqXFPA98e",
    "type": "session.updated",
    "session": {
        "id": "sess_A1LbG2D63WELBSawRbpq8",
        "object": "realtime.session",
        "model": "pre-fun-realtime-audiochat-beta-3",
        "modalities": ["text", "audio"],
        "voice": "longanqian",
        "input_audio_transcription": {
            "model": "fun-asr"
        },
        "turn_detection": {
            "type": "smart_turn",
            "threshold": 0.1,
            "silence_duration_ms": 900
        }
    }
}

type string

Event type. Always session.updated.

session object

Full session configuration after the update. Shares the same structure as the session object in session.created.

input_audio_buffer.speech_started

Description: Sent when Voice Activity Detection (VAD) detects the start of speech in server_vad / smart_turn mode.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "input_audio_buffer.speech_started",
    "audio_start_ms": 1200,
    "item_id": "item_xxx"
}

type string

Event type. Always input_audio_buffer.speech_started.

audio_start_ms integer

Timestamp in milliseconds when speech started.

item_id string

ID of the item that will be created when this speech segment is committed.

input_audio_buffer.speech_stopped

Description: Sent when VAD detects the end of speech in server_vad / smart_turn mode.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "input_audio_buffer.speech_stopped",
    "audio_end_ms": 3400,
    "item_id": "item_xxx",
    "reason": "turn_invalid"
}

type string

Event type. Always input_audio_buffer.speech_stopped.

audio_end_ms integer

Timestamp in milliseconds when speech ended.

item_id string

ID of the user message item to be created.

reason string

Returned only in smart_turn mode. When set to turn_invalid, indicates the current turn was classified as invalid (no semantic content) and inference will not be triggered. This field is absent for valid turns.

input_audio_buffer.committed

Description: The audio buffer has been committed as a user message, either through a push-to-talk commit or automatic VAD commit.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "input_audio_buffer.committed",
    "previous_item_id": "item_xxx",
    "item_id": "item_xxx"
}

type string

Event type. Always input_audio_buffer.committed.

previous_item_id string

ID of the previous conversation item.

item_id string

ID of the created user message item.

input_audio_buffer.cleared

Description: The audio buffer has been cleared (push-to-talk mode only).

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "input_audio_buffer.cleared"
}

type string

Event type. Always input_audio_buffer.cleared.

conversation.item.created

Description: A new conversation item was created. Triggered when user audio is committed, the client manually creates an item, or an assistant response begins.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "conversation.item.created",
    "previous_item_id": "item_xxx",
    "item": {
        "id": "item_xxx",
        "object": "realtime.item",
        "type": "message",
        "status": "in_progress",
        "role": "assistant",
        "content": []
    }
}

type string

Event type. Always conversation.item.created.

previous_item_id string

ID of the previous conversation item.

item object

The created conversation item.

Properties

id string

Unique identifier for the conversation item.

object string

Always realtime.item.

type string

Item type: message (standard message) or function_call (function call).

status string

Item status, such as in_progress or completed.

role string

Message role, such as user or assistant. Only present for message type items.

content array

Message content list. Only present for message type items.

conversation.item.deleted

Description: A conversation item was deleted. Returned as a confirmation after the client sends a conversation.item.delete event.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "conversation.item.deleted",
    "item_id": "item_xxx"
}

type string

Event type. Always conversation.item.deleted.

item_id string

ID of the deleted conversation item.

conversation.item.retrieved

Description: A conversation item was retrieved successfully. Returned after the client sends a conversation.item.retrieve event. For audio-type content, only the transcript text is included; raw audio data isn't returned.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "conversation.item.retrieved",
    "item": {
        "id": "item_xxx",
        "object": "realtime.item",
        "type": "message",
        "role": "user",
        "content": [
            {
                "type": "input_audio",
                "transcript": "Hello"
            }
        ]
    }
}

type string

Event type. Always conversation.item.retrieved.

item object

The retrieved conversation item.

Properties

id string

Unique identifier for the conversation item.

object string

Always realtime.item.

type string

Item type: message (standard message) or function_call (function call).

role string

Message role, such as user or assistant. Only present for message type items.

content array

Message content list. For audio-type content, only the transcript text is included; raw audio data isn't returned.

conversation.item.input_audio_transcription.delta

Description: Incremental Automatic Speech Recognition (ASR) transcription result, streamed during speech recognition. Includes emotion and language detection information.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "conversation.item.input_audio_transcription.delta",
    "item_id": "item_xxx",
    "content_index": 0,
    "text": "Hello",
    "stash": "world"
}

type string

Event type. Always conversation.item.input_audio_transcription.delta.

item_id string

ID of the associated conversation item.

content_index integer

Index of the content part.

text string

Finalized transcription text.

stash string

Tentative text that hasn't been finalized yet.

conversation.item.input_audio_transcription.completed

Description: Final ASR transcription result. The transcribed text is written to the transcript field of the corresponding item.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "conversation.item.input_audio_transcription.completed",
    "item_id": "item_xxx",
    "content_index": 0,
    "transcript": "Hello world"
}

type string

Event type. Always conversation.item.input_audio_transcription.completed.

item_id string

ID of the associated conversation item.

content_index integer

Index of the content part.

transcript string

The complete transcription text.

conversation.item.input_audio_transcription.failed

Description: ASR transcription failed.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "conversation.item.input_audio_transcription.failed",
    "item_id": "item_xxx",
    "content_index": 0,
    "error": {
        "type": "transcription_error",
        "code": "transcription_failed",
        "message": "ASR transcription failed"
    }
}

type string

Event type. Always conversation.item.input_audio_transcription.failed.

item_id string

ID of the associated conversation item.

content_index integer

Index of the content part.

error object

Error details.

Properties

type string

Error type, such as transcription_error.

code string

Error code, such as transcription_failed.

message string

Error message.

conversation.item.ambient_audio_transcription.delta

Description: smart_turn mode only. Incremental transcription result for ambient audio. When VAD detects voice activity but semantic analysis classifies the turn as invalid (for example, background noise or filler words like "hmm" or "uh"), the ASR result is forwarded to the client as an ambient event. This event isn't linked to any conversation item. The item_id is a standalone temporary ID.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "conversation.item.ambient_audio_transcription.delta",
    "item_id": "item_xxx",
    "content_index": 0,
    "text": "Hmm",
    "stash": ""
}

type string

Event type. Always conversation.item.ambient_audio_transcription.delta.

item_id string

A standalone temporary ID, not associated with any conversation item.

content_index integer

Index of the content part.

text string

Finalized transcription text.

stash string

Tentative text that hasn't been finalized yet.

conversation.item.ambient_audio_transcription.completed

Description: smart_turn mode only. Final transcription result for ambient audio. Paired with the corresponding delta event to mark the end of an ambient audio transcription segment. This result isn't written to the conversation context.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "conversation.item.ambient_audio_transcription.completed",
    "item_id": "item_xxx",
    "content_index": 0,
    "transcript": "Hmm",
}

type string

Event type. Always conversation.item.ambient_audio_transcription.completed.

item_id string

A standalone temporary ID, not associated with the conversation context.

content_index integer

Index of the content part.

transcript string

The complete transcription text.

response.created

Description: Sent when a model inference round starts.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "response.created",
    "response": {
        "id": "resp_xxx",
        "object": "realtime.response",
        "status": "in_progress",
        "modalities": ["text", "audio"],
        "voice": "longanqian",
        "output": []
    }
}

type string

Event type. Always response.created.

response object

The response object.

Properties

id string

Unique identifier for the response.

object string

Always realtime.response.

status string

Response status, such as in_progress.

modalities array

Output modalities of the model.

voice string

Voice used for audio generation. Either a system voice name or the voice_id of a cloned voice.

output array

Output items of the response. Initially an empty array.

response.output_item.added

Description: A new output item was added to the response. The output item type is message for standard replies, or function_call for Function Calling.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "response.output_item.added",
    "response_id": "resp_xxx",
    "output_index": 0,
    "item": {
        "id": "item_xxx",
        "object": "realtime.item",
        "type": "message",
        "status": "in_progress",
        "role": "assistant",
        "content": []
    }
}

Function Calling output item example:

When the output item is a function call, the response.output_item.added / conversation.item.created / response.output_item.done events contain the following item structure:

{
          "id": "item_xxx",
          "object": "realtime.item",
          "type": "function_call",
          "status": "completed",
          "call_id": "call_xxx",
          "name": "get_weather",
          "arguments": "{\"city\":\"Hangzhou\"}"
      }

Note

A single response can include multiple function_call items and may also contain standard message output alongside function_call output. The Function Calling portion isn't sent to TTS for audio playback.

type string

Event type. Always response.output_item.added.

response_id string

Associated response ID.

output_index integer

Index of the output item within the response.

item object

The added output item.

Properties

id string

Unique identifier for the output item.

object string

Always realtime.item.

type string

Output item type: message (standard message) or function_call (function call).

status string

Output item status, such as in_progress.

role string

Message role. Always assistant. Only present for message type items.

content array

Message content list. Only present for message type items.

response.content_part.added

Description: A new content part was added to an output item.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "response.content_part.added",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0,
    "part": {
        "type": "audio",
        "text": ""
    }
}

type string

Event type. Always response.content_part.added.

response_id string

Associated response ID.

item_id string

Associated output item ID.

output_index integer

Index of the output item within the response.

content_index integer

Index of the content part within the output item.

part object

The added content part.

Properties

type string

Content type, such as audio or text.

text string

Text content. Initially an empty string.

response.text.delta

Description: Text delta event in text-only mode. Streams text fragments incrementally.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "response.text.delta",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0,
    "delta": "Hello"
}

type string

Event type. Always response.text.delta.

response_id string

Associated response ID.

item_id string

Associated output item ID.

output_index integer

Index of the output item within the response.

content_index integer

Index of the content part within the output item.

delta string

Incremental text fragment.

response.text.done

Description: Text output complete event in text-only mode.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "response.text.done",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0,
    "text": "Hello, how can I help you?"
}

type string

Event type. Always response.text.done.

response_id string

Associated response ID.

item_id string

Associated output item ID.

output_index integer

Index of the output item within the response.

content_index integer

Index of the content part within the output item.

text string

The complete text output.

response.audio_transcript.delta

Description: Transcript delta event in audio mode. Streams transcript fragments incrementally.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "response.audio_transcript.delta",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0,
    "delta": "Hello"
}

type string

Event type. Always response.audio_transcript.delta.

response_id string

Associated response ID.

item_id string

Associated output item ID.

output_index integer

Index of the output item within the response.

content_index integer

Index of the content part within the output item.

delta string

Incremental transcript fragment.

response.audio_transcript.done

Description: Transcript output complete event in audio mode.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "response.audio_transcript.done",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0,
    "transcript": "Hello, how can I help you?"
}

type string

Event type. Always response.audio_transcript.done.

response_id string

Associated response ID.

item_id string

Associated output item ID.

output_index integer

Index of the output item within the response.

content_index integer

Index of the content part within the output item.

transcript string

The complete transcript text.

response.audio.delta

Description: Audio data delta event in audio mode. The delta field contains Base64-encoded PCM audio data.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "response.audio.delta",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0,
    "delta": "<base64-encoded audio data>"
}

type string

Event type. Always response.audio.delta.

response_id string

Associated response ID.

item_id string

Associated output item ID.

output_index integer

Index of the output item within the response.

content_index integer

Index of the content part within the output item.

delta string

Base64-encoded PCM audio data fragment.

response.audio.done

Description: Audio output complete event in audio mode. Contains no audio data.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "response.audio.done",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0
}

type string

Event type. Always response.audio.done.

response_id string

Associated response ID.

item_id string

Associated output item ID.

output_index integer

Index of the output item within the response.

content_index integer

Index of the content part within the output item.

response.content_part.done

Description: A content part within an output item has completed.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "response.content_part.done",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "content_index": 0,
    "part": {
        "type": "audio",
        "text": "Hello, how can I help you?"
    }
}

type string

Event type. Always response.content_part.done.

response_id string

Associated response ID.

item_id string

Associated output item ID.

output_index integer

Index of the output item within the response.

content_index integer

Index of the content part within the output item.

part object

The completed content part.

Properties

type string

Content type, such as audio or text.

text string

Text content or audio transcript text.

response.output_item.done

Description: An output item within the response has completed.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "response.output_item.done",
    "response_id": "resp_xxx",
    "output_index": 0,
    "item": {
        "id": "item_xxx",
        "object": "realtime.item",
        "type": "message",
        "status": "completed",
        "role": "assistant",
        "content": [
            {
                "type": "text",
                "text": "Hello, how can I help you?"
            }
        ]
    }
}

type string

Event type. Always response.output_item.done.

response_id string

Associated response ID.

output_index integer

Index of the output item within the response.

item object

The completed output item with full details.

Properties

id string

Unique identifier for the output item.

object string

Always realtime.item.

type string

Output item type: message (standard message) or function_call (function call).

status string

Output item status, such as completed.

role string

Message role. Always assistant. Only present for message type items.

content array

Message content list. Only present for message type items.

response.function_call_arguments.delta

Description: Function Calling argument delta. When the model decides to invoke a tool, the server first sends response.output_item.added (with item.type=function_call) and conversation.item.created, then streams argument fragments through this event.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "response.function_call_arguments.delta",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "call_id": "call_xxx",
    "delta": "{\"city"
}

type string

Event type. Always response.function_call_arguments.delta.

response_id string

Associated response ID.

item_id string

Associated output item ID.

output_index integer

Index of the output item within the response.

call_id string

Unique identifier for the function call.

delta string

Incremental fragment of the function call arguments (JSON string fragment).

response.function_call_arguments.done

Description: Sent when Function Calling argument output is complete. After receiving this event, the client should: execute the corresponding tool, write the result to the conversation via conversation.item.create with a function_call_output item, and then send response.create to trigger a follow-up inference round.

event_id string

Unique identifier for this event.

{
    "event_id": "event_xxx",
    "type": "response.function_call_arguments.done",
    "response_id": "resp_xxx",
    "item_id": "item_xxx",
    "output_index": 0,
    "call_id": "call_xxx",
    "name": "get_weather",
    "arguments": "{\"city\":\"Hangzhou\"}"
}

type string

Event type. Always response.function_call_arguments.done.

response_id string

Associated response ID.

item_id string

Associated output item ID.

output_index integer

Index of the output item within the response.

call_id string

Unique identifier for the function call.

name string

Name of the invoked function.

arguments string

Complete function call arguments (JSON string).

response.done

Description: An inference round is complete. status indicates the reason for completion.

event_id string

Unique identifier for this event.

Completed normally

{
    "event_id": "event_xxx",
    "type": "response.done",
    "response": {
        "id": "resp_xxx",
        "object": "realtime.response",
        "status": "completed",
        "modalities": ["text", "audio"],
        "voice": "longanqian",
        "output": [
            {
                "id": "item_xxx",
                "object": "realtime.item",
                "type": "message",
                "status": "completed",
                "role": "assistant",
                "content": [
                    {
                        "type": "audio",
                        "transcript": "Hello, how can I help you?"
                    }
                ]
            }
        ]
    }
}

Interrupted and cancelled

{
    "event_id": "event_xxx",
    "type": "response.done",
    "response": {
        "id": "resp_xxx",
        "status": "cancelled",
        "status_details": {
            "type": "cancelled",
            "reason": "turn_detected"
        }
    }
}

type string

Event type. Always response.done.

response object

The complete response object.

Properties

id string

Unique identifier of the response.

object string

Always realtime.response.

status string

Completion status of the response. Valid values:

completed: Completed normally.
cancelled: Interrupted and cancelled. status_details.reason is turn_detected (VAD interruption) or client_cancelled (client-initiated cancellation).
failed: LLM or TTS error.

status_details object

Status details. Present only when the status is cancelled or failed.

Properties

type string

Status type, such as cancelled.

reason string

Cancellation reason: turn_detected (VAD interruption) or client_cancelled (client-initiated cancellation).

modalities array

Output modalities of the model.

voice string

Voice used for audio generation. Either a system voice name or the voice_id of a cloned voice.

output array

List of output items in the response, containing complete item objects.

voiceprint_audio_list.in_progress

Description: The voiceprint registration process is in progress asynchronously.

event_id string

Unique identifier for this event.

{
    "event_id": "event_PaEcN7CCrlhE8q4MM2yND",
    "type": "voiceprint_audio_list.in_progress",
    "item_id": "vp_Y12cA986j1KZ9O9YmAXOA"
}

type string

Event type. Always voiceprint_audio_list.in_progress.

item_id string

Unique identifier for the voiceprint registration task.

voiceprint_audio_list.completed

Description: The voiceprint registration process has completed successfully.

event_id string

Unique identifier for this event.

{
    "event_id": "event_PaEcN7CCrlhE8q4MM2yND",
    "type": "voiceprint_audio_list.completed",
    "item_id": "vp_Y12cA986j1KZ9O9YmAXOA"
}

type string

Event type. Always voiceprint_audio_list.completed.

item_id string

Unique identifier for the voiceprint registration task.

voiceprint_audio_list.failed

Description: The voiceprint registration failed. Normal conversation calls are not blocked.

event_id string

Unique identifier for this event.

{
    "event_id": "event_PaEcN7CCrlhE8q4MM2yND",
    "type": "voiceprint_audio_list.failed",
    "item_id": "vp_Y12cA986j1KZ9O9YmAXOA",
    "reason": ""
}

type string

Event type. Always voiceprint_audio_list.failed.

item_id string

Unique identifier for the voiceprint registration task.

reason string

Description of the failure reason.