DocsICP FilingConsole
官方文档
首页

Server events

更新时间：
复制 MD 格式
产品详情
我的收藏

Server-side events for the qwen3.5-livetranslate-flash-realtime API.

Reference: Real-time audio and video translation - Qwen

error

Error message returned by the server.

event_id string

The unique identifier for this event.

 
{
  "event_id": "event_RoUu4T8yExPMI37GKwaOC",
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "code": "invalid_value",
    "message": "Invalid modalities: ['audio']. Supported combinations are: ['text'] and ['audio', 'text'].",
    "param": "session.modalities"
  }
}

type string

The event type. The value is always error.

error object

Detailed information about the error.

Properties

type string

The error type.

code string

The error code.

message string

The error message.

param string

The parameter that is related to the error, such as session.modalities.

session.created

When a client connects, the server returns this event first with the default session configurations.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_QxBGpjBDmDDQQWDtrqBKB",
    "type": "session.created",
    "session": {
        "id": "sess_OozZ1vtbPt2muDflHODIH",
        "object": "realtime.session",
        "model": "qwen3.5-livetranslate-flash-realtime",
        "modalities": [
            "text",
            "audio"
        ],
        "voice": "Cherry",
        "input_audio_format": "pcm",
        "sample_rate": 16000,
        "output_audio_format": "pcm",
        "turn_detection": {
            "type": "server_vad",
            "threshold": 0.2,
            "silence_duration_ms": 1000
        },
        "translation": {
           "language": "en",
           "corpus": {
               "phrases": {
                   "人工智能": "Artificial Intelligence",
                   "机器学习": "Machine Learning"
               }
           }
        }
    }
}

type string

The event type. The value is always session.created.

session object

The session configuration.

Properties

id string

The unique identifier for the session.

object string

The value is always realtime.session.

model string

The model in use.

modalities array

The output modality settings for the model.

voice string

The voice for the audio generated by the model.

input_audio_format string

The format of the input audio. Defaults to pcm.

sample_rate integer

The sample rate of the input audio, in Hz.

output_audio_format string

The format of the output audio. Defaults to pcm.

turn_detection object

The VAD (Voice Activity Detection) configuration.

translation object (Optional)

The translation configuration.

Properties

language string (Optional)

The target language for translation.

corpus object (Optional)

Custom terminology configuration for improving translation accuracy of domain-specific terms.

Properties

corpus.phrases object (Optional)

A mapping of source-language terms to target-language translations. For details, see Supported languages.

session.updated

Returned after a successful session.update request. If the request fails, an error event is returned instead.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_QxBGpjBDmDDQQWDtrqBKB",
    "type": "session.updated",
    "session": {
        "id": "sess_OozZ1vtbPt2muDflHODIH",
        "object": "realtime.session",
        "model": "qwen3.5-livetranslate-flash-realtime",
        "modalities": [
            "text",
            "audio"
        ],
        "voice": "Ethan",
        "sample_rate": 16000,
        "input_audio_format": "pcm",
        "output_audio_format": "pcm",
        "input_audio_transcription": {
            "model": "qwen3-asr-flash-realtime",
            "language": "zh"
        },
        "turn_detection": {
            "type": "server_vad",
            "threshold": 0.2,
            "prefix_padding_ms": 300,
            "silence_duration_ms": 1000,
            "create_response": true,
            "interrupt_response": true
        },
        "translation": {
           "language": "en",
           "corpus": {
               "phrases": {
                   "人工智能": "Artificial Intelligence",
                   "机器学习": "Machine Learning"
               }
           }
        },
        "enable_voice_clone": true,
        "voice_clone_options": {
            "frequency": "once"
        }
    }
}

type string

The event type. The value is always session.updated.

session object

The session configuration.

Properties

id string

The unique identifier for the session.

object string

The value is always realtime.session.

model string

The model in use.

modalities array

The output modality settings for the model.

voice string

The voice for the audio generated by the model.

sample_rate integer (Optional)

The sample rate of the input audio, in Hz.

input_audio_format string

The format of the input audio. The value is always pcm.

output_audio_format string

The format of the output audio. The value is always pcm.

input_audio_transcription object

The input audio transcription configuration. Returned only when the input_audio_transcription.model parameter is configured in the session.

Properties

model string

The speech recognition model.

language string

The configured speech recognition language.

turn_detection object

The VAD (Voice Activity Detection) configuration. This field is not returned in Manual mode (when the client sets this parameter to null in session.update).

Properties

type string

The VAD type. The value is always server_vad.

threshold float

The VAD detection sensitivity.

prefix_padding_ms integer

The duration of audio (in milliseconds) retained before speech starts, to avoid losing the beginning of speech.

silence_duration_ms integer

The minimum duration of silence (in milliseconds) required after speech ends to determine that speech has concluded.

create_response boolean

Whether to automatically trigger a translation response after VAD detects that speech has ended.

interrupt_response boolean

Whether to interrupt the current translation response being generated when VAD detects a new speech turn.

translation object (Optional)

The translation configuration.

Properties

language string (Optional)

The target language for translation.

corpus object (Optional)

Custom terminology configuration for improving translation accuracy of domain-specific terms.

Properties

corpus.phrases object (Optional)

A mapping of source-language terms to target-language translations.

enable_voice_clone boolean

Whether to enable voice cloning.

voice_clone_options object

Voice cloning control parameters. Returned only when enable_voice_clone is true.

Properties

frequency string

The voice cloning frequency.

session.finished

Session is finished and all audio translations are complete.

Sent only after the client sends a session.finish request. The client can then disconnect.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_xxx",
    "type": "session.finished"
}

type string

The event type. The value is always session.finished.

response.created

Returned when the server generates a new model response.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_L8hHVI5jYis6BzAjnPWJh",
    "type": "response.created",
    "response": {
        "id": "resp_P79OOMs8LnrXVpiIHUCKR",
        "object": "realtime.response",
        "conversation_id": "conv_UFClXtYkRkFXrs48y8pmK",
        "status": "in_progress",
        "modalities": [
            "text",
            "audio"
        ],
        "voice": "Cherry",
        "output_audio_format": "pcm16",
        "output": []
    }
}

type string

The event type. The value is always response.created.

response object

The response object.

Properties

id string

The unique identifier for the response.

conversation_id string

The unique identifier for the current session.

object string

The object type. For this event, the value is always realtime.response.

status string

The response status. Valid values:

  • completed

  • failed

  • in_progress

  • incomplete

modalities array

Response modality.

voice string

The voice of the generated audio.

output_audio_format string

The format of the output audio.

output array

This event is currently empty.

response.done

Returned after response generation is complete. Contains all output items except raw audio data.

event_id string

The unique identifier for this event.

 
{
  "event_id": "event_CNea8oXNipVanSg2VIzkO",
  "type": "response.done",
  "response": {
    "id": "resp_TfhYTqej692vsGA2jNEtH",
    "object": "realtime.response",
    "conversation_id": "conv_ZtyLfKVm8XqLwYRlsuDih",
    "status": "completed",
    "modalities": [
      "text",
      "audio"
    ],
    "voice": "Cherry",
    "output_audio_format": "pcm16",
    "output": [
      {
        "id": "item_MKtkMwN9RtcyE9eJShyWy",
        "object": "realtime.item",
        "type": "message",
        "status": "completed",
        "role": "assistant",
        "content": [
          {
            "type": "audio",
            "transcript": "Hello? "
          }
        ]
      }
    ],
    "usage": {
      "total_tokens": 56,
      "input_tokens": 47,
      "output_tokens": 9,
      "input_tokens_details": {
        "text_tokens": 20,
        "audio_tokens": 27
      },
      "output_tokens_details": {
        "text_tokens": 2,
        "audio_tokens": 7
      }
    }
  }
}

type string

The event type. The value is always response.done.

response object

The response object.

Properties

id string

The unique identifier for the response.

conversation_id string

The unique identifier for the current session.

object string

The object type. For this event, the value is always realtime.response.

status string

The status of the response.

modalities array

The modality of the response.

voice string

The voice used for the audio generated by the model.

output_audio_format string

The format of the output audio.

output object

The output of the response.

Properties

id string

The unique identifier for the response output.

type string

The type of the output item. The value is currently always message.

object string

The object type of the output item. The value is currently always realtime.item.

status string

The status of the output item.

role string

The role of the output item.

content array

The content of the output item.

Properties

type string

The type of the output content. The value is text for plain text output and audio when the output includes audio.

text string

The text content of the output.

transcript string

The text transcription of the audio content.

usage object

The token consumption information for this response.

response.text.text

Returned for text-only output when the model generates text incrementally.

event_id string

A unique identifier for the event.

 
{
    "event_id": "event_B1lIeyOXR7qJMEExbqtTG",
    "type": "response.text.text",
    "response_id": "resp_B1lIdtjF4Noqpn5NOjznj",
    "item_id": "item_B1lIdJsAJlJiFs8ztWpJt",
    "output_index": 0,
    "content_index": 0,
    "text": "How are",
    "stash": " you today?"
}

type string

The type of the event. The value is always response.text.text.

text string

The incremental text that is returned.

response_id string

The response ID.

item_id string

A unique identifier for the message item.

output_index integer

Currently, the value is always 0.

content_index integer

Currently, the value is always 0.

stash string

Temporary text generated by the model. Concatenate stash with the current text to form an interim result. The system updates text and stash continuously using response.text.text events until it receives a response.text.done event. The final complete text is then available in the text field.

response.text.done

Returned when text generation finishes for text-only output, or if the response is interrupted, incomplete, or canceled.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_B1lIeE2Nac33zn5V7h2mm",
    "type": "response.text.done",
    "response_id": "resp_B1lIdtjF4Noqpn5NOjznj",
    "item_id": "item_B1lIdJsAJlJiFs8ztWpJt",
    "output_index": 0,
    "content_index": 0,
    "text": "How can I assist you today?"
}

type string

The event type. The value is always response.text.done.

response_id string

The unique identifier for the response.

item_id string

The unique identifier for the message item.

output_index integer

The value is currently always 0.

content_index integer

The value is currently always 0.

text string

The complete text output from the model.

response.audio.delta

Returned when audio output is enabled and the model generates audio incrementally.

event_id string

A unique identifier for the event.

 
{
    "event_id": "event_B1osWMZBtrEQbiIwW0qHQ",
    "type": "response.audio.delta",
    "response_id": "resp_P79OOMs8LnrXVpiIHUCKR",
    "item_id": "item_OFaPGtzfWCPyGzxnuEX9i",
    "output_index": 0,
    "content_index": 0,
    "delta": "UklGRnoGAABXQVZFZm10IBAAAAAB..."
}

type string

The event type. The value is always response.audio.delta.

response_id string

A unique identifier for the response.

item_id string

A unique identifier for the message item.

output_index integer

The value is always 0.

content_index integer

The value is always 0.

delta string

The incremental audio data that is output by the model. The data is Base64-encoded.

response.audio.done

Returned when audio generation is complete. Also returned if the response is interrupted, incomplete, or canceled. Does not contain complete audio data.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_B1osWMWoDRYyITDyNYcBu",
    "type": "response.audio.done",
    "response_id": "resp_P79OOMs8LnrXVpiIHUCKR",
    "item_id": "item_OFaPGtzfWCPyGzxnuEX9i",
    "output_index": 0,
    "content_index": 0
}

type string

The event type. This is always response.audio.done.

response_id string

The unique identifier for the response.

item_id string

The unique identifier for the message item.

output_index integer

The value is always 0.

content_index integer

The value is always 0.

input_audio_buffer.speech_started

Returned when the server-side VAD (Voice Activity Detection) detects that the user has started speaking.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_xxx",
    "type": "input_audio_buffer.speech_started",
    "audio_start_ms": 568,
    "item_id": "item_xxx"
}

type string

The event type. The value is always input_audio_buffer.speech_started.

audio_start_ms integer

The time point (in milliseconds) when speech was detected to start, as an offset from the beginning of the audio stream.

item_id string

The unique identifier for the associated message item.

input_audio_buffer.speech_stopped

Returned when the server-side VAD detects that the user has stopped speaking, indicating the end of the current speech input turn. Because the translation response is generated synchronously with the streaming speech input, the actual translation response may have already started during the speech input process, without needing to wait for this event.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_xxx",
    "type": "input_audio_buffer.speech_stopped",
    "audio_end_ms": 3900,
    "item_id": "item_xxx"
}

type string

The event type. The value is always input_audio_buffer.speech_stopped.

audio_end_ms integer

The time point (in milliseconds) when speech was detected to end, as an offset from the beginning of the audio stream.

item_id string

The unique identifier for the associated message item.

input_audio_buffer.committed

In Manual mode (turn_detection is null), after the client sends an input_audio_buffer.commit event, the server returns this event as confirmation and automatically starts generating the translation response.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_xxx",
    "type": "input_audio_buffer.committed"
}

type string

The event type. The value is always input_audio_buffer.committed.

input_audio_buffer.cleared

After the client sends an input_audio_buffer.clear event, the server returns this event as confirmation that the uncommitted audio data in the buffer has been cleared.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_xxx",
    "type": "input_audio_buffer.cleared"
}

type string

The event type. The value is always input_audio_buffer.cleared.

conversation.item.created

Returned when a new message item is created in the conversation. This event is triggered in the following scenarios:

  • When the server starts generating a translation response, it creates a corresponding assistant message item (at this point, content is an empty array, and the content is progressively populated with the streaming response).

  • In Manual mode, after the client sends an input_audio_buffer.commit event, the server additionally creates a message item corresponding to the user's input audio (content contains {"type": "input_audio"}).

For the same VAD segment, the server creates separate message items for the speech recognition result and the translation result. The item.id of the speech recognition result matches the previous_item_id in the translation result event. Use these values to associate and display the source text with its translation.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_xxx",
    "type": "conversation.item.created",
    "previous_item_id": "item_asr_xxx",
    "item": {
        "id": "item_translation_xxx",
        "object": "realtime.item",
        "type": "message",
        "status": "in_progress",
        "role": "assistant",
        "content": []
    }
}

type string

The event type. The value is always conversation.item.created.

previous_item_id string

The unique identifier of the previous message item. For a translation result event, this value matches the item.id of the speech recognition result for the same VAD segment.

item object

The message item information.

Properties

id string

The unique identifier for the message item.

type string

The value is always message.

object string

The value is always realtime.item.

status string

The status of the message item.

role string

The role of the message. Valid values: assistant or user.

content array

The content of the message. When a response is first created, this is an empty array that is progressively populated with the streaming response. For user message items created by commit in Manual mode, it contains {"type": "input_audio"}.

conversation.item.input_audio_transcription.text

When input_audio_transcription.model is configured, the server streams speech recognition results in the original source language.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_xxx",
    "type": "conversation.item.input_audio_transcription.text",
    "item_id": "item_xxx",
    "content_index": 0,
    "text": "",
    "stash": "The weather is really nice today",
    "language": "zh",
    "emotion": "neutral"
}

type string

The event type. The value is always conversation.item.input_audio_transcription.text.

item_id string

The unique identifier for the message item.

content_index integer

The value is currently always 0.

text string

The confirmed recognition text.

stash string

The recognition text that is pending confirmation. This text may be corrected by subsequent events.

language string

The detected source language.

emotion string

The detected emotion of the recognized audio. Supported emotions are as follows:

  • surprised: Surprised

  • neutral: Neutral

  • happy: Happy

  • sad: Sad

  • disgusted: Disgusted

  • angry: Angry

  • fearful: Fearful

conversation.item.input_audio_transcription.completed

When input_audio_transcription.model is configured, returns the final recognition result after speech recognition completes.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_xxx",
    "type": "conversation.item.input_audio_transcription.completed",
    "item_id": "item_xxx",
    "content_index": 0,
    "transcript": "What a beautiful day! Let's go for a walk in the park together.",
    "language": "en",
    "emotion": ""
}

type string

The event type. This is always conversation.item.input_audio_transcription.completed.

item_id string

The unique identifier for the message item.

content_index integer

This is currently always 0.

transcript string

The complete speech recognition result in the original source language.

language string

The detected source language.

emotion string

The detected emotion of the recognized audio. Supported emotions:

  • surprised: surprised

  • neutral: calm/neutral

  • happy: happy

  • sad: sad

  • disgusted: disgusted

  • angry: angry

  • fearful: fearful

conversation.item.input_audio_transcription.failed

Returned when audio input was received but speech recognition failed. This event is handled separately from other error events, making it easier for the client to identify the specific associated item.

type string

The event type. The value is always conversation.item.input_audio_transcription.failed.

 
{
    "event_id": "event_xxx",
    "type": "conversation.item.input_audio_transcription.failed",
    "item_id": "item_xxx",
    "content_index": 0,
    "error": {
        "code": "xxx",
        "message": "xxx",
        "param": "xxx"
    }
}

item_id string

The ID of the associated conversation item.

content_index integer

The index of the content part that contains the audio.

error.code string

The error code.

error.message string

The error message.

response.audio_transcript.text

Returned for audio output to display real-time translation.

event_id string

The unique identifier for this event.

 
{
  "event_id": "event_xxx",
  "type": "response.audio_transcript.text",
  "response_id": "resp_xxx",
  "item_id": "item_xxx",
  "output_index": 0,
  "content_index": 0,
  "text": "Hello,",
  "stash": " who are you?"
}

type string

The type of the event. The value is always response.audio_transcript.text.

response_id string

The unique identifier for the response.

item_id string

The unique identifier for the message item.

output_index integer

Currently, the value is always 0.

content_index integer

Currently, the value is always 0.

text string

The confirmed translation text segment.

stash string

Temporary translation text, concatenated with text to form interim results. The system updates text and stash continuously via response.audio_transcript.text events until a response.audio_transcript.done event is received. Then retrieve the final translation from transcript.

response.audio_transcript.done

Returned when audio output is enabled and text generation finishes.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_VN4Q4GJugLcc1S23viW8E",
    "type": "response.audio_transcript.done",
    "response_id": "resp_P79OOMs8LnrXVpiIHUCKR",
    "item_id": "item_JvJauNH2CTXb1D9WV6pD4",
    "output_index": 0,
    "content_index": 0,
    "transcript": "How can I assist you today?"
}

type string

The event type. This is always response.audio_transcript.done.

response_id string

The unique identifier for the response.

item_id string

The unique identifier for the message item.

output_index integer

This is currently always 0.

content_index integer

This is currently always 0.

transcript string

The complete text.

response.output_item.added

Returned when a new output item is added during response generation.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_B4O5yPt3Gjnjy5eYH3plG",
    "type": "response.output_item.added",
    "response_id": "resp_P79OOMs8LnrXVpiIHUCKR",
    "output_index": 0,
    "item": {
        "id": "item_OFaPGtzfWCPyGzxnuEX9i",
        "object": "realtime.item",
        "type": "message",
        "status": "in_progress",
        "role": "assistant",
        "content": []
    }
}

type string

The event type. The value is always response.output_item.added.

response_id string

The unique identifier for the response.

output_index integer

The value is currently always 0.

item object

Information about the output item.

Properties

id string

The unique identifier for the output item.

type string

The value is always message.

object string

The value is always realtime.item.

status string

The status of the output item.

role string

The role of the message.

content string

The content of the message.

response.output_item.done

Returned when an output item is completed.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_XkiwbYTBC9Wcdwy6uYJ2G",
    "type": "response.output_item.done",
    "response_id": "resp_P79OOMs8LnrXVpiIHUCKR",
    "output_index": 0,
    "item": {
        "id": "item_JvJauNH2CTXb1D9WV6pD4",
        "object": "realtime.item",
        "type": "message",
        "status": "completed",
        "role": "assistant",
        "content": [
            {
                "type": "audio",
                "text": "Hello, I am a large language model developed by Alibaba Cloud. My name is Qwen. How can I help you?"
            }
        ]
    }
}

type string

The event type. The value is always response.output_item.done.

response_id string

The unique identifier for the response.

output_index integer

The value is currently always 0.

item object

Information about the output item.

Properties

id string

The unique identifier for the output item.

object string

The value is always realtime.item.

type string

The value is always message.

status string

The status of the output item.

role string

The role of the message sender.

content string

The content of the message.

response.content_part.added

Returned when a new content part is added.

event_id string

The unique ID of the event.

 
{
    "event_id": "event_J2UixwYKZsXg7c9YXZetL",
    "type": "response.content_part.added",
    "response_id": "resp_P79OOMs8LnrXVpiIHUCKR",
    "item_id": "item_OFaPGtzfWCPyGzxnuEX9i",
    "output_index": 0,
    "content_index": 0,
    "part": {
        "type": "audio",
        "text": ""
    }
}

type string

The type of the event. The value is always response.content_part.added.

response_id string

The unique ID of the response.

item_id string

The unique ID of the message item.

output_index integer

The value is always 0.

content_index integer

The value is always 0.

part object

Outputs item information.

Properties

type string

The type of the content part.

text string

The text of the content part.

response.content_part.done

Returned when a content part is completed.

event_id string

The unique identifier for this event.

 
{
    "event_id": "event_VN4Q4GJugLcc1S23viW8E",
    "type": "response.content_part.done",
    "response_id": "resp_P79OOMs8LnrXVpiIHUCKR",
    "item_id": "item_JvJauNH2CTXb1D9WV6pD4",
    "output_index": 0,
    "content_index": 0,
    "part": {
        "type": "audio",
        "text": "Hello, I am a large language model developed by Alibaba Cloud. My name is Qwen. How can I help you?"
    }
}

type string

The event type. This is always response.content_part.done.

response_id string

The unique identifier for the response.

item_id string

The unique identifier for the message item.

output_index integer

The value is always 0.

content_index integer

The value is always 0.

part object

Information about the content part.

Properties

type string

The type of the content part.

text string

The text of the content part.
Previous：Client eventsNext：Python SDK