WebSocket interaction flow

Learn about WebSocket endpoints, interaction flows, and SDKs for Qwen-Omni-Realtime models.

Connection endpoints

Supported models: qwen3.8-omni-flash-realtime, qwen3.5-omni-plus-realtime, and qwen3.5-omni-flash-realtime.

China (Beijing)

wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/realtime?model=qwen3.5-omni-plus-realtime

Singapore

wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/realtime?model=qwen3.5-omni-plus-realtime

Replace {WorkspaceId} with the workspace ID for the region, and replace model with the model name to call. For authentication and preparation before connecting, see WebSocket access.

Connection workflow

VAD mode

To enable VAD mode, set the session.turn_detection.type parameter of the client event to "server_vad". In this mode, the server performs Voice Activity Detection (VAD) on incoming audio and responds when it detects speech. This mode is the default and is designed for scenarios where the client continuously sends audio to the server.

• The server sends the input_audio_buffer.speech_started event when it detects the start of speech.
• The client can append audio to the buffer at any time by sending an input_audio_buffer.append event.
• The server sends the input_audio_buffer.speech_stopped event when it detects the end of speech.
• The server commits the input audio buffer by sending an input_audio_buffer.committed event.
• The server sends a conversation.item.created event, which contains the user message item created from the audio buffer.

Function Calling flow

In VAD mode, when a response requires the client to execute a Function Calling tool, use this flow:

• After detecting the end of speech and generating a response, the server determines that a tool call is required.
• The server sends a response.function_call_arguments.delta event, which contains incremental data for the tool call arguments.
• The server sends a response.function_call_arguments.done event to indicate that all tool call arguments have been sent.
• The client executes the tool call and obtains the result.
• The client sends a conversation.item.create event that contains the tool call result.
• The client sends a response.create event to trigger a response based on the tool call result.

Manual mode

To enable Manual mode, set the session.turn_detection parameter of the client event to null. In this mode, the client requests a server response by explicitly sending input_audio_buffer.commit and response.create events. This mode is suitable for push-to-talk scenarios, such as sending voice messages in chat applications.

• The client can append audio to the buffer by sending an input_audio_buffer.append event.
• The client commits the input audio buffer by sending an input_audio_buffer.commit event. This commit creates a new user message item in the conversation.
• The server responds by sending an input_audio_buffer.committed event.
• The client sends a response.create event to trigger the model to generate the final response.
• The server responds by sending a conversation.item.created event.

Function Calling flow

In Manual mode, when a response requires the client to execute a Function Calling tool, use this flow:

• After the client sends a response.create event, the server generates a response and detects that a tool call is required.
• The server sends a response.function_call_arguments.delta event, which contains incremental data for the tool call arguments.
• The server sends a response.function_call_arguments.done event to indicate that the transfer of tool call arguments is complete.
• The client executes the tool call and obtains the result.
• The client sends the tool call result using a conversation.item.create event.
• The client sends a response.create event to trigger the model to generate the final response.
• The server generates a response based on the tool call result and sends it to the client using response.audio.delta or response.text.delta events.

MCP tool calls

The following flow applies to Qwen3.8-Omni-Flash-Realtime and can be used together with the VAD or Manual modes described above.

Function Calling and MCP can be configured together in session.tools for the same session. They cannot be used together with web search (enable_search). Model Studio does not charge an additional MCP tool invocation fee. Model inference remains subject to model pricing.

MCP tools are executed by the server, whereas the Function Calling flows above are executed by the client, so the two differ in how their results are returned.

Tool discovery

After the client configures MCP servers with session.update, the Realtime service discovers their tools asynchronously. The interaction flow is as follows:

In the sequence diagrams in this section, opt indicates execution when a condition is met, alt indicates alternative branches selected by condition, and par indicates parallel processing.

• session.updated only indicates that the MCP configuration has been accepted, not that the tools are already available.
• mcp_list_tools.in_progress may arrive earlier than session.updated.
• The client can continue to send audio or response.create before tool discovery completes, but only Responses created after discovery is complete can use the corresponding MCP tools.
• When tool discovery succeeds, the server first sends a conversation.item.created event containing the final tool list, and then sends mcp_list_tools.completed.
• When tool discovery fails, the server first sends a conversation.item.created event containing an error, and then sends mcp_list_tools.failed.

MCP calls and continuation

When the model selects an MCP tool, the Realtime service executes the tool call. When approval is required, the client first replies with the approval result. The interaction flow is as follows:

Approval rejection or timeout directly triggers response.mcp_call.failed without executing the tool and without going through response.mcp_call.in_progress. When no approval is required or approval is granted, the typical event sequence is as follows:

response.output_item.added (mcp_call)
→ response.mcp_call_arguments.delta
→ Optional: conversation.item.created (mcp_approval_request)
→ Optional: conversation.item.create (mcp_approval_response)
→ response.mcp_call.in_progress
→ response.mcp_call.completed or response.mcp_call.failed
→ response.output_item.done (final mcp_call)
→ response.done (parent Response)

The server sends response.done after all MCP items of the same parent Response reach a terminal state. Its response.output already contains the final mcp_call snapshots: successful items are completed and contain output; failed items are failed and contain error.

If you want the model to continue generating an answer based on the MCP results, the client should send one response.create without MCP results attached after receiving the parent response.done.

See client MCP configuration, approval replies, MCP server events, and MCP conversation items.

Usage limits

The following default MCP constraints apply to each session:

Item	Default value
MCP servers	8
MCP tools	128
Tool discovery timeout	30 seconds
Approval timeout	60 seconds
Tool execution timeout	30 seconds
Single upstream response size	2 MB
Cumulative tool result size	8 MB
Cumulative tool calls	256

SDK

• Python SDK
• Java SDK