typestring(required) The event type. The value must be session.update. sessionobject(optional) The session configuration. For Qwen3.8-Omni-Flash-Realtime, session is required when updating a session. The total input token limit is 196608, calculated using the same rules as Qwen3.5-Omni-Realtime. Properties modalitiesarray(optional) The model output modalities. Valid values: voicestring(optional) The voice for the model's audio output. For supported voices, see Voice list. Default voices:
- Qwen3.8-Omni-Flash-Realtime:
Tina
- Qwen3.5-Omni-Realtime series model:
Tina
- qwen3-omni-flash-realtime-2025-09-15:
Cherry
audioobject (optional) Input and output audio configuration. Omitted fields retain their existing defaults. Properties inputobject (optional) User input audio configuration. Properties formatobject (optional) Input format and sample rate. Configure these while the session is IDLE, before sending the first audio segment. They cannot change after audio input starts. Properties typestring (optional) For qwen3.5-omni-plus-realtime and qwen3.5-omni-flash-realtime: pcm (default, mono 16-bit raw PCM) or wav (mono 16-bit PCM in a WAV container). Multichannel input for Qwen3.8-Omni-Flash-Realtime requires pcm, which is the default. sample_rateinteger (optional) For qwen3.5-omni-plus-realtime and qwen3.5-omni-flash-realtime: 8000, 16000 (default), 24000, or 48000 Hz. Multichannel input for Qwen3.8-Omni-Flash-Realtime requires 16000 Hz, which is the default. The following four fields apply to Qwen3.8-Omni-Flash-Realtime multichannel input over WebSocket: sample_formatstring (optional) PCM sample format. Only s16le is supported; default: s16le. channelsinteger (optional) Input channel count: 1 (default), 2, or 4. Two-channel and four-channel spatial audio use twice the input tokens of ordinary audio. See Token calculation. packingstring (optional) Sample packing. Only interleaved is supported; default: interleaved. channel_layoutstring (optional) Default depends on channels: mono for one channel, raw_mic_array for two, and foa_ambix for four. An explicit value must match channels. Multichannel input requires PCM, 16000 Hz, s16le, and interleaved samples. Omitted fields use their defaults; supplied fields must have valid types and values. Do not send null instead of omitting a field. outputobject (optional) Model output audio configuration. Properties formatobject (optional) Output format and sample rate. Configure these at the start of the session, before audio interaction begins. Properties typestring (optional) For qwen3.5-omni-plus-realtime and qwen3.5-omni-flash-realtime: pcm (default, mono 16-bit raw PCM) or wav (mono 16-bit PCM in a WAV container). sample_rateinteger (optional) For qwen3.5-omni-plus-realtime and qwen3.5-omni-flash-realtime: 8000, 16000, 24000 (default), or 48000 Hz. voicestring (optional) For Qwen3.8-Omni-Flash-Realtime, the output voice defaults to Tina. See the voice list for available voices and their voice values. session.voice remains available for existing integrations. For new integrations, use session.audio.output.voice. If both fields are set, the audio output uses the voice specified by session.audio.output.voice. input_audio_transcriptionobject | null (optional) Input audio transcription configuration. Set to null to disable it. Properties modelstring (required) The value is fixed to qwen3-asr-flash-realtime. Set {"model":"qwen3-asr-flash-realtime"} to enable transcription; the model value cannot be changed. The Python SDK's enable_input_audio_transcription is an update_session method argument, not a top-level session.update field. input_audio_formatstring(optional) A legacy compatibility field. For new integrations, use audio.input.format to configure both the input format and sample rate. output_audio_formatstring(optional) A legacy compatibility field. For new integrations, use audio.output.format to configure both the output format and sample rate. smooth_outputboolean | null(optional) Applies only to qwen3-omni-flash-realtime-2025-09-15.Controls the response style. Valid values:
-
true (default): Colloquial responses.
-
false: Formal, written responses.
Content that is difficult to read aloud may not perform well.
-
null: The model automatically selects between colloquial and formal style.
instructionsstring(optional) System message that sets the model's role or objective. A new instructions value fully replaces the previous instructions value instead of appending to it. To keep the role, send the existing role instructions together with the new requirements. turn_detectionobject(optional) Voice activity detection (VAD) configuration. When you first configure the session, omitting this field enables VAD with the default parameters. Set it to null to disable VAD and trigger model responses manually from the client. Properties typestring(optional) The VAD type. Valid values:
server_vad (default): Detects end of speech based on acoustic features.semantic_vad: Detects end of speech based on semantic validity, filtering out backchannels and background noise. Supported by Qwen3.8-Omni-Flash-Realtime and the Qwen3.5-Omni-Realtime series.semantic_vad_ec: Semantic validity detection for customer service and business scenarios. Compared with semantic_vad, this mode detects user speech more readily to interrupt the model response, while still filtering out meaningless sounds such as background noise. Supported by the Qwen3.8-Omni-Flash-Realtime series. thresholdfloat(optional) The VAD sensitivity. A lower value makes VAD more sensitive—more likely to detect faint sounds as speech, including background noise. A higher value requires clearer and louder speech to trigger. Valid range: [-1.0, 1.0]. Default: 0.5. silence_duration_msinteger(optional) The minimum silence duration after speech ends, in milliseconds. When exceeded, the model generates a response. A lower value speeds up responses but may cause false triggers during brief pauses. Default: 800. Valid range: [200, 6000]. idle_timeout_msinteger(optional) Applies only to qwen3.5-omni-plus-realtime and qwen3.5-omni-flash-realtime models in server_vad mode.The idle timeout in milliseconds. After the server finishes audio playback and the user remains silent beyond this duration (no speech.started triggered), the model proactively generates a response to prompt the user to continue the conversation based on the current context. The timeout starts after the last model response audio finishes playing. Valid range: [5000, 30000]. enable_searchboolean(optional) Applies to Qwen3.8-Omni-Flash-Realtime and Qwen3.5-Omni-Realtime models. Whether to enable web search. Set to true to enable; default is false. When enabled, the model decides autonomously whether it needs to search the web to answer the user's questions.
tools and enable_search are incompatible. Do not enable both at the same time.
search_optionsobject(optional) Web search configuration. Takes effect only when enable_search is enabled. Properties enable_sourceboolean(optional) Whether to return the list of search result sources. Set to true to enable. videoobject (optional) Video input configuration for Qwen3.8-Omni-Flash-Realtime. Properties inputobject (optional) Video input parameters. Properties representation_compactstring (optional) Video representation aggregation. The initial session default is none, which preserves full fine-grained input. normal aggregates it to reduce computation when fine visual detail is unnecessary; for the same video input, it uses one quarter of the tokens used by none. See Token calculation. Set this before the first audio segment; it cannot change after audio input starts. toolsarray(optional) A list of tool definitions. The model decides whether to call a tool based on user input. For Qwen3.8-Omni-Flash-Realtime, Function Calling and MCP tools can be configured in the same session. Array elements are distinguished by type; tools and enable_search cannot both be enabled. Function Calling (type=function) typestring (required) Must be function. functionobject (required) The custom function definition. Properties namestring (required) The function name, matching the actual function name, such as get_current_weather or get_current_time. descriptionstring (optional) A description of what the function does. The model uses it to decide whether to call the tool. parametersobject (optional) The function's input parameters. The model uses this to extract parameter values. Omit this field if the function has no input. Properties typestring (required) Must be object. propertiesobject (optional) Defines each input parameter by name. In the example on the right, location is a custom parameter name; its type and description describe the data type and purpose. requiredarray (optional) Lists which parameters are required. MCP (type=mcp) For the MCP configuration and invocation flow of Qwen3.8-Omni-Flash-Realtime, see Interaction flow. Required fields must be supplied when this MCP array element is present. Every addition or update requires both server_label and server_url; a label alone cannot reuse an existing configuration. Update MCP configuration only when there is no active Response. Connections, discovery, and calls are subject to quotas and timeouts. See MCP call limits. typestring (required) Must be mcp. server_labelstring (required) Unique MCP server label within the session. Length: 1–64 characters; letters, digits, underscores, and hyphens only. server_urlstring (required) MCP Streamable HTTP endpoint. Must be a public HTTPS address on port 443, up to 4096 characters, with no username, password, or fragment. Its domain must resolve to public addresses. authorizationstring (optional) Outbound Authorization value, up to 8192 printable ASCII characters. headersobject (optional) Additional outbound HTTP headers, up to 16 string-to-string pairs. Names must be 1–128 characters using standard HTTP header-name characters. Values must be printable ASCII and at most 8192 characters. Names cannot start with mcp-, proxy-, or x-forwarded-, or equal host, authorization, connection, content-length, transfer-encoding, accept, content-type, forwarded, cookie, origin, upgrade, te, or trailer. allowed_toolsarray of strings (optional) Filter applied after discovery. Omission allows all tools; an empty array exposes none. Each name is 1–64 characters, with letters, digits, underscores, periods, and hyphens only. require_approvalstring (optional) Approval policy: always (default) or never. server_url, authorization, and headers are used only for the server-side connection and are not echoed in session.updated. Do not reconstruct sensitive settings from that event.
temperaturefloat(optional) The sampling temperature, controlling output diversity. A higher value produces more varied content; a lower value produces more deterministic output. Valid range: [0, 2). Because temperature and top_p both control diversity, set only one. Default values:
qwen3.8-omni-flash-realtime: 0.6- Qwen3.5-Omni-Realtime series model: 0.7
qwen3-omni-flash-realtime-2025-09-15: 0.9 top_pfloat(optional) The nucleus sampling probability threshold, controlling output diversity. A higher value produces more varied content; a lower value produces more deterministic output. Valid range: (0, 1.0]. Because temperature and top_p both control diversity, set only one. Default values:
qwen3.8-omni-flash-realtime: 0.95- Qwen3.5-Omni-Realtime series model: 0.8
qwen3-omni-flash-realtime-2025-09-15: 1.0 top_kinteger(optional) The candidate set size for sampling. For example, a value of 50 limits each generation step to the 50 highest-scoring tokens. A larger value increases randomness; a smaller value increases determinism. Set to null or a value greater than 100 to disable top_k, in which case only top_p applies. Minimum value: 0. Default values:
qwen3.8-omni-flash-realtime: 20- Qwen3.5-Omni-Realtime series model: 20
qwen3-omni-flash-realtime-2025-09-15: 50 max_tokensinteger(optional) The maximum number of tokens to return.
The max_tokens setting does not affect the model's generation process. If the generated output exceeds max_tokens, the response is truncated.
For qwen3.8-omni-flash-realtime, the valid range is [1, 65536]. For other models, the default and maximum values equal the model's maximum output length. See Model list. Use max_tokens to limit output length for tasks like summarization, keyword extraction, cost control, or latency reduction. repetition_penaltyfloat(optional) Controls repetition in the model's output. A higher value reduces repetition; 1.0 means no penalty. qwen3.8-omni-flash-realtime also accepts 0. For other models, the value must be greater than 0, with no strict upper limit. Default values:
qwen3.8-omni-flash-realtime: 0- Qwen3.5-Omni-Realtime series model: 1.0
qwen3-omni-flash-realtime-2025-09-15: 1.05 presence_penalty float(optional) Controls content repetition in the model's output. Valid range: [-2.0, 2.0]. A positive value reduces repetition; a negative value increases it. Default values:
qwen3.8-omni-flash-realtime: 1- Qwen3.5-Omni-Realtime series model: 1.5
qwen3-omni-flash-realtime-2025-09-15: 0.0 When to use: Higher values suit creative tasks like storytelling or brainstorming, where output diversity and interest matter. Lower values suit technical or formal content where consistency and precise terminology are required. seedinteger(optional) Makes generation more deterministic. Pass the same seed value with unchanged parameters to get consistent results across runs. Valid range: 0 to 231−1. Default: -1. | {
"event_id": "event_ToPZqeobitzUJnt3QqtWg",
"type": "session.update",
"session": {
"modalities": [
"text",
"audio"
],
"model": "qwen3.5-omni-flash-realtime",
"voice": "Tina",
"audio": {
"input": {
"format": {
"type": "pcm",
"sample_rate": 16000
}
},
"output": {
"format": {
"type": "wav",
"sample_rate": 24000
}
}
},
"instructions": "You are an AI customer service specialist for a five-star hotel. Accurately and friendly answer customer inquiries about room types, facilities, prices, and booking policies. Always respond with a professional and helpful attitude. Do not provide unverified information or information outside the scope of hotel services.",
"turn_detection": {
"type": "server_vad",
"threshold": 0.5,
"silence_duration_ms": 800
},
"enable_search": false,
"tools": [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Useful for querying the weather in a specific city.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city or district, such as Beijing, Hangzhou, or Yuhang District."
}
},
"required": ["location"]
}
}
}
],
"seed": 1314,
"max_tokens": 16384,
"repetition_penalty": 1.05,
"presence_penalty": 0.0,
"top_k": 50,
"top_p": 1.0,
"temperature": 0.9
}
}
|