Qwen-Audio-ASR-Message client events

Updated at:

Client event reference for Qwen-Audio-ASR-Message: run-task, continue-task, and finish-task fields, event order, and corresponding server events.

Event flow

  1. Send run-task. Wait for task-started before sending audio.
  2. Continue sending audio while receiving result-generated events.
  3. During the task, use continue-task to update the conversation context. Continue sending audio through the audio channel.
  4. After sending all audio, send finish-task and receive results until task-finished arrives. A failed task returns task-failed.

Audio is not sent in run-task: WebSocket uses binary frames, and AOQ uses an Audio Track. For connection setup, authentication, and audio transport, see WebSocket API or AOQ access.

run-task

After establishing the connection, the client sends a run-task event to start a recognition task and configure the model, audio format, sample rate, and other parameters.

ImportantWait for task-started before sending audio. run-task only starts and configures the task; it does not carry audio data.

headerobjectRequired
actionstringRequired

The command type. Set to run-task.

task_idstringRequired

A client-generated task ID (UUID format) that links subsequent events to this task.

streamingstringRequired

Set to duplex.

payloadobjectRequired
task_groupstringRequired

The task group. Set to audio.

taskstringRequired

The task type. Set to asr.

functionstringRequired

The function type. Set to recognition.

modelstringRequired

The model name.

parametersobjectRequired

The speech recognition parameters.

formatstringRequired

The audio format.

Valid values:

  • pcm
  • wav
  • mp3
  • opus
  • speex
  • aac
  • amr

Note

  • opus/speex: Must use Ogg encapsulation.
  • wav: Must use PCM encoding.
  • amr: Only the AMR-NB type is supported.
sample_rateintegerRequired

The sample rate, in Hz.

Only 16000 Hz is supported.

disfluency_removal_enabledbooleanOptional

Whether to filter filler words and polish the output. Defaults to false. Set to true to enable this feature.

intermediate_result_enabledbooleanOptional

Whether to return intermediate streaming results. Defaults to false. Set to true to return intermediate streaming results.

keep_dialectbooleanOptional

Specifies whether to preserve dialect expressions. Default: false. Set to false to transcribe dialect speech as standard Mandarin text, or true to preserve dialect expressions.

NoteThis parameter applies only to final results (sentence_end=true) and is not supported for intermediate results (sentence_end=false).

vad_modelstringOptional

The VAD model. Default: far_field_meeting_16k.

  • near_meeting_16k: For audio captured close to the speaker, such as customer service calls using a headset or voice input in an app on a handheld phone.
  • far_field_meeting_16k: For audio captured at a distance from the speaker, such as smart speaker interactions, shared microphones in meeting rooms, and in-vehicle hands-free voice interactions.
max_sentence_silenceintegerOptional

The VAD silence threshold for segmentation, in ms. When the silence after a segment of speech exceeds this threshold, the system determines that the sentence has ended.

Default value: 800.

Valid values: [200, 6000].

speech_noise_thresholdfloatOptional

The threshold for distinguishing speech from noise, used to adjust the sensitivity of Voice Activity Detection (VAD).

Valid values: [-1.0, 1.0].

Value descriptions:

  • The closer the value is to -1: The noise threshold decreases, so noise is more likely to be recognized as speech, which may cause more noise to be transcribed.
  • The closer the value is to +1: The noise threshold increases, so speech is more likely to be misjudged as noise, which may cause some speech to be filtered out.

This is an advanced configuration parameter. Adjusting it can significantly affect recognition results. Recommendations:

  • Thoroughly test and verify the results before adjusting.
  • Adjust in small increments based on the actual audio environment (a step of 0.1 is recommended).
vocabulary_idstringOptional

The ID of a precompiled hot word list.

Generate this ID in advance by calling the create hot word list API. Pass the ID during recognition to use the hot words in the list.

Suitable for scenarios where the vocabulary is known and relatively stable, and where you need to reuse the same word list across requests.

For usage details, see Precompiled hotwords.

vocabularyobjectOptional

Instant hot words.

Passed as key-value pairs, where the key is the hot word text (string) and the value is the hot word weight (integer). No hot word list needs to be created in advance. The weight ranges from [1, 5] or is set to 50: a value in [1, 5] makes the model more likely to output the word as the value increases; a value of 50 designates a super hot word, which greatly improves recall, but the number of super hot words cannot exceed 50.

Suitable for temporary, session-level hot word optimization.

NoteWhen instant and precompiled hotwords are configured together, the system merges both sets. If the merged set contains more than 2000 hotwords, the system randomly selects 2000 to use. For usage details, see Instant hotwords.

language_hintsarray[string]Optional

The language of the audio to recognize. There is no default value; if not set, the model detects the language automatically.

You can set up to 4 values. If you set more, only the first 4 take effect.

Click to view the supported language codes

  • zh: Chinese
  • en: English
  • ja: Japanese
  • ko: Korean
  • vi: Vietnamese
  • th: Thai
  • id: Indonesian
  • ms: Malay
  • tl: Filipino
  • hi: Hindi
  • ar: Arabic
  • fr: French
  • de: German
  • es: Spanish
  • pt: Portuguese
  • ru: Russian
  • it: Italian
  • nl: Dutch
  • sv: Swedish
  • da: Danish
  • fi: Finnish
  • no: Norwegian
  • el: Greek
  • pl: Polish
  • cs: Czech
  • hu: Hungarian
  • ro: Romanian
  • bg: Bulgarian
  • hr: Croatian
  • sk: Slovak

NoteThis parameter applies only to final results (sentence_end=true) and is not supported for intermediate results (sentence_end=false).

heartbeatbooleanOptional

Whether to enable heartbeat packets.

Default value: false.

  • true: Keeps the connection to the server alive even when silent audio is sent continuously.
  • false (default): Even when silent audio is continuously sent, the connection times out and closes after a period of time.

Silent audio refers to content in an audio file or data stream that contains no sound signal. You can generate silent audio in several ways, such as using audio editing software like Audacity or Adobe Audition, or using a command-line tool like FFmpeg.

inputobjectRequired

The input object. Pass {} when no context is provided.

contextarray[object]Optional

The conversation context, which improves recognition accuracy for domain-specific vocabulary. For usage details, see Context enhancement.

Note

  • Message count: Provide at most 5 context messages of each type (input_text and text). If this limit is exceeded, only the 5 most recent messages are kept.
  • Text length: The combined length of the user and assistant text fields in each turn must not exceed 400 characters. Each character counts as 1. Excess text is truncated from the end.
  • Message order: Arrange messages by conversation turn. Within each turn, the user message (input_text) must precede its corresponding assistant message (text).
rolestringRequired

The message role. Valid values:

  • user: The recognition results from previous turns of user speech, or a domain-specific word list.
  • assistant: The responses from the large language model in previous turns.
contentarray[object]Required

The list of message contents.

typestringRequired

The content type. Valid values:

  • input_text: The recognition results from previous turns of user speech, or a domain-specific word list (used when role is user). You must also provide the text field.
  • text: The responses from the large language model in previous turns (used when role is assistant). You must also provide the text field.
textstringRequired

The text content. When type is input_text, enter the recognition results from previous turns of user speech or a domain-specific word list. When type is text, enter the responses from the large language model in previous turns.

Basic request

{
  "header": {
    "action": "run-task",
    "task_id": "ab4f7812-2d71-4e3d-93d3-b68ab80d3755",
    "streaming": "duplex"
  },
  "payload": {
    "input": {},
    "task_group": "audio",
    "task": "asr",
    "function": "recognition",
    "model": "qwen-audio-3.1-asr-flash-message",
    "parameters": {
      "format": "pcm",
      "sample_rate": 16000
    }
  }
}
{
  "header": {
    "task_id": "ab4f7812-2d71-4e3d-93d3-b68ab80d3755",
    "task_uuid": "852196dba9d14bea9cf0d6e04ff0b027",
    "event": "task-started",
    "attributes": {}
  },
  "payload": {}
}

With context

{
  "header": {
    "action": "run-task",
    "task_id": "b5930843-7a81-400d-b639-c94a38e83e72",
    "streaming": "duplex"
  },
  "payload": {
    "input": {
      "context": [
        {
          "role": "user",
          "content": [
            {
              "type": "input_text",
              "text": "We were discussing the quarterly report."
            }
          ]
        },
        {
          "role": "assistant",
          "content": [
            {
              "type": "text",
              "text": "Please send the quarterly report to the project lead."
            }
          ]
        }
      ]
    },
    "task_group": "audio",
    "task": "asr",
    "function": "recognition",
    "model": "qwen-audio-3.1-asr-flash-message",
    "parameters": {
      "format": "pcm",
      "sample_rate": 16000
    }
  }
}
{
  "header": {
    "task_id": "b5930843-7a81-400d-b639-c94a38e83e72",
    "task_uuid": "7f6dd3db3375475888ee4f769bf0ef53",
    "event": "task-started",
    "attributes": {}
  },
  "payload": {}
}

Instant hotwords

{
  "header": {
    "action": "run-task",
    "task_id": "be6a9cb0-9bcb-4581-a0a0-0f6151787444",
    "streaming": "duplex"
  },
  "payload": {
    "input": {},
    "task_group": "audio",
    "task": "asr",
    "function": "recognition",
    "model": "qwen-audio-3.1-asr-flash-message",
    "parameters": {
      "format": "pcm",
      "sample_rate": 16000,
      "vocabulary": {
        "quarterly report": 5,
        "project": 5
      }
    }
  }
}
{
  "header": {
    "task_id": "be6a9cb0-9bcb-4581-a0a0-0f6151787444",
    "task_uuid": "3f735378a14548f88e92d32b3a327e9f",
    "event": "task-started",
    "attributes": {}
  },
  "payload": {}
}

continue-task

After receiving task-started, the client can send continue-task during the task to update the conversation context for subsequent recognition.

headerobjectRequired
actionstringRequired

The command type. Set to continue-task.

task_idstringRequired

A client-generated task ID (UUID format) that must match the task_id in the run-task event.

streamingstringRequired

Set to duplex.

payloadobjectRequired
inputobjectRequired
contextarray[object]Optional

The conversation context, which improves recognition accuracy for domain-specific vocabulary. For usage details, see Context enhancement.

Note

  • Message count: Provide at most 5 context messages of each type (input_text and text). If this limit is exceeded, only the 5 most recent messages are kept.
  • Text length: The combined length of the user and assistant text fields in each turn must not exceed 400 characters. Each character counts as 1. Excess text is truncated from the end.
  • Message order: Arrange messages by conversation turn. Within each turn, the user message (input_text) must precede its corresponding assistant message (text).
rolestringRequired

The message role. Valid values:

  • user: The recognition results from previous turns of user speech, or a domain-specific word list.
  • assistant: The responses from the large language model in previous turns.
contentarray[object]Required

The list of message contents.

typestringRequired

The content type. Valid values:

  • input_text: The recognition results from previous turns of user speech, or a domain-specific word list (used when role is user). You must also provide the text field.
  • text: The responses from the large language model in previous turns (used when role is assistant). You must also provide the text field.
textstringRequired

The text content. When type is input_text, enter the recognition results from previous turns of user speech or a domain-specific word list. When type is text, enter the responses from the large language model in previous turns.

This example updates the context and then continues sending audio. The response example is a recognition result received afterward.

{
  "header": {
    "action": "continue-task",
    "task_id": "bdebb544-9411-413b-a199-aa43485716a0",
    "streaming": "duplex"
  },
  "payload": {
    "input": {
      "context": [
        {
          "role": "user",
          "content": [
            {
              "type": "input_text",
              "text": "We were discussing the quarterly report."
            }
          ]
        },
        {
          "role": "assistant",
          "content": [
            {
              "type": "text",
              "text": "Please send the quarterly report to the project lead."
            }
          ]
        }
      ]
    }
  }
}
{
  "header": {
    "task_id": "bdebb544-9411-413b-a199-aa43485716a0",
    "event": "result-generated",
    "attributes": {}
  },
  "payload": {
    "output": {
      "sentence": {
        "sentence_id": 1,
        "begin_time": 120,
        "end_time": 4040,
        "text": "Please send the quarterly project report to Kellera before Friday. ",
        "sentence_end": true,
        "words": [
          {
            "begin_time": 120,
            "end_time": 320,
            "text": "Please",
            "punctuation": "",
            "fixed": false
          },
          {
            "begin_time": 320,
            "end_time": 560,
            "text": " send",
            "punctuation": "",
            "fixed": false
          },
          {
            "begin_time": 560,
            "end_time": 760,
            "text": " the",
            "punctuation": "",
            "fixed": false
          },
          {
            "begin_time": 760,
            "end_time": 1360,
            "text": " quarterly",
            "punctuation": "",
            "fixed": false
          },
          {
            "begin_time": 1360,
            "end_time": 1800,
            "text": " project",
            "punctuation": "",
            "fixed": false
          },
          {
            "begin_time": 1800,
            "end_time": 2120,
            "text": " report",
            "punctuation": "",
            "fixed": false
          },
          {
            "begin_time": 2120,
            "end_time": 2280,
            "text": " to",
            "punctuation": "",
            "fixed": false
          },
          {
            "begin_time": 2280,
            "end_time": 2336,
            "text": " ",
            "punctuation": "",
            "fixed": false
          },
          {
            "begin_time": 2336,
            "end_time": 2560,
            "text": "Kell",
            "punctuation": "",
            "fixed": false
          },
          {
            "begin_time": 2560,
            "end_time": 2800,
            "text": "era",
            "punctuation": "",
            "fixed": false
          },
          {
            "begin_time": 2800,
            "end_time": 3160,
            "text": " before",
            "punctuation": "",
            "fixed": false
          },
          {
            "begin_time": 3160,
            "end_time": 4040,
            "text": " Friday",
            "punctuation": ". ",
            "fixed": false
          }
        ],
        "stash": {
          "sentence_id": 2,
          "text": "",
          "begin_time": 4040,
          "current_time": 4040,
          "words": []
        }
      }
    },
    "usage": {
      "duration": 4,
      "input_tokens": 73,
      "output_tokens": 13,
      "total_tokens": 86
    }
  }
}

finish-task

After sending all audio data, the client sends a finish-task event to notify the server that audio input is complete and request task completion.

ImportantAfter sending finish-task, continue receiving remaining results until task-finished arrives. Use the same task_id as in run-task.

headerobjectRequired
actionstringRequired

The command type. Set to finish-task.

task_idstringRequired

A client-generated task ID (UUID format) that must match the task_id in the run-task event.

streamingstringRequired

Set to duplex.

payloadobjectRequired
inputobjectRequired

Set to {}.

{
  "header": {
    "action": "finish-task",
    "task_id": "ab4f7812-2d71-4e3d-93d3-b68ab80d3755",
    "streaming": "duplex"
  },
  "payload": {
    "input": {}
  }
}
{
  "header": {
    "task_id": "ab4f7812-2d71-4e3d-93d3-b68ab80d3755",
    "event": "task-finished",
    "attributes": {}
  },
  "payload": {
    "output": {},
    "usage": {
      "duration": 4,
      "input_tokens": 51,
      "output_tokens": 13,
      "total_tokens": 64
    }
  }
}