WebSocket connection guide
Qwen-ASR-Realtime receives audio streams and transcribes speech in real time over WebSocket. The service supports two interaction modes: VAD mode and Manual mode .
User guide: For model overviews and selection guidance, see Speech-to-text. For sample code, see Real-time speech recognition.
Service endpoint
Use the following WebSocket URL. The model query parameter specifies the model. Replace <model_name> with the model name:
China (Beijing)
wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/realtime?model=<model_name>
Replace {WorkspaceId} with your actual workspace ID.
Singapore
wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/realtime?model=<model_name>
Replace {WorkspaceId} with your actual Workspace ID.
ImportantAlibaba Cloud Model Studio has released workspace-specific domains for the China (Beijing) and Singapore regions. The new dedicated domains deliver superior performance and higher stability for inference requests. We recommend migrating to the new domains:
- China (Beijing): from
dashscope.aliyuncs.comto{WorkspaceId}.cn-beijing.maas.aliyuncs.com - Singapore: from
dashscope-intl.aliyuncs.comto{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com
Replace {WorkspaceId} with your actual Workspace ID. The existing domains remain fully functional.
ImportantUse the wss:// scheme. Set authorization in the request headers (see Request headers). Specify the model through the model query parameter.
Request headers
Include the following fields in the request headers:
Parameter | Type | Required | Description |
|---|---|---|---|
Authorization | string | Yes | Authentication token in the format |
user-agent | string | No | Client identifier that helps the server track request sources. |
X-DashScope-WorkSpace | string | No | The Alibaba Cloud Model Studio workspace ID. |
X-DashScope-DataInspection | string | No | Whether to enable data inspection. By default, this header is omitted or set to |
ImportantAuthorization is verified during the WebSocket handshake. If the API key is invalid or missing, the handshake fails with an HTTP 401 or 403 error.
Interaction flows
For details about client and server events, see Client events for Qwen-ASR-Realtime and Server events.
Qwen-ASR-Realtime supports two interaction modes:
- VAD mode (default): The server uses voice activity detection (VAD) to automatically detect the start and end of each utterance. Use this mode for real-time conversations, meeting transcription, and similar scenarios.
- Manual mode: The client controls utterance boundaries. Use this mode when the client can determine those boundaries explicitly, such as sending a voice message in a messaging app.
This model does not return sentence-level or word-level timestamps for the transcript. Speech start and end times in VAD events describe voice activity boundaries; see those events for the field definitions.
VAD mode (default)
The server default for silence_duration_ms is 800; 400 is a reference setting for conversations that need faster segmentation. The server default for threshold is 0.2. For the full configuration, see Client events.
The server automatically detects the start and end of each utterance. Send the audio stream continuously; the server returns the final transcription for each utterance once it detects the end. Use this mode for real-time conversations, meeting transcription, and similar scenarios.
To enable: Configure the session.turn_detection parameter in the client session.update event.
-
The client sends input_audio_buffer.append events to append audio to the buffer.
-
The server returns a input_audio_buffer.speech_started event when speech is detected.
Note: If the client sends session.finish to end the session before this event arrives, the server immediately returns a session.finished event. The client must then close the connection.
-
The client continues to send input_audio_buffer.append events to submit audio.
-
After sending all audio, the client sends a session.finish event to end the session.
WarningIn VAD mode, the client must send a
session.finishevent before closing the connection. If the client closes the WebSocket connection without first sendingsession.finish, the server discards the current in-progress item and events such asconversation.item.input_audio_transcription.completedwill not be delivered. Send{"type":"session.finish"}before callingws.Close(), and wait for thesession.finishedevent before closing the connection. -
The server returns a input_audio_buffer.speech_stopped event when it detects the end of speech.
-
The server returns a input_audio_buffer.committed event.
-
The server returns a conversation.item.created event.
-
The server returns a conversation.item.input_audio_transcription.text event that contains partial transcription results.
-
The server returns a conversation.item.input_audio_transcription.completed event that contains the final transcription result.
-
The server returns a session.finished event to signal that recognition is complete. The client must then close the connection.
Manual mode
The client controls utterance boundaries. After sending the audio for a complete utterance, the client sends a input_audio_buffer.commit event to notify the server. Use this mode when the client can determine those boundaries explicitly, such as sending a voice message in a messaging app.
To enable: Set session.turn_detection to null in the client session.update event.
- The client sends input_audio_buffer.append events to append audio to the buffer.
- The client sends a input_audio_buffer.commit event to commit the input audio buffer. The commit creates a new user message item in the conversation.
- The client sends a session.finish event to end the session.
- The server returns a input_audio_buffer.committed event.
- The server returns a conversation.item.input_audio_transcription.text event that contains partial transcription results.
- The server returns a conversation.item.input_audio_transcription.completed event that contains the final transcription result.
- The server returns a session.finished event to signal that recognition is complete. The client must then close the connection.
Examples
These examples use qwen3-asr-flash-realtime. Get your API key and Base URL from the console, and set them as DASHSCOPE_API_KEY and DASHSCOPE_BASE_URL.
Download the sample audio as asr_example.wav: a 16 kHz, mono WAV file with 16-bit PCM encoding.
NoteThe example code reads asr_example.pcm (PCM16, 16 kHz, mono). If you only have an MP3, WAV, or similar format, convert it with ffmpeg:
ffmpeg -i asr_example.wav -ar 16000 -ac 1 -f s16le asr_example.pcm
NoteIn non-VAD (Manual) mode, we recommend that the cumulative duration of audio sent in a single session does not exceed 60 seconds.
Java
Before you run the example, install the Java-WebSocket dependency:
<dependency>
<groupId>org.java-websocket</groupId>
<artifactId>Java-WebSocket</artifactId>
<version>1.5.6</version>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20240303</version>
</dependency>
implementation 'org.java-websocket:Java-WebSocket:1.5.6'
implementation 'org.json:json:20240303'
import org.java_websocket.client.WebSocketClient;
import org.java_websocket.handshake.ServerHandshake;
import org.json.JSONObject;
import java.net.URI;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Base64;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.logging.*;
public class QwenASRRealtimeClient {
private static final Logger logger = Logger.getLogger(QwenASRRealtimeClient.class.getName());
private static final String API_KEY = System.getenv("DASHSCOPE_API_KEY");
private static final String MODEL = "qwen3-asr-flash-realtime";
// Controls whether to use VAD mode
private static final boolean enableServerVad = true;
private static final AtomicBoolean isRunning = new AtomicBoolean(true);
private static WebSocketClient client;
public static void main(String[] args) throws Exception {
initLogger();
String baseUrl = "wss://" + URI.create(System.getenv("DASHSCOPE_BASE_URL")).getHost() + "/api-ws/v1/realtime";
String url = baseUrl + "?model=" + MODEL;
logger.info("Connecting to server: " + url);
client = new WebSocketClient(new URI(url)) {
@Override
public void onOpen(ServerHandshake handshake) {
logger.info("Connected to server.");
sendSessionUpdate();
}
@Override
public void onMessage(String message) {
try {
JSONObject data = new JSONObject(message);
String eventType = data.optString("type");
logger.info("Received event: " + data.toString(2));
// The final recognition result is in the transcription.completed event
if ("conversation.item.input_audio_transcription.completed".equals(eventType)) {
logger.info("Final transcript: " + data.optString("transcript"));
}
// On receiving the finished event -> stop the sending thread and close the connection
if ("session.finished".equals(eventType)) {
logger.info("Closing WebSocket connection after session finished...");
isRunning.set(false); // Stop the audio sending thread
if (this.isOpen()) {
this.close(1000, "ASR finished");
}
}
} catch (Exception e) {
logger.severe("Failed to parse message: " + message);
}
}
@Override
public void onClose(int code, String reason, boolean remote) {
logger.info("Connection closed: " + code + " - " + reason);
}
@Override
public void onError(Exception ex) {
logger.severe("Error: " + ex.getMessage());
}
};
// Add request headers
client.addHeader("Authorization", "Bearer " + API_KEY);
client.addHeader("OpenAI-Beta", "realtime=v1");
client.connectBlocking(); // Block until the connection is established
// Replace with the path to the audio file to be recognized
String localAudioPath = "asr_example.pcm";
Thread audioThread = new Thread(() -> {
try {
sendAudio(localAudioPath);
} catch (Exception e) {
logger.severe("Audio sending thread error: " + e.getMessage());
}
});
audioThread.start();
}
/** Session update event (enable/disable VAD) */
private static void sendSessionUpdate() {
JSONObject eventNoVad = new JSONObject()
.put("event_id", "event_123")
.put("type", "session.update")
.put("session", new JSONObject()
.put("modalities", new String[]{"text"})
.put("input_audio_format", "pcm")
.put("sample_rate", 16000)
// .put("input_audio_transcription", new JSONObject()
// .put("language", "zh"))
.put("turn_detection", JSONObject.NULL) // Manual mode
);
JSONObject eventVad = new JSONObject()
.put("event_id", "event_123")
.put("type", "session.update")
.put("session", new JSONObject()
.put("modalities", new String[]{"text"})
.put("input_audio_format", "pcm")
.put("sample_rate", 16000)
// .put("input_audio_transcription", new JSONObject()
// .put("language", "zh"))
.put("turn_detection", new JSONObject()
.put("type", "server_vad")
.put("threshold", 0.2)
.put("silence_duration_ms", 400))
);
if (enableServerVad) {
logger.info("Sending event (VAD):\n" + eventVad.toString(2));
client.send(eventVad.toString());
} else {
logger.info("Sending event (Manual):\n" + eventNoVad.toString(2));
client.send(eventNoVad.toString());
}
}
/** Send the audio file stream */
private static void sendAudio(String localAudioPath) throws Exception {
Thread.sleep(3000); // Wait for the session to be ready
byte[] allBytes = Files.readAllBytes(Paths.get(localAudioPath));
logger.info("File reading started");
int offset = 0;
while (isRunning.get() && offset < allBytes.length) {
int chunkSize = Math.min(3200, allBytes.length - offset);
byte[] chunk = new byte[chunkSize];
System.arraycopy(allBytes, offset, chunk, 0, chunkSize);
offset += chunkSize;
if (client != null && client.isOpen()) {
String encoded = Base64.getEncoder().encodeToString(chunk);
JSONObject eventd = new JSONObject()
.put("event_id", "event_" + System.currentTimeMillis())
.put("type", "input_audio_buffer.append")
.put("audio", encoded);
client.send(eventd.toString());
logger.info("Sending audio event: " + eventd.getString("event_id"));
} else {
break; // Avoid continuing to send after disconnection
}
Thread.sleep(100); // Simulate real-time sending
}
logger.info("File reading finished");
if (client != null && client.isOpen()) {
// Commit is required in non-VAD mode
if (!enableServerVad) {
JSONObject commitEvent = new JSONObject()
.put("event_id", "event_789")
.put("type", "input_audio_buffer.commit");
client.send(commitEvent.toString());
logger.info("Sent commit event for manual mode.");
}
JSONObject finishEvent = new JSONObject()
.put("event_id", "event_987")
.put("type", "session.finish");
client.send(finishEvent.toString());
logger.info("Sent finish event.");
}
}
/** Initialize logging */
private static void initLogger() {
logger.setLevel(Level.ALL);
Logger rootLogger = Logger.getLogger("");
for (Handler h : rootLogger.getHandlers()) {
rootLogger.removeHandler(h);
}
Handler consoleHandler = new ConsoleHandler();
consoleHandler.setLevel(Level.ALL);
consoleHandler.setFormatter(new SimpleFormatter());
logger.addHandler(consoleHandler);
}
}
Python
Before you run the example, install the dependencies with the following commands:
python -m pip install websocket-client
NoteDo not name the example file websocket.py, because it conflicts with the websocket module during import.
# pip install websocket-client
import os
import time
import json
import threading
import base64
from urllib.parse import urlsplit
import websocket
import logging
import logging.handlers
from datetime import datetime
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
API_KEY = os.environ["DASHSCOPE_API_KEY"]
QWEN_MODEL = "qwen3-asr-flash-realtime"
baseUrl = f"wss://{urlsplit(os.environ['DASHSCOPE_BASE_URL']).netloc}/api-ws/v1/realtime"
url = f"{baseUrl}?model={QWEN_MODEL}"
print(f"Connecting to server: {url}")
# Note: In non-VAD mode, it is recommended that the cumulative duration of continuously sent audio does not exceed 60s
enableServerVad = True
is_running = True # Add a running flag
headers = [
"Authorization: Bearer " + API_KEY,
"OpenAI-Beta: realtime=v1"
]
def init_logger():
formatter = logging.Formatter('%(asctime)s|%(levelname)s|%(message)s')
f_handler = logging.handlers.RotatingFileHandler(
"omni_tester.log", maxBytes=100 * 1024 * 1024, backupCount=3
)
f_handler.setLevel(logging.DEBUG)
f_handler.setFormatter(formatter)
console = logging.StreamHandler()
console.setLevel(logging.DEBUG)
console.setFormatter(formatter)
logger.addHandler(f_handler)
logger.addHandler(console)
def on_open(ws):
logger.info("Connected to server.")
# Session update event
event_manual = {
"event_id": "event_123",
"type": "session.update",
"session": {
"modalities": ["text"],
"input_audio_format": "pcm",
"sample_rate": 16000,
# "input_audio_transcription": {
# # Language identifier, optional; recommended to set it if the language is known
# "language": "zh"
# },
"turn_detection": None
}
}
event_vad = {
"event_id": "event_123",
"type": "session.update",
"session": {
"modalities": ["text"],
"input_audio_format": "pcm",
"sample_rate": 16000,
# "input_audio_transcription": {
# "language": "zh"
# },
"turn_detection": {
"type": "server_vad",
"threshold": 0.2,
"silence_duration_ms": 400
}
}
}
if enableServerVad:
logger.info(f"Sending event: {json.dumps(event_vad, indent=2)}")
ws.send(json.dumps(event_vad))
else:
logger.info(f"Sending event: {json.dumps(event_manual, indent=2)}")
ws.send(json.dumps(event_manual))
def on_message(ws, message):
global is_running
try:
data = json.loads(message)
logger.info(f"Received event: {json.dumps(data, ensure_ascii=False, indent=2)}")
if data.get("type") == "conversation.item.input_audio_transcription.completed":
logger.info(f"Final transcript: {data.get('transcript')}")
elif data.get("type") == "session.finished":
logger.info("Closing WebSocket connection after session finished...")
is_running = False # Stop the audio sending thread
ws.close()
except json.JSONDecodeError:
logger.error(f"Failed to parse message: {message}")
def on_error(ws, error):
logger.error(f"Error: {error}")
def on_close(ws, close_status_code, close_msg):
logger.info(f"Connection closed: {close_status_code} - {close_msg}")
def send_audio(ws, local_audio_path):
time.sleep(3) # Wait for the session update to complete
global is_running
with open(local_audio_path, 'rb') as audio_file:
logger.info(f"File reading started: {datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]}")
while is_running:
audio_data = audio_file.read(3200) # ~0.1s PCM16/16kHz
if not audio_data:
logger.info(f"File reading finished: {datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]}")
if ws.sock and ws.sock.connected:
if not enableServerVad:
commit_event = {
"event_id": "event_789",
"type": "input_audio_buffer.commit"
}
ws.send(json.dumps(commit_event))
finish_event = {
"event_id": "event_987",
"type": "session.finish"
}
ws.send(json.dumps(finish_event))
break
if not ws.sock or not ws.sock.connected:
logger.info("WebSocket is closed, stopping audio sending.")
break
encoded_data = base64.b64encode(audio_data).decode('utf-8')
eventd = {
"event_id": f"event_{int(time.time() * 1000)}",
"type": "input_audio_buffer.append",
"audio": encoded_data
}
ws.send(json.dumps(eventd))
logger.info(f"Sending audio event: {eventd['event_id']}")
time.sleep(0.1) # Simulate real-time capture
# Initialize logging
init_logger()
logger.info(f"Connecting to WebSocket server at {url}...")
local_audio_path = "asr_example.pcm"
ws = websocket.WebSocketApp(
url,
header=headers,
on_open=on_open,
on_message=on_message,
on_error=on_error,
on_close=on_close
)
thread = threading.Thread(target=send_audio, args=(ws, local_audio_path))
thread.start()
ws.run_forever()
Go
Initialize a module in a new Go example directory and install the dependencies:
go mod init asr-example
go get github.com/gorilla/websocket@v1.5.3
Before you run the example, install the required dependency:
package main
import (
"encoding/base64"
"encoding/json"
"fmt"
"io"
"log"
"net/http"
neturl "net/url"
"os"
"time"
"github.com/gorilla/websocket"
)
const (
model = "qwen3-asr-flash-realtime"
audioFile = "asr_example.pcm" // Replace with the path to your PCM audio file
enableServerVad = true // Controls whether to use VAD mode
)
// Server-side event structure
type ServerEvent struct {
Type string `json:"type"`
Transcript string `json:"transcript,omitempty"`
}
func main() {
base, err := neturl.Parse(os.Getenv("DASHSCOPE_BASE_URL"))
if err != nil {
log.Fatal(err)
}
baseURL := "wss://" + base.Host + "/api-ws/v1/realtime"
apiKey := os.Getenv("DASHSCOPE_API_KEY")
url := baseURL + "?model=" + model
log.Printf("Connecting to server: %s", url)
conn, err := connect(url, apiKey)
if err != nil {
log.Fatal("Failed to connect to WebSocket: ", err)
}
defer conn.Close()
// Start a goroutine to receive messages
sessionFinished := make(chan bool, 1)
go receiveMessages(conn, sessionFinished)
// Send session.update
if err := sendSessionUpdate(conn); err != nil {
log.Fatal("Failed to send session.update: ", err)
}
// Wait for the session configuration to complete
time.Sleep(3 * time.Second)
// Send the audio stream
if err := sendAudioStream(conn); err != nil {
log.Fatal("Failed to send audio: ", err)
}
// Wait for session.finished
<-sessionFinished
}
// Establish the WebSocket connection
func connect(url, apiKey string) (*websocket.Conn, error) {
headers := http.Header{}
headers.Set("Authorization", "Bearer "+apiKey)
headers.Set("OpenAI-Beta", "realtime=v1")
conn, _, err := websocket.DefaultDialer.Dial(url, headers)
return conn, err
}
// Send the session.update event
func sendSessionUpdate(conn *websocket.Conn) error {
session := map[string]interface{}{
"modalities": []string{"text"},
"input_audio_format": "pcm",
"sample_rate": 16000,
// "input_audio_transcription": map[string]interface{}{
// "language": "zh",
// },
}
if enableServerVad {
session["turn_detection"] = map[string]interface{}{
"type": "server_vad",
"threshold": 0.2,
"silence_duration_ms": 400,
}
} else {
session["turn_detection"] = nil
}
event := map[string]interface{}{
"event_id": "event_123",
"type": "session.update",
"session": session,
}
payload, _ := json.Marshal(event)
log.Printf("Sending session.update: %s", string(payload))
return conn.WriteMessage(websocket.TextMessage, payload)
}
// Send the audio stream (send one PCM chunk every 100ms)
func sendAudioStream(conn *websocket.Conn) error {
f, err := os.Open(audioFile)
if err != nil {
return err
}
defer f.Close()
chunk := make([]byte, 3200) // 100ms @ 16kHz 16bit mono
for {
n, err := f.Read(chunk)
if n > 0 {
event := map[string]interface{}{
"event_id": fmt.Sprintf("event_%d", time.Now().UnixMilli()),
"type": "input_audio_buffer.append",
"audio": base64.StdEncoding.EncodeToString(chunk[:n]),
}
payload, _ := json.Marshal(event)
if err := conn.WriteMessage(websocket.TextMessage, payload); err != nil {
return err
}
time.Sleep(100 * time.Millisecond)
}
if err == io.EOF {
break
}
if err != nil {
return err
}
}
log.Println("Audio stream ended")
if !enableServerVad {
commitEvt := map[string]interface{}{
"event_id": "event_789",
"type": "input_audio_buffer.commit",
}
payload, _ := json.Marshal(commitEvt)
if err := conn.WriteMessage(websocket.TextMessage, payload); err != nil {
return err
}
}
finishEvt := map[string]interface{}{
"event_id": "event_987",
"type": "session.finish",
}
payload, _ := json.Marshal(finishEvt)
return conn.WriteMessage(websocket.TextMessage, payload)
}
// Receive and handle server-side events
func receiveMessages(conn *websocket.Conn, sessionFinished chan<- bool) {
for {
_, msg, err := conn.ReadMessage()
if err != nil {
log.Println("Error reading message: ", err)
sessionFinished <- true
return
}
var evt ServerEvent
if err := json.Unmarshal(msg, &evt); err != nil {
log.Println("Error parsing message: ", err)
continue
}
log.Printf("Received event: %s", evt.Type)
switch evt.Type {
case "conversation.item.input_audio_transcription.completed":
log.Printf("Final transcript: %s", evt.Transcript)
case "session.finished":
log.Println("Session finished")
sessionFinished <- true
return
}
}
}
Node.js
Before you run the example, install the dependencies with the following command:
npm install ws
/**
* Qwen-ASR Realtime WebSocket client (Node.js version)
* Features:
* - Supports VAD mode and Manual mode
* - Sends session.update to start the session
* - Continuously sends audio chunks via input_audio_buffer.append
* - In Manual mode, sends input_audio_buffer.commit
* - Sends the session.finish event
* - Closes the connection after receiving the session.finished event
*/
import WebSocket from 'ws';
import fs from 'fs';
// ===== Configuration =====
const API_KEY = process.env.DASHSCOPE_API_KEY;
const MODEL = 'qwen3-asr-flash-realtime';
const enableServerVad = true; // true for VAD mode, false for Manual mode
const localAudioPath = 'asr_example.pcm'; // Path to the PCM16, 16kHz audio file
const baseUrl = `wss://${new URL(process.env.DASHSCOPE_BASE_URL).host}/api-ws/v1/realtime`;
const url = `${baseUrl}?model=${MODEL}`;
console.log(`Connecting to server: ${url}`);
// ===== State control =====
let isRunning = true;
// ===== Establish the connection =====
const ws = new WebSocket(url, {
headers: {
'Authorization': `Bearer ${API_KEY}`,
'OpenAI-Beta': 'realtime=v1'
}
});
// ===== Event bindings =====
ws.on('open', () => {
console.log('[WebSocket] Connected to server.');
sendSessionUpdate();
// Start the audio sending thread
sendAudio(localAudioPath);
});
ws.on('message', (message) => {
try {
const data = JSON.parse(message);
console.log('[Received Event]:', JSON.stringify(data, null, 2));
// The final recognition result is in the transcription.completed event
if (data.type === 'conversation.item.input_audio_transcription.completed') {
console.log(`[Final Transcript] ${data.transcript}`);
}
// Received the finished event
if (data.type === 'session.finished') {
console.log('[Action] Closing WebSocket connection after session finished...');
if (ws.readyState === WebSocket.OPEN) {
ws.close(1000, 'ASR finished');
}
}
} catch (e) {
console.error('[Error] Failed to parse message:', message);
}
});
ws.on('close', (code, reason) => {
console.log(`[WebSocket] Connection closed: ${code} - ${reason}`);
});
ws.on('error', (err) => {
console.error('[WebSocket Error]', err);
});
// ===== Session update =====
function sendSessionUpdate() {
const eventNoVad = {
event_id: 'event_123',
type: 'session.update',
session: {
modalities: ['text'],
input_audio_format: 'pcm',
sample_rate: 16000,
// input_audio_transcription: {
// language: 'zh'
// },
turn_detection: null
}
};
const eventVad = {
event_id: 'event_123',
type: 'session.update',
session: {
modalities: ['text'],
input_audio_format: 'pcm',
sample_rate: 16000,
// input_audio_transcription: {
// language: 'zh'
// },
turn_detection: {
type: 'server_vad',
threshold: 0.2,
silence_duration_ms: 400
}
}
};
if (enableServerVad) {
console.log('[Send Event] VAD Mode:\n', JSON.stringify(eventVad, null, 2));
ws.send(JSON.stringify(eventVad));
} else {
console.log('[Send Event] Manual Mode:\n', JSON.stringify(eventNoVad, null, 2));
ws.send(JSON.stringify(eventNoVad));
}
}
// ===== Send the audio file stream =====
function sendAudio(audioPath) {
setTimeout(() => {
console.log(`[File Read Start] ${audioPath}`);
const buffer = fs.readFileSync(audioPath);
let offset = 0;
const chunkSize = 3200; // About 0.1s of PCM16 audio
function sendChunk() {
if (!isRunning) return;
if (offset >= buffer.length) {
isRunning = false; // Stop sending audio
console.log('[File Read End]');
if (ws.readyState === WebSocket.OPEN) {
if (!enableServerVad) {
const commitEvent = {
event_id: 'event_789',
type: 'input_audio_buffer.commit'
};
ws.send(JSON.stringify(commitEvent));
console.log('[Send Commit Event]');
}
const finishEvent = {
event_id: 'event_987',
type: 'session.finish'
};
ws.send(JSON.stringify(finishEvent));
console.log('[Send Finish Event]');
}
return;
}
if (ws.readyState !== WebSocket.OPEN) {
console.log('[Stop] WebSocket is not open.');
return;
}
const chunk = buffer.slice(offset, offset + chunkSize);
offset += chunkSize;
const encoded = chunk.toString('base64');
const appendEvent = {
event_id: `event_${Date.now()}`,
type: 'input_audio_buffer.append',
audio: encoded
};
ws.send(JSON.stringify(appendEvent));
console.log(`[Send Audio Event] ${appendEvent.event_id}`);
setTimeout(sendChunk, 100); // Simulate real-time sending
}
sendChunk();
}, 3000); // Wait for the session configuration to complete
}
Save the code as asr_example.mjs and run node asr_example.mjs.
C#
Save the code as Program.cs and run it in a .NET console project.
The example code is as follows:
using System.Net.WebSockets;
using System.Text;
using System.Text.Json.Nodes;
class Program {
private static ClientWebSocket _webSocket = new ClientWebSocket();
private static CancellationTokenSource _cts = new CancellationTokenSource();
private static bool _sessionFinished = false;
private static bool _isRunning = true;
// Controls whether to use VAD mode
private const bool EnableServerVad = true;
private static readonly string ApiKey = Environment.GetEnvironmentVariable("DASHSCOPE_API_KEY") ?? throw new InvalidOperationException("DASHSCOPE_API_KEY environment variable is not set.");
private const string Model = "qwen3-asr-flash-realtime";
private static readonly string BaseUrl = "wss://" + new Uri(Environment.GetEnvironmentVariable("DASHSCOPE_BASE_URL")!).Authority + "/api-ws/v1/realtime";
private const string AudioFilePath = "asr_example.pcm"; // Replace with the path to your PCM audio file
static async Task Main(string[] args) {
var url = $"{BaseUrl}?model={Model}";
Console.WriteLine($"Connecting to server: {url}");
// Set authentication headers
_webSocket.Options.SetRequestHeader("Authorization", $"Bearer {ApiKey}");
_webSocket.Options.SetRequestHeader("OpenAI-Beta", "realtime=v1");
await _webSocket.ConnectAsync(new Uri(url), _cts.Token);
Console.WriteLine("Connected to server.");
// Start the message receiving task
var receiveTask = ReceiveMessagesAsync();
// Send the session.update configuration
await SendSessionUpdateAsync();
// Send the audio stream
await SendAudioStreamAsync();
// Wait for the session.finished event
while (!_sessionFinished && !_cts.IsCancellationRequested) {
await Task.Delay(100);
}
if (_webSocket.State == WebSocketState.Open) {
await _webSocket.CloseAsync(WebSocketCloseStatus.NormalClosure, "ASR finished", _cts.Token);
}
}
private static async Task SendAsync(string text) {
var bytes = Encoding.UTF8.GetBytes(text);
await _webSocket.SendAsync(new ArraySegment<byte>(bytes), WebSocketMessageType.Text, true, _cts.Token);
}
// Send the session.update event
private static async Task SendSessionUpdateAsync() {
var session = new JsonObject {
["modalities"] = new JsonArray { "text" },
["input_audio_format"] = "pcm",
["sample_rate"] = 16000,
// ["input_audio_transcription"] = new JsonObject { ["language"] = "zh" }
};
if (EnableServerVad) {
session["turn_detection"] = new JsonObject {
["type"] = "server_vad",
["threshold"] = 0.2,
["silence_duration_ms"] = 400
};
} else {
session["turn_detection"] = null;
}
var payload = new JsonObject {
["event_id"] = "event_123",
["type"] = "session.update",
["session"] = session
};
Console.WriteLine($"Sending session.update: {payload.ToJsonString()}");
await SendAsync(payload.ToJsonString());
}
// Send the audio stream (send one PCM chunk every 100ms)
private static async Task SendAudioStreamAsync() {
await Task.Delay(3000); // Wait for the session configuration to complete
const int chunkSize = 3200; // 100ms @ 16kHz 16bit mono
using var fs = new FileStream(AudioFilePath, FileMode.Open, FileAccess.Read);
var buffer = new byte[chunkSize];
int read;
while (_isRunning && (read = await fs.ReadAsync(buffer, 0, chunkSize)) > 0) {
if (_webSocket.State != WebSocketState.Open) break;
string b64 = Convert.ToBase64String(buffer, 0, read);
var append = new JsonObject {
["event_id"] = $"event_{DateTimeOffset.Now.ToUnixTimeMilliseconds()}",
["type"] = "input_audio_buffer.append",
["audio"] = b64
};
await SendAsync(append.ToJsonString());
await Task.Delay(100);
}
Console.WriteLine("File read end.");
if (_webSocket.State == WebSocketState.Open) {
if (!EnableServerVad) {
var commit = new JsonObject {
["event_id"] = "event_789",
["type"] = "input_audio_buffer.commit"
};
await SendAsync(commit.ToJsonString());
}
var finish = new JsonObject {
["event_id"] = "event_987",
["type"] = "session.finish"
};
await SendAsync(finish.ToJsonString());
}
}
// Receive and handle server-side events
private static async Task ReceiveMessagesAsync() {
var buffer = new byte[16384];
var sb = new StringBuilder();
while (_webSocket.State == WebSocketState.Open && !_cts.IsCancellationRequested) {
try {
var result = await _webSocket.ReceiveAsync(new ArraySegment<byte>(buffer), _cts.Token);
if (result.MessageType == WebSocketMessageType.Close) {
await _webSocket.CloseAsync(WebSocketCloseStatus.NormalClosure, "Closing", _cts.Token);
break;
}
sb.Append(Encoding.UTF8.GetString(buffer, 0, result.Count));
if (!result.EndOfMessage) continue;
string text = sb.ToString();
sb.Clear();
var data = JsonNode.Parse(text);
string? type = data?["type"]?.GetValue<string>();
Console.WriteLine($"Received event: {type}");
if (type == "conversation.item.input_audio_transcription.completed") {
Console.WriteLine($"Final transcript: {data!["transcript"]}");
} else if (type == "session.finished") {
Console.WriteLine("Session finished, closing...");
_sessionFinished = true;
_isRunning = false;
break;
}
} catch (Exception ex) {
Console.WriteLine($"Receive error: {ex.Message}");
break;
}
}
}
}
PHP
Save the dependency configuration as composer.json, run composer install, and then run the PHP example.
The example project has the following directory structure:
my-php-project/
├── composer.json
├── vendor/
└── index.php
The contents of composer.json are as follows. Adjust the dependency versions as needed:
{
"require": {
"react/event-loop": "^1.3",
"react/socket": "^1.11",
"ratchet/pawl": "^0.4"
}
}
The contents of index.php are as follows:
<?php
require __DIR__ . '/vendor/autoload.php';
use Ratchet\Client\Connector;
use React\EventLoop\Loop;
use React\Socket\Connector as SocketConnector;
$api_key = getenv("DASHSCOPE_API_KEY");
$model = 'qwen3-asr-flash-realtime';
$base_url = 'wss://' . parse_url(getenv('DASHSCOPE_BASE_URL'), PHP_URL_HOST) . '/api-ws/v1/realtime';
$websocket_url = $base_url . '?model=' . $model;
$audio_file_path = 'asr_example.pcm'; // Replace with the path to your PCM audio file
// Controls whether to use VAD mode
$enable_server_vad = true;
$loop = Loop::get();
$socketConnector = new SocketConnector($loop, [
'tls' => ['verify_peer' => true, 'verify_peer_name' => true],
]);
$connector = new Connector($loop, $socketConnector);
$headers = [
'Authorization' => 'Bearer ' . $api_key,
'OpenAI-Beta' => 'realtime=v1',
];
$is_running = true;
$connector($websocket_url, [], $headers)->then(function ($conn) use ($loop, $audio_file_path, $enable_server_vad, &$is_running) {
echo "Connected to WebSocket server\n";
// Listen for server-side events
$conn->on('message', function($msg) use ($conn, &$is_running) {
$event = json_decode($msg, true);
if (!isset($event['type'])) {
return;
}
echo "Received event: {$event['type']}\n";
if ($event['type'] === 'conversation.item.input_audio_transcription.completed') {
echo "Final transcript: {$event['transcript']}\n";
} elseif ($event['type'] === 'session.finished') {
echo "Session finished, closing...\n";
$is_running = false;
$conn->close();
}
});
$conn->on('close', function() {
echo "Connection closed\n";
});
// Send the session.update event
sendSessionUpdate($conn, $enable_server_vad);
// Start sending audio after the session configuration completes
$loop->addTimer(3, function () use ($conn, $audio_file_path, $enable_server_vad, $loop, &$is_running) {
sendAudioStream($conn, $audio_file_path, $enable_server_vad, $loop, $is_running);
});
}, function ($e) {
echo "Unable to connect: {$e->getMessage()}\n";
});
$loop->run();
// Send the session.update event
function sendSessionUpdate($conn, $enable_server_vad) {
$session = [
'modalities' => ['text'],
'input_audio_format' => 'pcm',
'sample_rate' => 16000,
// 'input_audio_transcription' => ['language' => 'zh'],
'turn_detection' => $enable_server_vad ? [
'type' => 'server_vad',
'threshold' => 0.2,
'silence_duration_ms' => 400,
] : null,
];
$event = [
'event_id' => 'event_123',
'type' => 'session.update',
'session' => $session,
];
$conn->send(json_encode($event));
echo "Sent session.update\n";
}
// Send the audio stream (send one PCM chunk every 100ms)
function sendAudioStream($conn, $audio_file_path, $enable_server_vad, $loop, &$is_running) {
$fp = fopen($audio_file_path, 'rb');
if (!$fp) {
echo "Unable to open the audio file\n";
return;
}
$send_chunk = function() use ($conn, $fp, $enable_server_vad, $loop, &$send_chunk, &$is_running) {
if (!$is_running) {
fclose($fp);
return;
}
$chunk = fread($fp, 3200); // 100ms @ 16kHz 16bit mono
if ($chunk === false || strlen($chunk) === 0) {
fclose($fp);
echo "Audio stream ended\n";
if (!$enable_server_vad) {
$conn->send(json_encode([
'event_id' => 'event_789',
'type' => 'input_audio_buffer.commit',
]));
}
$conn->send(json_encode([
'event_id' => 'event_987',
'type' => 'session.finish',
]));
return;
}
$append = [
'event_id' => 'event_' . round(microtime(true) * 1000),
'type' => 'input_audio_buffer.append',
'audio' => base64_encode($chunk),
];
$conn->send(json_encode($append));
$loop->addTimer(0.1, $send_chunk);
};
$send_chunk();
}