Qwen-ASR-Realtime Java SDK

Updated at:

Use the DashScope Java SDK to call Qwen-ASR-Realtime.

ImportantAlibaba Cloud Model Studio has released workspace-specific domains for the China (Beijing) and Singapore regions. The new dedicated domains deliver superior performance and higher stability for inference requests. We recommend migrating to the new domains:

  • China (Beijing): from dashscope.aliyuncs.com to {WorkspaceId}.cn-beijing.maas.aliyuncs.com
  • Singapore: from dashscope-intl.aliyuncs.com to {WorkspaceId}.ap-southeast-1.maas.aliyuncs.com

Replace {WorkspaceId} with your actual Workspace ID. The existing domains remain fully functional.

Prerequisites

  1. Install the SDK and make sure the DashScope SDK version is 2.22.5 or later.
  2. Obtain and configure an API key.
  3. Learn about the WebSocket API.

Quick start

NoteThe example code reads your_audio_file.pcm (PCM16, 16 kHz, mono). If you only have an MP3, WAV, or similar format, convert it with ffmpeg:

ffmpeg -i your_audio.mp3 -ar 16000 -ac 1 -f s16le your_audio_file.pcm
import com.alibaba.dashscope.audio.omni.*;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.google.gson.JsonObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import javax.sound.sampled.LineUnavailableException;
import java.io.File;
import java.io.FileInputStream;
import java.util.Base64;
import java.util.Collections;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.atomic.AtomicReference;

public class Qwen3AsrRealtimeUsage {
    private static final Logger log = LoggerFactory.getLogger(Qwen3AsrRealtimeUsage.class);
    private static final int AUDIO_CHUNK_SIZE = 1024; // Audio chunk size in bytes
    private static final int SLEEP_INTERVAL_MS = 30;  // Sleep interval in milliseconds

    public static void main(String[] args) throws InterruptedException, LineUnavailableException {
        CountDownLatch finishLatch = new CountDownLatch(1);

        OmniRealtimeParam param = OmniRealtimeParam.builder()
                .model("qwen3-asr-flash-realtime")
                // The following is the configuration for the China (Beijing) region. When calling, replace "{WorkspaceId}" with your actual workspace ID. Configurations differ by region.
                .url("wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/realtime")
                // The API Key differs between the Singapore and Beijing regions. Get your API Key: https://help.aliyun.com/zh/model-studio/get-api-key
                // If you have not configured the environment variable, replace the following line with your Alibaba Cloud Model Studio API Key: .apikey("sk-xxx")
                .apikey(System.getenv("DASHSCOPE_API_KEY"))
                .build();

        OmniRealtimeConversation conversation = null;
        final AtomicReference<OmniRealtimeConversation> conversationRef = new AtomicReference<>(null);
        conversation = new OmniRealtimeConversation(param, new OmniRealtimeCallback() {
            @Override
            public void onOpen() {
                System.out.println("connection opened");
            }
            @Override
            public void onEvent(JsonObject message) {
                String type = message.get("type").getAsString();
                switch(type) {
                    case "session.created":
                        System.out.println("start session: " + message.get("session").getAsJsonObject().get("id").getAsString());
                        break;
                    case "conversation.item.input_audio_transcription.completed":
                        System.out.println("transcription: " + message.get("transcript").getAsString());
                        finishLatch.countDown();
                        break;
                    case "input_audio_buffer.speech_started":
                        System.out.println("======VAD Speech Start======");
                        break;
                    case "input_audio_buffer.speech_stopped":
                        System.out.println("======VAD Speech Stop======");
                        break;
                    case "conversation.item.input_audio_transcription.text":
                        System.out.println("transcription: " + message.get("text").getAsString() + message.get("stash").getAsString());
                        break;
                    default:
                        break;
                }
            }
            @Override
            public void onClose(int code, String reason) {
                System.out.println("connection closed code: " + code + ", reason: " + reason);
            }
        });
        conversationRef.set(conversation);
        try {
            conversation.connect();
        } catch (NoApiKeyException e) {
            throw new RuntimeException(e);
        }

        OmniRealtimeTranscriptionParam transcriptionParam = new OmniRealtimeTranscriptionParam();
        transcriptionParam.setLanguage("zh");
        transcriptionParam.setInputAudioFormat("pcm");
        transcriptionParam.setInputSampleRate(16000);

        OmniRealtimeConfig config = OmniRealtimeConfig.builder()
                .modalities(Collections.singletonList(OmniRealtimeModality.TEXT))
                .transcriptionConfig(transcriptionParam)
                .build();
        conversation.updateSession(config);

        String filePath = "your_audio_file.pcm";
        File audioFile = new File(filePath);
        if (!audioFile.exists()) {
            log.error("Audio file not found: {}", filePath);
            return;
        }

        try (FileInputStream audioInputStream = new FileInputStream(audioFile)) {
            byte[] audioBuffer = new byte[AUDIO_CHUNK_SIZE];
            int bytesRead;
            int totalBytesRead = 0;

            log.info("Starting to send audio data from: {}", filePath);

            // Read and send audio data in chunks
            while ((bytesRead = audioInputStream.read(audioBuffer)) != -1) {
                totalBytesRead += bytesRead;
                byte[] chunk = new byte[bytesRead];
                System.arraycopy(audioBuffer, 0, chunk, 0, bytesRead);
                String audioB64 = Base64.getEncoder().encodeToString(chunk);
                // Send audio chunk to conversation
                conversation.appendAudio(audioB64);

                // Add small delay to simulate real-time audio streaming
                Thread.sleep(SLEEP_INTERVAL_MS);
            }

            log.info("Finished sending audio data. Total bytes sent: {}", totalBytesRead);

        } catch (Exception e) {
            log.error("Error sending audio from file: {}", filePath, e);
        }

        //send session.finish and wait for finish and close
        conversation.endSession();
        log.info("task finished");

        System.exit(0);
    }
}

Request parameters

  • The following parameters are set with the chained methods of OmniRealtimeParam.

    Click to view sample code

    OmniRealtimeParam param = OmniRealtimeParam.builder()
                    .model("qwen3-asr-flash-realtime")
                    // The following is the configuration for the China (Beijing) region. When calling, replace "{WorkspaceId}" with your actual workspace ID. Configurations differ by region.
                    .url("wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/realtime")
                    // The API Key differs between the Singapore and Beijing regions. Get your API Key: https://help.aliyun.com/zh/model-studio/get-api-key
                    // If you have not configured the environment variable, replace the following line with your Model Studio API Key: .apikey("sk-xxx")
                    .apikey(System.getenv("DASHSCOPE_API_KEY"))
                    .build();
    

    Parameter

    Type

    Required

    Description

    model

    String

    Yes

    The name of the model to use.

    url

    String

    Yes

    Speech recognition service endpoint:

    • China (Beijing): wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/realtime. Replace {WorkspaceId} with your actual Workspace ID when calling.

    • Singapore: wss://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api-ws/v1/realtime. Replace {WorkspaceId} with your actual workspace ID when calling.

    apikey

    String

    No

    Set the API key.

  • The following parameters are set with the chained methods of OmniRealtimeConfig.

    Click to view sample code

    OmniRealtimeTranscriptionParam transcriptionParam = new OmniRealtimeTranscriptionParam();
            transcriptionParam.setLanguage("zh");
            transcriptionParam.setInputSampleRate(16000);
            transcriptionParam.setInputAudioFormat("pcm");
    
    OmniRealtimeConfig config = OmniRealtimeConfig.builder()
                    .modalities(Collections.singletonList(OmniRealtimeModality.TEXT))
                    .enableTurnDetection( true)
                    .turnDetectionType("server_vad")
                    .turnDetectionThreshold(0.0f)
                    .turnDetectionSilenceDurationMs(400)
                    .transcriptionConfig(transcriptionParam)
                    .build();
    

    Parameter

    Type

    Required

    Description

    modalities

    List<OmniRealtimeModality>

    Yes

    The output modality of the model, fixed to [OmniRealtimeModality.TEXT].

    enableTurnDetection

    boolean

    No

    Whether to enable server-side voice activity detection (VAD). When disabled, you must call the commit() method to trigger recognition.

    Default: true.

    Valid values:

    • true: enabled

    • false: disabled

    turnDetectionType

    String

    No

    Server-side VAD type, fixed to server_vad.

    turnDetectionThreshold

    float

    No

    The VAD detection threshold. We recommend setting this value to 0.0.

    Default: 0.5.

    Valid range: [-1, 1].

    A lower threshold increases the sensitivity of the VAD, which may cause background noise to be misclassified as speech. A higher threshold reduces sensitivity, which helps reduce false triggers in noisy environments.

    turnDetectionSilenceDurationMs

    int

    No

    The VAD segmentation threshold (in ms). Silence lasting longer than this threshold is treated as the end of an utterance. We recommend setting this value to 400.

    Default: 800.

    Valid range: [200, 6000].

    A lower value (such as 300 ms) allows the model to respond faster, but may cause unreasonable segmentation at natural pauses. A higher value (such as 1200 ms) better handles pauses within long sentences, but increases the overall response latency.

    transcriptionConfig

    OmniRealtimeTranscriptionParam

    No

    Speech recognition configuration.

  • The following parameters are set with the setter methods of OmniRealtimeTranscriptionParam.

    Click to view sample code

    OmniRealtimeTranscriptionParam transcriptionParam = new OmniRealtimeTranscriptionParam();
            transcriptionParam.setLanguage("zh");
            transcriptionParam.setInputSampleRate(16000);
            transcriptionParam.setInputAudioFormat("pcm");
    

    Parameter

    Type

    Required

    Description

    language

    String

    No

    The source language of the audio.

    • zh: Chinese (Mandarin, Sichuan dialect, Minnan dialect, Wu dialect)

    • yue: Cantonese

    • en: English

    • ja: Japanese

    • de: German

    • ko: Korean

    • ru: Russian

    • fr: French

    • pt: Portuguese

    • ar: Arabic

    • it: Italian

    • es: Spanish

    • hi: Hindi

    • id: Indonesian

    • th: Thai

    • tr: Turkish

    • uk: Ukrainian

    • vi: Vietnamese

    • cs: Czech

    • da: Danish

    • fil: Filipino

    • fi: Finnish

    • is: Icelandic

    • ms: Malay

    • no: Norwegian

    • pl: Polish

    • sv: Swedish

    inputSampleRate

    int

    No

    The audio sampling rate (Hz). Supports 16000 and 8000.

    Default: 16000.

    When set to 8000, the server first upsamples the audio to 16,000 Hz before recognition, which may introduce minor latency. We recommend using it only when the source audio is 8,000 Hz (such as telephone lines).

    inputAudioFormat

    String

    No

    The audio format. Supports pcm and opus.

    Default: pcm.

API reference

OmniRealtimeConversation class

OmniRealtimeConversation is imported via import com.alibaba.dashscope.audio.omni.OmniRealtimeConversation;.

Method signatureServer response events (delivered via callback)Description
public OmniRealtimeConversation(OmniRealtimeParam param, OmniRealtimeCallback callback)

None

Constructor.

public void connect() throws NoApiKeyException, InterruptedException

session.created

Session created

session.updated

Session configuration updated

Creates a connection with the server.

public void updateSession(OmniRealtimeConfig config)

session.updated

Session configuration updated

Used to update the session configuration. We recommend calling this method first to apply the settings after the connection is established. If this method is not called, the system uses the default configuration. You only need to focus on the parameters involved in Request parameters.

public void appendAudio(String audioBase64)

None

Appends a Base64-encoded audio chunk to the cloud input audio buffer.

  • When Request parameters enableTurnDetection is set to true, the audio buffer is used to detect speech, and the server decides when to commit.
  • When Request parameters enableTurnDetection is set to false, the client can choose how much audio to include in each event, up to 15 MiB. For example, streaming smaller chunks from the client can make the VAD respond faster.
public void commit()

input_audio_buffer.committed

The server received the committed audio

Commits the audio and video previously appended to the cloud buffer via append. An error occurs if the input audio buffer is empty.

Disabled when: Request parameters enableTurnDetection is set to true.

public void endSession() throws InterruptedException

session.finished

The server completes speech recognition and ends the session

Notifies the server to end the session. After receiving the end-of-session notification, the server completes the final speech recognition.

When to call:

endSessionAsync is the asynchronous version of endSession. The two are functionally identical.

public void close()

None

Terminates the task and closes the connection.

public String getSessionId()

None

Gets the session_id of the current task.

public String getResponseId()

None

Gets the response_id of the most recent response.

Callback interface (OmniRealtimeCallback)

The server returns response events and data to the client via callbacks.

Inherit this class and implement the corresponding methods to handle server events.

Import it via import com.alibaba.dashscope.audio.omni.OmniRealtimeCallback;.

Method signatureParametersDescription
public void onOpen()

None

Triggered when the WebSocket connection is successfully established.

public abstract void onEvent(JsonObject message)

message: server event

Triggered when a server event is received.

public abstract void onClose(int code, String reason)

code: status code

reason: log information when the WebSocket connection is closed

Triggered when the WebSocket connection is closed.