Real-time speech synthesis

Learn about real-time speech synthesis features, streaming calls, and ways to improve synthesis quality.

Features

Real-time speech synthesis accepts text segments and streams audio back, enabling speech output as a large language model generates text.

• Languages and dialects: Select a model and voice for the target language, or use an instruction to specify a dialect. See Dialects.
• Expressive speech: Adjust delivery with natural-language instructions, or insert emotion and nonverbal tags into the text. See Instruction control and Emotion and nonverbal tags.
• Custom voices: Create a voice from an audio sample with voice cloning, or from a text description with voice design.
• Fine-grained control and formulas: Use SSML and LaTeX to control pauses and pronunciation, or convert formulas to speech.
• Audio controls: Adjust speed, pitch, volume, and bitrate, and output common audio formats such as PCM, WAV, MP3, and Opus at sample rates up to 48 kHz. See the corresponding API parameters for supported values and conditions.

Get started

• Obtain an API key and set the environment variable DASHSCOPE_API_KEY. Use a key for the region in the example.
• For a workspace-specific endpoint, replace {WorkspaceId} with the workspace ID for that region.
• Install the DashScope SDK and choose the Python or Java example.

import os
import dashscope
from dashscope.audio.tts_v2 import SpeechSynthesizer, ResultCallback, AudioFormat

dashscope.api_key = os.environ["DASHSCOPE_API_KEY"]
dashscope.base_websocket_api_url = "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference"

with open("output.mp3", "wb") as audio:
    class Callback(ResultCallback):
        def on_data(self, data: bytes):
            audio.write(data)

        def on_error(self, message):
            print(f"Synthesis failed: {message}")

    synthesizer = SpeechSynthesizer(
        model="qwen-audio-3.1-tts-flash",
        voice="longanhuan_v3.1",
        format=AudioFormat.MP3_24000HZ_MONO_256KBPS,
        callback=Callback(),
    )
    for text in ["你好，欢迎使用阿里云百炼。", "文本可以分段发送，音频会逐段返回。"]:
        synthesizer.streaming_call(text)
    synthesizer.streaming_complete()

import com.alibaba.dashscope.audio.tts.SpeechSynthesisResult;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisAudioFormat;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisParam;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesizer;
import com.alibaba.dashscope.common.ResultCallback;
import com.alibaba.dashscope.utils.Constants;
import java.io.FileOutputStream;
import java.nio.ByteBuffer;

public class Main {
    public static void main(String[] args) throws Exception {
        Constants.baseWebsocketApiUrl = "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference";
        try (FileOutputStream audio = new FileOutputStream("output.mp3")) {
            SpeechSynthesisParam param = SpeechSynthesisParam.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .model("qwen-audio-3.1-tts-flash")
                .voice("longanhuan_v3.1")
                .format(SpeechSynthesisAudioFormat.MP3_24000HZ_MONO_256KBPS)
                .build();
            ResultCallback<SpeechSynthesisResult> callback = new ResultCallback<>() {
                @Override
                public void onEvent(SpeechSynthesisResult result) {
                    ByteBuffer buffer = result.getAudioFrame();
                    if (buffer != null) {
                        byte[] bytes = new byte[buffer.remaining()];
                        buffer.get(bytes);
                        try { audio.write(bytes); }
                        catch (Exception e) { throw new RuntimeException(e); }
                    }
                }
                @Override
                public void onComplete() {}
                public void onError(Exception e) { e.printStackTrace(); }
            };
            SpeechSynthesizer synthesizer = new SpeechSynthesizer(param, callback);
            try {
                for (String text : new String[]{"你好，欢迎使用阿里云百炼。", "文本可以分段发送，音频会逐段返回。"}) {
                    synthesizer.streamingCall(text);
                }
                synthesizer.streamingComplete(60000);
            } finally {
                synthesizer.getDuplexApi().close(1000, "done");
            }
        }
    }
}

import os
import dashscope
from dashscope.audio.tts_v2 import SpeechSynthesizer, ResultCallback, AudioFormat

dashscope.api_key = os.environ["DASHSCOPE_API_KEY"]
dashscope.base_websocket_api_url = "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference"

with open("output.mp3", "wb") as audio:
    class Callback(ResultCallback):
        def on_data(self, data: bytes):
            audio.write(data)

        def on_error(self, message):
            print(f"Synthesis failed: {message}")

    synthesizer = SpeechSynthesizer(
        model="cosyvoice-v3-flash",
        voice="longanyang",
        format=AudioFormat.MP3_24000HZ_MONO_256KBPS,
        callback=Callback(),
    )
    for text in ["你好，欢迎使用阿里云百炼。", "文本可以分段发送，音频会逐段返回。"]:
        synthesizer.streaming_call(text)
    synthesizer.streaming_complete()

import com.alibaba.dashscope.audio.tts.SpeechSynthesisResult;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisAudioFormat;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisParam;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesizer;
import com.alibaba.dashscope.common.ResultCallback;
import com.alibaba.dashscope.utils.Constants;
import java.io.FileOutputStream;
import java.nio.ByteBuffer;

public class Main {
    public static void main(String[] args) throws Exception {
        Constants.baseWebsocketApiUrl = "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference";
        try (FileOutputStream audio = new FileOutputStream("output.mp3")) {
            SpeechSynthesisParam param = SpeechSynthesisParam.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .model("cosyvoice-v3-flash")
                .voice("longanyang")
                .format(SpeechSynthesisAudioFormat.MP3_24000HZ_MONO_256KBPS)
                .build();
            ResultCallback<SpeechSynthesisResult> callback = new ResultCallback<>() {
                @Override
                public void onEvent(SpeechSynthesisResult result) {
                    ByteBuffer buffer = result.getAudioFrame();
                    if (buffer != null) {
                        byte[] bytes = new byte[buffer.remaining()];
                        buffer.get(bytes);
                        try { audio.write(bytes); }
                        catch (Exception e) { throw new RuntimeException(e); }
                    }
                }
                @Override
                public void onComplete() {}
                public void onError(Exception e) { e.printStackTrace(); }
            };
            SpeechSynthesizer synthesizer = new SpeechSynthesizer(param, callback);
            try {
                for (String text : new String[]{"你好，欢迎使用阿里云百炼。", "文本可以分段发送，音频会逐段返回。"}) {
                    synthesizer.streamingCall(text);
                }
                synthesizer.streamingComplete(60000);
            } finally {
                synthesizer.getDuplexApi().close(1000, "done");
            }
        }
    }
}

Improve synthesis quality

Languages and pronunciation

Choose an approach based on what you need to control:

For supported models and parameter configuration, see the Qwen-Audio-TTS API or CosyVoice API. For the models, voices, interfaces, and usage of SSML and LaTeX, see SSML and LaTeX.

Instruction control

Use instruction to describe the speed, emotion, or speaking style in natural language and control the synthesized speech.

请用非常缓慢的语速说话，每个字都慢慢说。

Dialects

Use dialect synthesis to generate speech in Chinese dialects such as Henan or Sichuan dialect. How you select a dialect depends on the model and voice type.

synthesizer = SpeechSynthesizer(
    model="cosyvoice-v3-flash",
    voice="longanhuan_v3",
    instruction="请用河南话表达。",
    format=AudioFormat.MP3_24000HZ_MONO_256KBPS,
    callback=Callback(),
)
synthesizer.streaming_call("叫你去买盐，你买回来一袋面，这不是弄啥嘞吗！")
synthesizer.streaming_complete()

For Qwen3-TTS, select a voice that supports the target dialect from the Qwen3-TTS voice list.

Emotion and nonverbal tags

Emotion and paralinguistic tags control the emotion and style of speech and insert sounds such as laughter or sighs at specific positions. Write the tags directly in the text to synthesize, text, without additional parameters.

Usage examples

This text starts with an excited delivery, inserts laughter, and then continues speaking:

[excited]今天的天气真不错！[laughing]我们一起出去玩吧！

This text switches to an excited delivery in the second sentence:

[serious]请注意安全事项。[excited]好了，现在让我们开始吧！

Supported tags

Apply in production

Cancel task

With Qwen-Audio-TTS or CosyVoice, send a cancellation instruction to interrupt the current synthesis task. The server ends the task and returns task-finished.

WebSocket connection reuse

A WebSocket connection can handle successive synthesis rounds. Start the next round as follows for each model series:

For SDK object pool configuration and complete examples, see High-concurrency speech synthesis. For the Qwen3-TTS session flow, see the WebSocket API.

For Qwen-Audio-TTS / CosyVoice / Sambert, a connection closes automatically if no new task starts within 60 seconds after the previous task ends. Reconnect if the connection has closed or the task state is invalid.

Rate limits

Control request frequency and concurrency according to the model’s rate limits. For limits, common errors, and handling instructions, see Rate limits.

High-concurrency best practices

Use object pools to reuse synthesis objects and connections, and configure concurrency, warm-up, and error handling for your workload. See High-concurrency speech synthesis.

Supported models

For system voices and audio samples, see the Qwen-Audio-TTS, CosyVoice, Qwen3-TTS, and Sambert voice lists. A cloned or designed voice must be used with the target model specified when the voice was created.

API reference

For the connection workflow and examples, see AOQ integration. For supported models and versions, see Model and protocol support.

• Real-time speech synthesis - Qwen-Audio-TTS API reference
• Real-time speech synthesis - CosyVoice API reference
• Real-time speech synthesis - Qwen-TTS API reference
• Real-time speech synthesis - Sambert API reference

For complete raw WebSocket examples, see the following pages. Use the API entry for each series to find Python and Java SDK parameters and methods.

• Qwen-Audio-TTS WebSocket

• CosyVoice WebSocket

• Qwen3-TTS WebSocket

• Sambert WebSocket

FAQ

How do I troubleshoot silent audio when using a cloned voice?

Check that voice is the ID of a successfully created voice and that its target_model matches the synthesis model. For voice status, source audio requirements, and quality guidance, see Voice cloning.

Why does the actual duration of synthesized audio differ from the duration shown in the WAV file?

For streamed WAV output, the file header may not reflect the final audio length. To obtain accurate duration metadata, set format to pcm, receive the complete audio, and add a WAV header containing the actual length.

Why can't the audio file play?

Troubleshoot based on your scenario:

1. Audio saved as a complete file (such as xx.mp3)

   1. Audio format consistency: The audio format in the request parameters must match the file extension (for example, if the parameter is wav, the file must be .wav).
   2. Player compatibility: Confirm that the player supports the audio format and sample rate.

2. Streaming audio playback

   1. Save the audio stream as a complete file and try playing it with a media player. If the file doesn't play, refer to scenario 1 above.
   2. If the file plays correctly, the issue is in the streaming playback implementation. Confirm that the player supports streaming playback (such as ffmpeg, pyaudio, AudioFormat, or MediaSource).

Why is audio playback stuttering?

Troubleshoot with the following steps:

1. Check the text sending rate: Make sure the sending interval is reasonable to avoid the previous audio segment finishing before the next text arrives.

2. Check callback function performance:

   • Confirm that no blocking logic exists in the callback function.
   • Callbacks run on the WebSocket thread. Blocking operations affect data reception. Write audio data to a separate buffer and process it in another thread.

3. Check network stability: Network fluctuations can cause audio transmission interruptions or delays.

Why is speech synthesis taking a long time?

With segmented input, long gaps between text submissions increase total synthesis time. The first call also includes WebSocket connection overhead. For warm-up and timing in high-concurrency scenarios, see High-concurrency speech synthesis.

How do I restrict an API key to only the speech synthesis service (permission isolation)?

Create a new workspace and grant access to specific models only. This limits the scope of the API key. For details, see Manage workspaces.

Can an API key from a sub-workspace call Qwen-Audio-TTS/CosyVoice models?

In the default workspace, all models are accessible.

In a sub-workspace, you need to authorize models for the sub-workspace associated with the API key. For details, see Model calls in a sub-workspace.