Real-time speech synthesis

Learn about real-time speech synthesis features, streaming calls, and ways to improve synthesis quality.

Features

Real-time speech synthesis accepts text segments and streams audio back, enabling speech output as a large language model generates text.

• Languages and dialects: Select a model and voice for the target language, or use an instruction to specify a dialect. See Dialects.
• Expressive speech: Adjust delivery with natural-language instructions, or insert emotion and nonverbal tags into the text. See Instruction control and Emotion and nonverbal tags.
• Custom voices: Create a voice from an audio sample with voice cloning, or from a text description with voice design.
• Fine-grained control and formulas: Use SSML and LaTeX to control pauses and pronunciation, or convert formulas to speech.
• Audio controls: Adjust speed, pitch, volume, and bitrate, and output common audio formats such as PCM, WAV, MP3, and Opus at sample rates up to 48 kHz. See the corresponding API parameters for supported values and conditions.

Get started

• Obtain an API key and set the environment variable DASHSCOPE_API_KEY. Use a key for the region in the example.
• For a workspace-specific endpoint, replace {WorkspaceId} with the workspace ID for that region.
• Install the DashScope SDK and choose the Python or Java example.

import os
import dashscope

# If you have not set the API key in an environment variable, replace the following line with your own API key: api_key="sk-xxx"
# China (Beijing): Replace {WorkspaceId} with your actual workspace ID.
client = dashscope.Client(
    api_key=os.environ["DASHSCOPE_API_KEY"],
    workspace="{WorkspaceId}",
    region="cn-beijing",
)

with open("output.mp3", "wb") as audio:
    # Send text in segments and receive audio in segments
    with client.audio.tts.connect(
        model="qwen-audio-3.1-tts-flash",
        voice="longanhuan_v3.1",
        format="mp3",
        sample_rate=24000,
    ) as session:
        for text in ["你好，欢迎使用阿里云百炼。", "文本可以分段发送，音频会逐段返回。"]:
            session.send_text(text)
        session.flush()  # Trigger synthesis of buffered text (input not ended)
        session.stop()   # End of input; wait for the remaining audio
        for chunk in session.audio_chunks():
            audio.write(chunk)

import com.alibaba.dashscope.audio.tts.SpeechSynthesisResult;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisAudioFormat;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisParam;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesizer;
import com.alibaba.dashscope.common.ResultCallback;
import com.alibaba.dashscope.utils.Constants;
import java.io.FileOutputStream;
import java.nio.ByteBuffer;

public class Main {
    public static void main(String[] args) throws Exception {
        Constants.baseWebsocketApiUrl = "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference";
        try (FileOutputStream audio = new FileOutputStream("output.mp3")) {
            SpeechSynthesisParam param = SpeechSynthesisParam.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .model("qwen-audio-3.1-tts-flash")
                .voice("longanhuan_v3.1")
                .format(SpeechSynthesisAudioFormat.MP3_24000HZ_MONO_256KBPS)
                .build();
            ResultCallback<SpeechSynthesisResult> callback = new ResultCallback<>() {
                @Override
                public void onEvent(SpeechSynthesisResult result) {
                    ByteBuffer buffer = result.getAudioFrame();
                    if (buffer != null) {
                        byte[] bytes = new byte[buffer.remaining()];
                        buffer.get(bytes);
                        try { audio.write(bytes); }
                        catch (Exception e) { throw new RuntimeException(e); }
                    }
                }
                @Override
                public void onComplete() {}
                public void onError(Exception e) { e.printStackTrace(); }
            };
            SpeechSynthesizer synthesizer = new SpeechSynthesizer(param, callback);
            try {
                for (String text : new String[]{"你好，欢迎使用阿里云百炼。", "文本可以分段发送，音频会逐段返回。"}) {
                    synthesizer.streamingCall(text);
                }
                synthesizer.streamingComplete(60000);
            } finally {
                synthesizer.getDuplexApi().close(1000, "done");
            }
        }
    }
}

import os
import dashscope

client = dashscope.Client(
    api_key=os.environ["DASHSCOPE_API_KEY"],
    workspace="{WorkspaceId}",
    region="cn-beijing",
)

with open("output.mp3", "wb") as audio:
    # Send text in segments and receive audio in segments
    with client.audio.tts.connect(
        model="cosyvoice-v3-flash",
        voice="longanyang",
        format="mp3",
        sample_rate=24000,
    ) as session:
        for text in ["你好，欢迎使用阿里云百炼。", "文本可以分段发送，音频会逐段返回。"]:
            session.send_text(text)
        session.flush()  # Trigger synthesis of buffered text (input not ended)
        session.stop()   # End of input; wait for the remaining audio
        for chunk in session.audio_chunks():
            audio.write(chunk)

import com.alibaba.dashscope.audio.tts.SpeechSynthesisResult;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisAudioFormat;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisParam;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesizer;
import com.alibaba.dashscope.common.ResultCallback;
import com.alibaba.dashscope.utils.Constants;
import java.io.FileOutputStream;
import java.nio.ByteBuffer;

public class Main {
    public static void main(String[] args) throws Exception {
        Constants.baseWebsocketApiUrl = "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference";
        try (FileOutputStream audio = new FileOutputStream("output.mp3")) {
            SpeechSynthesisParam param = SpeechSynthesisParam.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .model("cosyvoice-v3-flash")
                .voice("longanyang")
                .format(SpeechSynthesisAudioFormat.MP3_24000HZ_MONO_256KBPS)
                .build();
            ResultCallback<SpeechSynthesisResult> callback = new ResultCallback<>() {
                @Override
                public void onEvent(SpeechSynthesisResult result) {
                    ByteBuffer buffer = result.getAudioFrame();
                    if (buffer != null) {
                        byte[] bytes = new byte[buffer.remaining()];
                        buffer.get(bytes);
                        try { audio.write(bytes); }
                        catch (Exception e) { throw new RuntimeException(e); }
                    }
                }
                @Override
                public void onComplete() {}
                public void onError(Exception e) { e.printStackTrace(); }
            };
            SpeechSynthesizer synthesizer = new SpeechSynthesizer(param, callback);
            try {
                for (String text : new String[]{"你好，欢迎使用阿里云百炼。", "文本可以分段发送，音频会逐段返回。"}) {
                    synthesizer.streamingCall(text);
                }
                synthesizer.streamingComplete(60000);
            } finally {
                synthesizer.getDuplexApi().close(1000, "done");
            }
        }
    }
}

Improve synthesis quality

Languages and pronunciation

Choose an approach based on what you need to control:

For supported models and parameter configuration, see the Qwen-Audio-TTS API or CosyVoice API. For the models, voices, interfaces, and usage of SSML and LaTeX, see SSML and LaTeX.

Instruction control

Use instruction to specify the synthesis language and describe the speech rate, emotion, or speaking style in natural language to control the synthesized speech.

请用荷兰话说，每个字都慢慢说

Dialects

Use dialect synthesis to generate speech in Chinese dialects such as Henan or Sichuan dialect. How you select a dialect depends on the model and voice type.

with client.audio.tts.connect(
    model="cosyvoice-v3-flash",
    voice="longanhuan_v3",
    parameters={"instruction": "请用河南话表达。"},  # Instruction control
    format="mp3",
    sample_rate=24000,
) as session:
    session.send_text("叫你去买盐，你买回来一袋面，这不是弄啥嘞吗！")
    session.flush()

For Qwen3-TTS, select a voice that supports the target dialect from the Qwen3-TTS voice list.

Emotion and nonverbal tags

Emotion and paralinguistic tags control the emotion and style of speech and insert sounds such as laughter or sighs at specific positions. Write the tags directly in the text to synthesize, text, without additional parameters.

Usage examples

This text starts with an excited delivery, inserts laughter, and then continues speaking:

[excited]今天的天气真不错！[laughing]我们一起出去玩吧！

This text switches to an excited delivery in the second sentence:

[serious]请注意安全事项。[excited]好了，现在让我们开始吧！

Supported tags

Apply in production

Cancel task

With Qwen-Audio-TTS or CosyVoice, send a cancellation instruction to interrupt the current synthesis task. The server ends the task and returns task-finished.

WebSocket connection reuse

A WebSocket connection can handle successive synthesis rounds. Start the next round as follows for each model series:

For SDK object pool configuration and complete examples, see High-concurrency speech synthesis. For the Qwen3-TTS session flow, see the WebSocket API.

For Qwen-Audio-TTS / CosyVoice / Sambert, a connection closes automatically if no new task starts within 60 seconds after the previous task ends. Reconnect if the connection has closed or the task state is invalid.

Rate limits

Control request frequency and concurrency according to the model’s rate limits. For limits, common errors, and handling instructions, see Rate limits.

High-concurrency best practices

Use object pools to reuse synthesis objects and connections, and configure concurrency, warm-up, and error handling for your workload. See High-concurrency speech synthesis.

Supported models

For system voices and audio samples, see the Qwen-Audio-TTS, CosyVoice, Qwen3-TTS, and Sambert voice lists. A cloned or designed voice must be used with the target model specified when the voice was created.

API reference

For AOQ access, see Qwen-Audio-TTS AOQ access or CosyVoice AOQ access. For the exact support scope, see Model and protocol support.

• Qwen-Audio-TTS API reference
• CosyVoice API reference
• Qwen-TTS-Realtime API reference
• Sambert API reference

For complete raw WebSocket examples, see the following pages. Use the API entry for each series to find Python and Java SDK parameters and methods.

• Qwen-Audio-TTS WebSocket

• CosyVoice WebSocket

• Qwen3-TTS WebSocket

• Sambert WebSocket

FAQ

How do I troubleshoot silent audio when using a cloned voice?

Check that voice is the ID of a successfully created voice and that its target_model matches the synthesis model. For voice status, source audio requirements, and quality guidance, see Voice cloning.

Why does the actual duration of synthesized audio differ from the duration shown in the WAV file?

For streamed WAV output, the file header may not reflect the final audio length. To obtain accurate duration metadata, set format to pcm, receive the complete audio, and add a WAV header containing the actual length.

Why can't the audio file play?

Troubleshoot based on your scenario:

1. Audio saved as a complete file (such as xx.mp3)

   1. Audio format consistency: The audio format in the request parameters must match the file extension (for example, if the parameter is wav, the file must be .wav).
   2. Player compatibility: Confirm that the player supports the audio format and sample rate.

2. Streaming audio playback

   1. Save the audio stream as a complete file and try playing it with a media player. If the file doesn't play, refer to scenario 1 above.
   2. If the file plays correctly, the issue is in the streaming playback implementation. Confirm that the player supports streaming playback (such as ffmpeg, pyaudio, AudioFormat, or MediaSource).

Why is audio playback stuttering?

Troubleshoot with the following steps:

1. Check the text sending rate: Make sure the sending interval is reasonable to avoid the previous audio segment finishing before the next text arrives.

2. Check callback function performance:

   • Confirm that no blocking logic exists in the callback function.
   • Callbacks run on the WebSocket thread. Blocking operations affect data reception. Write audio data to a separate buffer and process it in another thread.

3. Check network stability: Network fluctuations can cause audio transmission interruptions or delays.

Why is speech synthesis taking a long time?

With segmented input, long gaps between text submissions increase total synthesis time. The first call also includes WebSocket connection overhead. For warm-up and timing in high-concurrency scenarios, see High-concurrency speech synthesis.

How do I restrict an API key to only the speech synthesis service (permission isolation)?

Create a new workspace and grant access to specific models only. This limits the scope of the API key. For details, see Manage workspaces.

Can an API key from a sub-workspace call Qwen-Audio-TTS/CosyVoice models?

In the default workspace, all models are accessible.

In a sub-workspace, you need to authorize models for the sub-workspace associated with the API key. For details, see Model calls in a sub-workspace.