Best practices for high-concurrency speech synthesis

Reuse synthesis objects and WebSocket connections with object pools, and configure concurrency, warm-up, and error handling to reduce connection overhead.

Scope

Object pools retain synthesis objects and their WebSocket connections to reduce object creation, TCP/TLS handshakes, and protocol upgrades. The following model series and SDKs support object pooling:

• Qwen-Audio-TTS, CosyVoice: Python and Java object pools.
• Sambert: Java object pools. The Python SDK does not provide an equivalent pool wrapper.

Prerequisites

• Get an API key and set DASHSCOPE_API_KEY. Use a key for the region you call.
• Replace {WorkspaceId} with the region’s workspace ID and configure its WebSocket endpoint.

pip install "dashscope>=1.25.2"

<dependencies>
    <dependency>
        <groupId>com.alibaba</groupId>
        <artifactId>dashscope-sdk-java</artifactId>
        <version>2.23.2</version>
    </dependency>
    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-pool2</artifactId>
        <version>2.12.0</version>
    </dependency>
</dependencies>

dependencies {
    implementation 'com.alibaba:dashscope-sdk-java:2.23.2'
    implementation 'org.apache.commons:commons-pool2:2.12.0'
}

Configure concurrency and resource pools

Configure resource pools for the expected number of simultaneous tasks and server capacity. Model limits apply to the Alibaba Cloud account and are shared by its RAM users, workspaces, and API keys. For model request-rate and concurrency limits, see Rate limits.

• Start with a pool capacity of 1.5–2 times peak concurrency and adjust for the actual workload.
• The Python pool may create extra objects when all pooled objects are borrowed. These objects need new connections when used.

Code examples

Qwen-Audio-TTS/CosyVoice

The examples use qwen-audio-3.1-tts-flash and longanhuan_v3.1 for three concurrent synthesis tasks. For CosyVoice, change the model and voice to cosyvoice-v3-flash and longanyang. For other models and voice requirements, see Real-time speech synthesis.

import os
import threading
from concurrent.futures import ThreadPoolExecutor
import dashscope
from dashscope.audio.tts_v2 import SpeechSynthesizerObjectPool, ResultCallback, AudioFormat

dashscope.api_key = os.environ["DASHSCOPE_API_KEY"]
dashscope.base_websocket_api_url = "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference"
pool = SpeechSynthesizerObjectPool(max_size=3)

def synthesize(index):
    done = threading.Event()
    errors = []
    with open(f"output_{index}.mp3", "wb") as audio:
        class Callback(ResultCallback):
            def on_data(self, data: bytes):
                audio.write(data)

            def on_complete(self):
                done.set()

            def on_error(self, message):
                errors.append(message)
                done.set()

        synthesizer = pool.borrow_synthesizer(
            model="qwen-audio-3.1-tts-flash",
            voice="longanhuan_v3.1",
            format=AudioFormat.MP3_24000HZ_MONO_256KBPS,
            callback=Callback(),
        )
        successful = False
        try:
            synthesizer.streaming_call("你好，欢迎使用阿里云百炼。")
            synthesizer.streaming_complete(60000)
            if not done.wait(60):
                raise TimeoutError("Synthesis timed out")
            if errors:
                raise RuntimeError(errors[0])
            successful = True
        finally:
            if successful:
                pool.return_synthesizer(synthesizer)
            else:
                synthesizer.close()

try:
    with ThreadPoolExecutor(max_workers=3) as executor:
        list(executor.map(synthesize, range(3)))
finally:
    pool.shutdown()

import com.alibaba.dashscope.audio.tts.SpeechSynthesisResult;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisAudioFormat;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesisParam;
import com.alibaba.dashscope.audio.ttsv2.SpeechSynthesizer;
import com.alibaba.dashscope.common.ResultCallback;
import com.alibaba.dashscope.utils.Constants;
import org.apache.commons.pool2.BasePooledObjectFactory;
import org.apache.commons.pool2.PooledObject;
import org.apache.commons.pool2.impl.DefaultPooledObject;
import org.apache.commons.pool2.impl.GenericObjectPool;
import org.apache.commons.pool2.impl.GenericObjectPoolConfig;
import java.io.FileOutputStream;
import java.nio.ByteBuffer;
import java.time.Duration;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicReference;

public class PooledTts {
    static class Factory extends BasePooledObjectFactory<SpeechSynthesizer> {
        public SpeechSynthesizer create() { return new SpeechSynthesizer(); }
        public PooledObject<SpeechSynthesizer> wrap(SpeechSynthesizer value) {
            return new DefaultPooledObject<>(value);
        }
        public void destroyObject(PooledObject<SpeechSynthesizer> value) {
            value.getObject().getDuplexApi().close(1000, "done");
        }
    }

    static void synthesize(GenericObjectPool<SpeechSynthesizer> pool, int index) throws Exception {
        SpeechSynthesizer synthesizer = pool.borrowObject();
        boolean successful = false;
        AtomicReference<Exception> error = new AtomicReference<>();
        try (FileOutputStream audio = new FileOutputStream("output_" + index + ".mp3")) {
            SpeechSynthesisParam param = SpeechSynthesisParam.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .model("qwen-audio-3.1-tts-flash")
                .voice("longanhuan_v3.1")
                .format(SpeechSynthesisAudioFormat.MP3_24000HZ_MONO_256KBPS)
                .build();
            synthesizer.updateParamAndCallback(param, new ResultCallback<SpeechSynthesisResult>() {
                public void onEvent(SpeechSynthesisResult result) {
                    ByteBuffer buffer = result.getAudioFrame();
                    if (buffer != null) {
                        byte[] bytes = new byte[buffer.remaining()];
                        buffer.get(bytes);
                        try { audio.write(bytes); }
                        catch (Exception e) { error.set(e); }
                    }
                }
                public void onComplete() {}
                public void onError(Exception e) { error.set(e); }
            });
            synthesizer.streamingCall("你好，欢迎使用阿里云百炼。");
            synthesizer.streamingComplete(60000);
            if (error.get() != null) { throw error.get(); }
            successful = true;
        } finally {
            if (successful) { pool.returnObject(synthesizer); }
            else { pool.invalidateObject(synthesizer); }
        }
    }

    public static void main(String[] args) throws Exception {
        Constants.baseWebsocketApiUrl = "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference";
        GenericObjectPoolConfig<SpeechSynthesizer> config = new GenericObjectPoolConfig<>();
        int poolSize = Integer.parseInt(System.getenv().getOrDefault("COSYVOICE_OBJECTPOOL_SIZE", "500"));
        config.setMaxTotal(poolSize);
        config.setMaxIdle(poolSize);
        config.setMaxWait(Duration.ofSeconds(60));
        ExecutorService executor = Executors.newFixedThreadPool(3);
        try (GenericObjectPool<SpeechSynthesizer> pool = new GenericObjectPool<>(new Factory(), config)) {
            java.util.List<Future<?>> futures = new java.util.ArrayList<>();
            for (int i = 0; i < 3; i++) {
                final int index = i;
                futures.add(executor.submit(() -> {
                    try { synthesize(pool, index); }
                    catch (Exception e) { throw new RuntimeException(e); }
                }));
            }
            for (Future<?> future : futures) { future.get(); }
        } finally {
            executor.shutdownNow();
        }
    }
}

Sambert

This example shares one object pool across three concurrent one-way streaming synthesis tasks and saves audio to sambert_0.mp3, sambert_1.mp3, and sambert_2.mp3.

import com.alibaba.dashscope.audio.tts.SpeechSynthesisAudioFormat;
import com.alibaba.dashscope.audio.tts.SpeechSynthesisParam;
import com.alibaba.dashscope.audio.tts.SpeechSynthesisResult;
import com.alibaba.dashscope.audio.tts.SpeechSynthesizer;
import com.alibaba.dashscope.common.ResultCallback;
import com.alibaba.dashscope.utils.Constants;
import org.apache.commons.pool2.BasePooledObjectFactory;
import org.apache.commons.pool2.PooledObject;
import org.apache.commons.pool2.impl.DefaultPooledObject;
import org.apache.commons.pool2.impl.GenericObjectPool;
import org.apache.commons.pool2.impl.GenericObjectPoolConfig;

import java.io.FileOutputStream;
import java.time.Duration;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicReference;

public class PooledSambert {
    static class Factory extends BasePooledObjectFactory<SpeechSynthesizer> {
        @Override
        public SpeechSynthesizer create() { return new SpeechSynthesizer(); }

        @Override
        public PooledObject<SpeechSynthesizer> wrap(SpeechSynthesizer value) {
            return new DefaultPooledObject<>(value);
        }

        @Override
        public void destroyObject(PooledObject<SpeechSynthesizer> value) {
            value.getObject().getSyncApi().close(1000, "done");
        }
    }

    static void synthesize(GenericObjectPool<SpeechSynthesizer> pool, int index, String text)
            throws Exception {
        SpeechSynthesizer synthesizer = pool.borrowObject();
        boolean successful = false;
        CountDownLatch finished = new CountDownLatch(1);
        AtomicReference<Exception> error = new AtomicReference<>();
        try (FileOutputStream audio = new FileOutputStream("sambert_" + index + ".mp3")) {
            SpeechSynthesisParam param = SpeechSynthesisParam.builder()
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                .model("sambert-zhichu-v1")
                .text(text)
                .format(SpeechSynthesisAudioFormat.MP3)
                .enablePhonemeTimestamp(true)
                .enableWordTimestamp(true)
                .build();
            synthesizer.call(param, new ResultCallback<SpeechSynthesisResult>() {
                @Override
                public void onEvent(SpeechSynthesisResult result) {
                    if (result.getAudioFrame() != null) {
                        byte[] bytes = result.getAudioFrame().array();
                        try { audio.write(bytes); }
                        catch (Exception e) { error.compareAndSet(null, e); }
                    }
                }

                @Override
                public void onComplete() { finished.countDown(); }

                @Override
                public void onError(Exception e) {
                    error.compareAndSet(null, e);
                    finished.countDown();
                }
            });
            if (!finished.await(60, TimeUnit.SECONDS)) {
                throw new TimeoutException("Synthesis timed out");
            }
            if (error.get() != null) { throw error.get(); }
            successful = true;
        } finally {
            if (successful) { pool.returnObject(synthesizer); }
            else { pool.invalidateObject(synthesizer); }
        }
    }

    public static void main(String[] args) throws Exception {
        Constants.baseWebsocketApiUrl = "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference";
        int poolSize = Integer.parseInt(System.getenv().getOrDefault("SAMBERT_OBJECTPOOL_SIZE", "500"));
        GenericObjectPoolConfig<SpeechSynthesizer> config = new GenericObjectPoolConfig<>();
        config.setMaxTotal(poolSize);
        config.setMaxIdle(poolSize);
        config.setMaxWait(Duration.ofSeconds(60));
        ExecutorService executor = Executors.newFixedThreadPool(3);
        try (GenericObjectPool<SpeechSynthesizer> pool = new GenericObjectPool<>(new Factory(), config)) {
            String[] texts = {"床前明月光，疑似地上霜。", "举头望明月，低头思故乡。", "你好，欢迎使用阿里云百炼。"};
            List<Future<?>> futures = new ArrayList<>();
            for (int i = 0; i < texts.length; i++) {
                final int index = i;
                futures.add(executor.submit(() -> {
                    try { synthesize(pool, index, texts[index]); }
                    catch (Exception e) { throw new RuntimeException(e); }
                }));
            }
            Exception failure = null;
            for (Future<?> future : futures) {
                try { future.get(); }
                catch (ExecutionException e) { failure = e; }
            }
            if (failure != null) { throw failure; }
        } finally {
            executor.shutdownNow();
        }
    }
}

Production use

Resource management

1. Configure the API key and endpoint, then create a shared object pool for application tasks.
2. Borrow and configure an object. In Python, pass the model, voice, format, and callback to borrow_synthesizer. In the Qwen-Audio-TTS/CosyVoice Java example, call borrowObject, then call updateParamAndCallback(param, callback) every time to set the parameters and callback for that task.
3. Run synthesis. Qwen-Audio-TTS/CosyVoice objects support call or streaming_call in Python, and call or streamingCall in Java. Sambert uses call(param, callback) for one-way streaming synthesis.
4. After normal completion, return the object with return_synthesizer in Python or returnObject in Java. Do not directly return unfinished or failed objects: call the object's close() in Python, or discard it with invalidateObject in Java and let the factory close its connection.
5. After all tasks finish, release pool resources with shutdown() in Python or the pool's close() in Java.

Authentication and multiple API keys

• The API key is sent in the Authorization header only during the WebSocket handshake. Later run-task messages do not carry it. Changing a key in parameters or global configuration does not change an existing connection's identity, which can assign identity, quota usage, or charges differently from what you expect.
• In Python, set dashscope.api_key before creating the pool. Connections established during initialization retain the handshake key, and borrow_synthesizer has no API key parameter. SpeechSynthesizerObjectPool is a process-wide singleton, so isolate different keys in different processes.
• In Java, create a separate pool for each API key and use the same key throughout each pool. For Qwen-Audio-TTS/CosyVoice, call updateParamAndCallback each time an object is borrowed. This method does not reauthenticate an existing connection.

Warm-up and latency measurement

1. After startup or network recovery, run warm-up calls and increase concurrency gradually. Before performance testing, warm up at the target concurrency, for example for 1–2 minutes, and collect measurements once connections are being reused steadily.
2. The Python pool establishes connections during initialization. Java GenericObjectPool creates objects on demand and connects on the first synthesis call, so warming up Java connections requires actual synthesis requests.
3. The first call includes WebSocket connection overhead. Measure its first-audio latency and total task time separately from calls that reuse a connection. You can also use the real-time factor (RTF, total synthesis time / audio duration) to evaluate synthesis speed.
4. For Qwen-Audio-TTS, CosyVoice, and Sambert, connections close after 60 seconds of inactivity following task completion. Evaluate the next call after a long idle period as a new connection.

SDK-side first-audio latency includes connection establishment and network transmission, so it is not a measure of model inference time alone. For Qwen-Audio-TTS/CosyVoice, a Python object provides get_first_package_delay() for first-audio latency and get_last_request_id() for the request ID used to correlate request logs. Java objects provide getLastRequestId() for the request ID.

Troubleshooting

TCP connections keep increasing despite steady traffic

Frequently creating synthesis objects without closing old connections can continually increase connection counts and server resource usage. Check that objects are returned after task completion and unused connections are closed.

In Java, when maxIdle is less than maxTotal, returned objects above the idle capacity are destroyed. If the factory's destroyObject does not close the WebSocket, the connection may remain open after the object is destroyed. Set maxIdle equal to maxTotal to reduce repeated destruction and creation, and close connections whenever objects are destroyed.

Tasks incur extra waiting time or time out

At maxTotal, borrowing from a Java object pool waits for an available object. Check pool capacity, object returns, and connection cleanup to avoid extra time spent waiting for objects.

Tasks slow down after startup or network recovery

After startup or network recovery, run warm-up calls and increase concurrency gradually to avoid blocking on many simultaneous WebSocket handshakes.

The server returns Invalid action('run-task')! Please follow the protocol!

After a client error interrupts a task, the server may still consider the previous task active. Reusing that connection for a new run-task causes a protocol sequence error. Close the connection and discard the object, then establish a new connection for later tasks.

Calls spike despite steady application traffic

Mass reconnection after startup, network interruptions, or many server errors can cause tasks to accumulate. When connections recover, simultaneous submission of those tasks can create a spike and trigger account limits.

1. Check for network problems or many server errors before the spike, such as Requests rate limit exceeded, please try again later..
2. Control concurrency during startup and recovery. In Java, maxTotal limits the number of objects borrowed simultaneously from the same pool.
3. Check account quotas. To request an increase, follow the procedure in Rate limits.
4. Adjust server specifications or add servers for the actual workload.

All tasks slow down as concurrency grows

Check whether network bandwidth is saturated or concurrency is too high. Adjust concurrency or server capacity for the workload.