Python SDK

Updated at:

Use the Intelligent Speech Interaction Python SDK to synthesize text into speech and receive audio data, synthesis results, and subtitle information through callbacks.

Prerequisites

For service endpoints, voices, input text requirements, and other service settings, see Speech synthesis API reference.

Download and install the SDK

The SDK supports Python 3 only, not Python 2. The following examples use SDK version 1.1.0.

  1. Get the source code from GitHub, or download and extract the Python SDK source package.

  2. Install the setuptools build tool.

    python -m pip install setuptools
  3. In the SDK root directory, install the dependencies and the SDK.

    python -m pip install -r requirements.txt
    python -m pip install .

After installation, import the SDK in Python:

import nls

Code examples

Set the NLS_APP_KEY and NLS_TOKEN environment variables in your application's runtime environment to the project Appkey and NLS token. The following example synthesizes text with the ailun voice and saves the audio to output.pcm.

import os

import nls

URL = "wss://nls-gateway-cn-shanghai.aliyuncs.com/ws/v1"
APPKEY = os.environ["NLS_APP_KEY"]
TOKEN = os.environ["NLS_TOKEN"]
TEXT = "Hello, welcome to speech synthesis."

def on_metainfo(message, *args):
    print("Subtitle information:", message)

def on_completed(message, *args):
    print("Synthesis completed:", message)

def on_error(message, *args):
    print("Synthesis failed:", message)

def on_close(*args):
    print("Connection closed.")

nls.enableTrace(False)

with open("output.pcm", "wb") as audio_file:
    synthesizer = nls.NlsSpeechSynthesizer(
        url=URL,
        token=TOKEN,
        appkey=APPKEY,
        on_metainfo=on_metainfo,
        on_data=lambda data, *args: audio_file.write(data),
        on_completed=on_completed,
        on_error=on_error,
        on_close=on_close,
    )
    try:
        synthesizer.start(TEXT, voice="ailun")
    finally:
        synthesizer.shutdown()

For low-latency playback, play audio as it arrives in the on_data callback.

Note

For access over a VPC from an ECS instance in the China (Shanghai) region, set URL to ws://nls-gateway-cn-shanghai-internal.aliyuncs.com:80/ws/v1.

Note

To specify an emotion for a multi-emotion voice, use the SSML <emotion> tag in text. This tag applies only to voices that support multiple emotions. Requests using the tag with other voices return Illegal ssml text. For usage details, see Introduction to SSML.

Receive subtitles and timestamps

When using a voice that supports timestamps, replace the start call in the basic example with the following code. Subtitle information is returned through the on_metainfo callback.

synthesizer.start(TEXT, voice="ailun", ex={"enable_subtitle": True})

For phoneme-level timestamps, also pass enable_phoneme_timestamp=True:

synthesizer.start(
    TEXT,
    voice="ailun",
    ex={"enable_subtitle": True, "enable_phoneme_timestamp": True},
)

For supported voices and timestamp fields, see Speech synthesis timestamps.

Key interfaces

Use the nls.NlsSpeechSynthesizer class for speech synthesis.

Initialization (__init__)

Create a synthesizer and configure its service endpoint, credentials, and callbacks.

Parameter

Type

Description

url

str

WebSocket service endpoint. The SDK default is wss://nls-gateway.cn-shanghai.aliyuncs.com/ws/v1.

token

str

The NLS token used for authentication. Required.

appkey

str

The project Appkey. Required.

long_tts

bool

Whether to use real-time long-text speech synthesis. Set to True for long-text synthesis or False for short-text synthesis. Default: False.

on_metainfo

Callable

Called when a MetaInfo event arrives. The first argument is a JSON string containing information such as subtitles, followed by the unpacked callback_args. The returned content depends on the request configuration and voice capabilities.

on_data

Callable

Called when audio data arrives. The first argument is audio data of type bytes, in the format specified by aformat in start(), followed by the unpacked callback_args.

on_completed

Callable

Called when a SynthesisCompleted event arrives. The first argument is a JSON string for the completion event, followed by the unpacked callback_args.

on_error

Callable

Called when a TaskFailed event arrives from the service. The first argument is a JSON string for the error event, followed by the unpacked callback_args.

on_close

Callable

Called when the connection closes. Receives the unpacked callback_args.

callback_args

list

Custom callback arguments. Defaults to an empty list. Elements are unpacked using *callback_args; for example, ["a", "b"] supplies two positional arguments.

If a callback is not configured, the corresponding user callback is not executed. The initialization method has no return value.

start

Start speech synthesis. By default, the call waits for synthesis to finish. With wait_complete=False, it returns after the connection is established, while audio data and events continue to arrive through callbacks.

Parameter

Type

Description

text

str

The text to synthesize.

voice

str

The voice to use. Default: xiaoyun.

aformat

str

The output audio format. Valid values: pcm, wav, and mp3. Default: pcm.

sample_rate

int

The sample rate of the synthesized audio, in Hz. Default: 16000.

volume

int

The volume, from 0 to 100. Default: 50.

speech_rate

int

The speech rate, from -500 to 500. Default: 0.

pitch_rate

int

The pitch, from -500 to 500. Default: 0.

wait_complete

bool

Whether to wait for synthesis to complete. Default: True. When set to False, the application must keep the connection and audio-receiving resources available until synthesis ends.

start_timeout

int

The timeout for waiting for the connection to be established, in seconds. Default: 10.

completed_timeout

int

The timeout for waiting for synthesis to complete, in seconds. Default: 60. Applies when wait_complete=True.

ex

dict

Additional request parameters. Default: None. The dictionary is merged into the request payload. Examples include enable_subtitle and enable_phoneme_timestamp. Parameters and their conditions depend on the service and voice.

start() has no return value. Synthesis completion invokes on_completed, and a service-side task failure invokes on_error. Invalid parameters or a timeout may raise exceptions such as InvalidParameter, StartTimeoutException, or CompleteTimeoutException.

shutdown

Close the connection for the current request. Any unfinished synthesis ends when the connection closes. After start() creates the connection, shutdown() can be called repeatedly.

This method takes no parameters and has no return value.

Multithreading and concurrency

In CPython with the global interpreter lock (GIL) enabled, only one thread executes Python code at a time. Some extension libraries release the GIL. Threads can handle multiple I/O-bound tasks. For CPU-bound tasks that need multiple CPU cores, use multiprocessing or concurrent.futures.ProcessPoolExecutor.

Too many threads in a single interpreter increase context-switching overhead and may cause SDK runtime errors. We recommend using no more than 200 threads per interpreter. For more threads, use multiple processes or start multiple interpreters.

For concurrent synthesis, create a separate synthesizer and use a different audio output file for each task.