Python SDK
Use the Intelligent Speech Interaction Python SDK to synthesize text into speech and receive audio data, synthesis results, and subtitle information through callbacks.
Prerequisites
Create a project and obtain its Appkey. For instructions, see Manage projects.
Obtain a valid NLS token. For instructions, see Obtain a token.
For service endpoints, voices, input text requirements, and other service settings, see Speech synthesis API reference.
Download and install the SDK
The SDK supports Python 3 only, not Python 2. The following examples use SDK version 1.1.0.
Get the source code from GitHub, or download and extract the Python SDK source package.
-
Install the
setuptoolsbuild tool.python -m pip install setuptools -
In the SDK root directory, install the dependencies and the SDK.
python -m pip install -r requirements.txt python -m pip install .
After installation, import the SDK in Python:
import nls
Code examples
Set the NLS_APP_KEY and NLS_TOKEN environment variables in your application's runtime environment to the project Appkey and NLS token. The following example synthesizes text with the ailun voice and saves the audio to output.pcm.
import os
import nls
URL = "wss://nls-gateway-cn-shanghai.aliyuncs.com/ws/v1"
APPKEY = os.environ["NLS_APP_KEY"]
TOKEN = os.environ["NLS_TOKEN"]
TEXT = "Hello, welcome to speech synthesis."
def on_metainfo(message, *args):
print("Subtitle information:", message)
def on_completed(message, *args):
print("Synthesis completed:", message)
def on_error(message, *args):
print("Synthesis failed:", message)
def on_close(*args):
print("Connection closed.")
nls.enableTrace(False)
with open("output.pcm", "wb") as audio_file:
synthesizer = nls.NlsSpeechSynthesizer(
url=URL,
token=TOKEN,
appkey=APPKEY,
on_metainfo=on_metainfo,
on_data=lambda data, *args: audio_file.write(data),
on_completed=on_completed,
on_error=on_error,
on_close=on_close,
)
try:
synthesizer.start(TEXT, voice="ailun")
finally:
synthesizer.shutdown()
For low-latency playback, play audio as it arrives in the on_data callback.
For access over a VPC from an ECS instance in the China (Shanghai) region, set URL to ws://nls-gateway-cn-shanghai-internal.aliyuncs.com:80/ws/v1.
To specify an emotion for a multi-emotion voice, use the SSML <emotion> tag in text. This tag applies only to voices that support multiple emotions. Requests using the tag with other voices return Illegal ssml text. For usage details, see Introduction to SSML.
Receive subtitles and timestamps
When using a voice that supports timestamps, replace the start call in the basic example with the following code. Subtitle information is returned through the on_metainfo callback.
synthesizer.start(TEXT, voice="ailun", ex={"enable_subtitle": True})
For phoneme-level timestamps, also pass enable_phoneme_timestamp=True:
synthesizer.start(
TEXT,
voice="ailun",
ex={"enable_subtitle": True, "enable_phoneme_timestamp": True},
)
For supported voices and timestamp fields, see Speech synthesis timestamps.
Key interfaces
Use the nls.NlsSpeechSynthesizer class for speech synthesis.
Initialization (__init__)
Create a synthesizer and configure its service endpoint, credentials, and callbacks.
|
Parameter |
Type |
Description |
|
|
str |
WebSocket service endpoint. The SDK default is |
|
|
str |
The NLS token used for authentication. Required. |
|
|
str |
The project Appkey. Required. |
|
|
bool |
Whether to use real-time long-text speech synthesis. Set to |
|
|
Callable |
Called when a |
|
|
Callable |
Called when audio data arrives. The first argument is audio data of type |
|
|
Callable |
Called when a |
|
|
Callable |
Called when a |
|
|
Callable |
Called when the connection closes. Receives the unpacked |
|
|
list |
Custom callback arguments. Defaults to an empty list. Elements are unpacked using |
If a callback is not configured, the corresponding user callback is not executed. The initialization method has no return value.
start
Start speech synthesis. By default, the call waits for synthesis to finish. With wait_complete=False, it returns after the connection is established, while audio data and events continue to arrive through callbacks.
|
Parameter |
Type |
Description |
|
|
str |
The text to synthesize. |
|
|
str |
The voice to use. Default: |
|
|
str |
The output audio format. Valid values: |
|
|
int |
The sample rate of the synthesized audio, in Hz. Default: |
|
|
int |
The volume, from 0 to 100. Default: |
|
|
int |
The speech rate, from -500 to 500. Default: |
|
|
int |
The pitch, from -500 to 500. Default: |
|
|
bool |
Whether to wait for synthesis to complete. Default: |
|
|
int |
The timeout for waiting for the connection to be established, in seconds. Default: |
|
|
int |
The timeout for waiting for synthesis to complete, in seconds. Default: |
|
|
dict |
Additional request parameters. Default: |
start() has no return value. Synthesis completion invokes on_completed, and a service-side task failure invokes on_error. Invalid parameters or a timeout may raise exceptions such as InvalidParameter, StartTimeoutException, or CompleteTimeoutException.
shutdown
Close the connection for the current request. Any unfinished synthesis ends when the connection closes. After start() creates the connection, shutdown() can be called repeatedly.
This method takes no parameters and has no return value.
Multithreading and concurrency
In CPython with the global interpreter lock (GIL) enabled, only one thread executes Python code at a time. Some extension libraries release the GIL. Threads can handle multiple I/O-bound tasks. For CPU-bound tasks that need multiple CPU cores, use multiprocessing or concurrent.futures.ProcessPoolExecutor.
Too many threads in a single interpreter increase context-switching overhead and may cause SDK runtime errors. We recommend using no more than 200 threads per interpreter. For more threads, use multiple processes or start multiple interpreters.
For concurrent synthesis, create a separate synthesizer and use a different audio output file for each task.