Sambert speech synthesis Python SDK
This topic describes the parameters and interfaces of the Sambert speech synthesis Python SDK.
ImportantAlibaba Cloud Model Studio has released a workspace-specific domain for the China (Beijing) region. The new dedicated domain delivers superior performance and higher stability for inference requests. We recommend migrating from dashscope.aliyuncs.com to {WorkspaceId}.cn-beijing.maas.aliyuncs.com.
Replace {WorkspaceId} with your actual Workspace ID. The existing domain remains fully functional.
User guide: For more information about model introductions and selection suggestions, see Real-time Speech Synthesis - CosyVoice/Sambert.
Online experience: Not supported.
Prerequisites
You have activated the service and obtained and configured an API key. To prevent security risks from code leaks, set the API key as an environment variable instead of hard-coding it in your code.
Getting started
The core resource (client.audio.tts) provides interfaces for non-streaming calls and one-way streaming calls. Choose the appropriate method based on your scenario:
- Non-streaming call: Submit the text. The server processes it and returns the complete speech synthesis result. The entire process is blocking, which means the client must wait for the server to finish before it can proceed. This method is suitable for short text synthesis.
- One-way streaming call: Send the text to the server in a single request and receive the speech synthesis result in real time. You cannot send the text in segments. This method is suitable for scenarios that require low latency.
Non-streaming call
Submit a single speech synthesis task for speech synthesis (intermediate results are not streamed) and retrieve the complete result at once.
Call the synthesize method of the core resource (client.audio.tts) to synthesize speech. You can use the synthesize method to set request parameters.
After the task is complete, the method returns the complete audio data (bytes). Timestamp information is available only through a streaming session. For more information, see Response results.
Click to view the complete example
The following example shows how to use the synchronous interface to call the Zhichu (sambert-zhichu-v1) model to synthesize the text "How is the weather today?". The synthesized audio is in the WAV format with a sample rate of 48 kHz and is saved to a file named output.wav.
# coding=utf-8
import os
import sys
import dashscope
# If you do not configure the API key as an environment variable, replace the following line with your API key: api_key="sk-xxx"
# The following configuration is for the China (Beijing) region. Replace "{WorkspaceId}" with your actual workspace ID when you call the API. The configuration varies by region.
client = dashscope.Client(
api_key=os.getenv("DASHSCOPE_API_KEY"),
workspace="{WorkspaceId}",
region="cn-beijing",
)
audio = client.audio.tts.synthesize(
'How is the weather today?',
model='sambert-zhichu-v1',
sample_rate=48000,
format='wav')
with open('output.wav', 'wb') as f:
f.write(audio)
print('SUCCESS: get audio data: %dbytes in output.wav' % (sys.getsizeof(audio)))
One-way streaming call
Submit a single speech synthesis task. Intermediate results are streamed back, and the synthesis results are retrieved by iterating events()/audio_chunks() of the streaming synthesis session.
- Call the
connectmethod of the core resource (client.audio.tts) to create a streaming synthesis session. The connection is automatically established when thewithcontext is entered. - Call the
send_textmethod of the session to send the text to be synthesized, and call thestopmethod to notify the server that the input is complete. - In the receiving thread, iterate
events()(audiobytesand JSON events, including timestamps) oraudio_chunks()(audio only) of the session to retrieve the synthesis results.
Click to view the complete example
The following example shows how to use the streaming interface to call the Zhichu (sambert-zhichu-v1) model. It synthesizes the text "How is the weather today?" into streaming audio in the default WAV format with a 48 kHz sample rate, and retrieves the corresponding timestamp.
# coding=utf-8
import os
import sys
import dashscope
# If you do not configure the API key as an environment variable, replace the following line with your API key: api_key="sk-xxx"
# The following configuration is for the China (Beijing) region. Replace "{WorkspaceId}" with your actual workspace ID when you call the API. The configuration varies by region.
client = dashscope.Client(
api_key=os.getenv("DASHSCOPE_API_KEY"),
workspace="{WorkspaceId}",
region="cn-beijing",
)
# Create a streaming synthesis session: the connection is automatically established when the with context is entered
with client.audio.tts.connect(
model='sambert-zhichu-v1',
sample_rate=48000,
parameters={'word_timestamp_enabled': True,
'phoneme_timestamp_enabled': True},
) as session:
session.send_text('How is the weather today?')
# Notify the server that the input is complete
session.stop()
# events() yields both audio bytes and server JSON events (including timestamps)
for event in session.events():
if isinstance(event, bytes):
print('audio result length:', sys.getsizeof(event))
else:
output = (event.get('payload') or {}).get('output') or {}
if output.get('timestamp') is not None:
print('timestamp result:', str(output.get('timestamp')))
Request parameters
You can set the request parameters in the synthesize, stream, and connect methods of the core resource (client.audio.tts). Named parameters are passed directly, and word_timestamp_enabled and phoneme_timestamp_enabled are passed in the parameters dictionary.
| Parameter | Type | Default value | Required | Description |
|---|---|---|---|---|
model | str | Yes | The name of the voice model to use for speech synthesis. For a complete list, see Model list. | |
text | str | Yes | The text to be synthesized. The text must be UTF-8 encoded and cannot be empty. Maximum number of characters: 10,000.
The SSML format is supported. For more information, see Introduction to SSML. | |
format | str | wav | No | The encoding format of the synthesized audio. The |
sample_rate | int | 16000 | No | The sample rate of the synthesized audio in Hz. We recommend that you use the model's default sample rate (see the Model list). If there is a mismatch, the service performs the necessary upsampling or downsampling. |
volume | int | 50 | No | The volume of the synthesized audio. Valid values: 0 to 100. |
rate | float | 1.0 | No | The speech rate of the synthesized audio. Valid values: 0.5 to 2.
|
pitch | float | 1.0 | No | The pitch of the synthesized audio. Valid values: 0.5 to 2. |
word_timestamp_enabled | bool | False | No | Specifies whether to enable word-level timestamps. The default value is false. |
phoneme_timestamp_enabled | bool | False | No | Specifies whether to generate phoneme-level timestamp information based on word-level timestamps ( |
Key interfaces
Core resource (client.audio.tts)
The speech synthesis capability is mounted on the client.audio.tts resource: synthesize returns the complete audio at once, stream yields the audio chunks, and connect creates a streaming synthesis session. The key interfaces are as follows:
| Interface/Method | Parameters | Return value | Description |
|---|---|---|---|
|
| The complete audio data ( | Starts a speech synthesis task and returns the complete audio data after the synthesis is complete. You can use the |
| Same as | The audio data yielded chunk by chunk ( | Starts a speech synthesis task, sends the text in a single request, and yields the audio chunks one by one. It yields only audio and does not expose per-event JSON metadata. To obtain timestamps, use the |
|
| The | Creates a streaming synthesis session. The connection is automatically established when the |
Event iteration (events()/audio_chunks())
When you make a one-way streaming call, the server returns key process information and data to the client as events. In the receiving thread, iterate events() or audio_chunks() of the session to process the information and data returned by the server.
Click to view the example
# events() yields both audio bytes and server JSON events (including timestamps)
for event in session.events():
if isinstance(event, bytes):
print('Received binary audio data:', event)
else:
output = (event.get('payload') or {}).get('output') or {}
if output.get('timestamp') is not None:
print('Received timestamp data:', str(output.get('timestamp')))
# Alternatively, iterate only the audio (the session still updates last_response and usage)
for chunk in session.audio_chunks():
print('Received binary audio data:', chunk)
| Interface/Method | Parameters | Return value | Description |
|---|---|---|---|
|
| None | Sends the text to be synthesized. Sending the text and receiving the results must be performed concurrently. |
| None | None | Notifies the server that the input is complete. Then, continue to consume |
| None |
| Yields items as the server returns the synthesis data: audio as |
| None |
| Iterates only the audio data. The session still updates its |
| None |
| The delay (in milliseconds) from sending the text to receiving the first audio data chunk. Read this property after the task is complete. |
Response results
Synthesis result (audio data and timestamps)
synthesize returns the complete audio bytes. In streaming calls, you receive the audio bytes chunk by chunk through stream or the audio_chunks() method of the session. Timestamp information is retrieved from the JSON events yielded by events() of the session (payload.output.timestamp).
| Interface/Method | Parameters | Return value | Description |
|---|---|---|---|
| None | The complete binary audio data | In non-streaming synthesis, obtains the complete binary audio data. |
| None | The current segment of synthesized binary audio data | In streaming synthesis, receives the synthesized audio frame data chunk by chunk. |
| None | The currently synthesized sentence's timestamp information | In streaming synthesis, obtains the timestamp information of the currently synthesized sentence from the JSON events yielded by |
| None | The timestamp information of all sentences | In streaming synthesis, accumulating the timestamps of the sentences yielded by |
Timestamp information
In the JSON events yielded by events() of the session, payload.output.timestamp is the timestamp information of the currently synthesized sentence. Accumulating the timestamps of all sentences gives the timestamp information of all sentences.
The following code provides an example of the timestamp information for a single sentence. words corresponds to word-level timestamp information, and phonemes corresponds to phoneme-level timestamp information:
{
"begin_time":0,
"end_time":1412,
"words":[
{
"text":"jin",
"begin_time":0,
"end_time":200,
"phonemes":[
{
"begin_time":0,
"end_time":82,
"text":"j_c",
"tone":1
},
{
"begin_time":82,
"end_time":200,
"text":"in_c",
"tone":1
}
]
}
]
}
The parameters are as follows:
Parameter | Type | Description |
|---|---|---|
begin_time | int | The start time of a sentence, word, or phoneme in milliseconds. |
end_time | int | The end time of a sentence, word, or phoneme in milliseconds. |
words | list | The word-level timestamp information. The |
text | str | The text information. |
phonemes | list | The phoneme-level timestamp information. The |
tone | str | The tone.
|
Error codes
If an API call fails and returns an error message, see Error codes to resolve the issue.
More examples
For more examples, see GitHub.
FAQ
For more information, see the QA on GitHub.
Model list
NoteThe default sample rate is the optimal sample rate for the current model. By default, the output is based on this sample rate. Downsampling and upsampling are also supported. For example, for the Zhimiao voice, the default sample rate is 16 kHz. You can downsample it to 8 kHz, but upsampling it to 48 kHz does not provide additional benefits.
| Timbre | Audio sample (Right-click to save the audio) | model parameter | Timestamp support | Scenarios | Features | Language | Default sample rate (Hz) |
|---|---|---|---|---|---|---|---|
Zhinan | sambert-zhinan-v1 | Yes | General scenarios | Advertising male voice | Chinese and English | 48 kHz | |
Zhiqi | sambert-zhiqi-v1 | Yes | General scenarios | Gentle female voice | Chinese and English | 48 kHz | |
Zhichu | sambert-zhichu-v1 | Yes | News broadcasting | A Bite of China male voice | Chinese and English | 48 kHz | |
Zhide | sambert-zhide-v1 | Yes | News broadcasting | News male voice | Chinese and English | 48 kHz | |
Zhijia | sambert-zhijia-v1 | Yes | News broadcasting | Standard female voice | Chinese and English | 48 kHz | |
Zhiru | sambert-zhiru-v1 | Yes | News broadcasting | News female voice | Chinese and English | 48 kHz | |
Zhiqian | sambert-zhiqian-v1 | Yes | Dubbing and news broadcasting | Information female voice | Chinese and English | 48 kHz | |
Zhixiang | sambert-zhixiang-v1 | Yes | Dubbing | Magnetic male voice | Chinese and English | 48 kHz | |
Zhiwei | sambert-zhiwei-v1 | Yes | Reading product introductions | Lolita female voice | Chinese and English | 48 kHz | |
Zhihao | sambert-zhihao-v1 | Yes | General scenarios | Consultation male voice | Chinese and English | 16 KB | |
Zhijing | sambert-zhijing-v1 | Yes | General scenarios | Strict female voice | Chinese and English | 16 k | |
Zhiming | sambert-zhiming-v1 | Yes | General scenarios | Humorous male voice | Chinese and English | 16 KB | |
Zhimo | sambert-zhimo-v1 | Yes | General scenarios | Emotional male voice | Chinese and English | 16 kHz | |
Zhina | sambert-zhina-v1 | Yes | General scenarios | Zhejiang Mandarin female voice | Chinese and English | 16 kHz | |
Zhishu | sambert-zhishu-v1 | Yes | General scenarios | Information male voice | Chinese and English | 16 KB | |
Zhisha | sambert-zhistella-v1 | Yes | General scenarios | Intellectual female voice | Chinese and English | 16 kHz | |
Zhiting | sambert-zhiting-v1 | Yes | General scenarios | Radio female voice | Chinese and English | 16 kHz | |
Zhixiao | sambert-zhixiao-v1 | Yes | General scenarios | Information female voice | Chinese and English | 16 kHz | |
Zhiya | sambert-zhiya-v1 | Yes | General scenarios | Strict female voice | Chinese and English | 16 kHz | |
Zhiye | sambert-zhiye-v1 | Yes | General scenarios | Young male voice | Chinese and English | 16 kHz | |
Zhiying | sambert-zhiying-v1 | Yes | General scenarios | Cute child voice | Chinese and English | 16 kHz | |
Zhiyuan | sambert-zhiyuan-v1 | Yes | General scenarios | Caring sister voice | Chinese and English | 16 kHz | |
Zhiyue | sambert-zhiyue-v1 | Yes | Customer service | Gentle female voice | Chinese and English | 16 kHz | |
Zhigui | sambert-zhigui-v1 | Yes | Reading product introductions | Livestreaming female voice | Chinese and English | 16 kHz | |
Zhishuo | sambert-zhishuo-v1 | Yes | Digital human | Natural male voice | Chinese and English | 16 kHz | |
Zhimiao (multi-emotional) | sambert-zhimiao-emo-v1 | Yes | Reading product introductions, digital humans, and livestreaming | Multi-emotional female voice | Chinese and English | 16 kHz | |
Zhimao | sambert-zhimao-v1 | Yes | Reading product introductions, dubbing, digital humans, and livestreaming | Livestreaming female voice | Chinese and English | 16 kHz | |
Zhilun | sambert-zhilun-v1 | Yes | Dubbing | Suspense narration | Chinese and English | 16 kHz | |
Zhifei | sambert-zhifei-v1 | Yes | Dubbing | Passionate narration | Chinese and English | 16 KB | |
Zhida | sambert-zhida-v1 | Yes | News broadcasting | Standard male voice | Chinese and English | 16 KB | |
Camila | sambert-camila-v1 | No | General scenarios | Spanish female voice | Spanish | 16 kHz | |
Perla | sambert-perla-v1 | No | General scenarios | Italian female voice | Italian | 16 k | |
Indah | sambert-indah-v1 | No | General scenarios | Indonesian female voice | Indonesian | 16 kHz | |
Clara | sambert-clara-v1 | No | General scenarios | French female voice | French | 16 kHz | |
Hanna | sambert-hanna-v1 | No | General scenarios | German female voice | German | 16 kHz | |
Beth | sambert-beth-v1 | Yes | General scenarios | Consultation female voice | American English | 16 kHz | |
Betty | sambert-betty-v1 | Yes | General scenarios | Customer service female voice | American English | 16 kHz | |
Cally | sambert-cally-v1 | Yes | General scenarios | Natural female voice | American English | 16 kHz | |
Cindy | sambert-cindy-v1 | Yes | General scenarios | Conversational female voice | American English | 16 kHz | |
Eva | sambert-eva-v1 | Yes | General scenarios | Companion female voice | American English | 16 KB | |
Donna | sambert-donna-v1 | Yes | General scenarios | Education female voice | American English | 16 kHz | |
Brian | sambert-brian-v1 | Yes | General scenarios | Customer service male voice | American English | 16 kHz | |
Waan | sambert-waan-v1 | No | General scenarios | Thai female voice | Thai | 16 kHz |