Sambert speech synthesis Python SDK-Alibaba Cloud Model Studio(Model Studio)-阿里云帮助中心

This topic describes the parameters and interfaces of the Sambert speech synthesis Python SDK.

Important

Alibaba Cloud Model Studio has released a workspace-specific domain for the China (Beijing) region. The new dedicated domain delivers superior performance and higher stability for inference requests. We recommend migrating from dashscope.aliyuncs.com to {WorkspaceId}.cn-beijing.maas.aliyuncs.com.

Replace {WorkspaceId} with your actual Workspace ID. The existing domain remains fully functional.

User guide: For more information about model introductions and selection suggestions, see Real-time Speech Synthesis - CosyVoice/Sambert.

Online experience: Not supported.

Prerequisites

You have activated the service and obtained and configured an API key. To prevent security risks from code leaks, set the API key as an environment variable instead of hard-coding it in your code.
Install the latest version of the DashScope SDK.

Getting started

The SpeechSynthesizer class provides interfaces for synchronous and streaming calls. Choose the appropriate method based on your scenario:

Synchronous call: Submit text in a single request. The server then processes the text and returns the complete synthesized audio. This is a blocking process, which means the client must wait for the server to finish before it can proceed. This method is suitable for short text synthesis.
Streaming call: Send text to the server in a single request. The synthesized audio is then streamed back in real time. You cannot send the text in segments. This method is suitable for scenarios that require low latency.

Non-streaming call

Submit a single speech synthesis task and retrieve the complete result at once. You do not need to use a callback because intermediate results are not streamed.

To make a synchronous call, use the call method of the SpeechSynthesizer class. You can use the call method to set request parameters. Ensure that you do not set the callback parameter.

After the task is complete, the method returns the audio data and timestamp information (SpeechSynthesisResult).

Click to view the complete example

The following example shows how to use the synchronous interface to call the Zhichu (sambert-zhichu-v1) model to synthesize the text "How is the weather today?". The synthesized audio is in the WAV format with a sample rate of 48 kHz and is saved to a file named output.wav.

# coding=utf-8
import sys
import dashscope
from dashscope.audio.tts import SpeechSynthesizer
# If you do not configure the API key as an environment variable, replace apiKey with your API key.
# dashscope.api_key = "apiKey"
# Replace "{WorkspaceId}" with your workspace ID.
dashscope.base_websocket_api_url='wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference'
result = SpeechSynthesizer.call(model='sambert-zhichu-v1',
                                text='How is the weather today?',
                                sample_rate=48000,
                                format='wav')
if result.get_audio_data() is not None:
    with open('output.wav', 'wb') as f:
        f.write(result.get_audio_data())
    print('SUCCESS: get audio data: %dbytes in output.wav' %
          (sys.getsizeof(result.get_audio_data())))
else:
    print('ERROR: response is %s' % (result.get_response()))

Streaming call

Submit a single speech synthesis task. The results are streamed back through a callback method in the ResultCallback class.

Instantiate ResultCallback.
Call the call method of the SpeechSynthesizer class to synthesize speech. You can use the call method to set request parameters. Ensure that you set the callback parameter.

Click to view the complete example

The following example shows how to use the streaming interface to call the Zhichu (sambert-zhichu-v1) model. It synthesizes the text "How is the weather today?" into streaming audio in the default WAV format with a 48 kHz sample rate, and retrieves the corresponding timestamp.

# coding=utf-8

import sys
import dashscope
from dashscope.api_entities.dashscope_response import SpeechSynthesisResponse
from dashscope.audio.tts import ResultCallback, SpeechSynthesizer, SpeechSynthesisResult

# If you do not configure the API key as an environment variable, replace apiKey with your API key.
# dashscope.api_key = "apiKey"
# Replace "{WorkspaceId}" with your workspace ID.
dashscope.base_websocket_api_url='wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference'

class Callback(ResultCallback):
    def on_open(self):
        print('Speech synthesizer is opened.')

    def on_complete(self):
        print('Speech synthesizer is completed.')

    def on_error(self, response: SpeechSynthesisResponse):
        print('Speech synthesizer failed, response is %s' % (str(response)))

    def on_close(self):
        print('Speech synthesizer is closed.')

    def on_event(self, result: SpeechSynthesisResult):
        if result.get_audio_frame() is not None:
            print('audio result length:', sys.getsizeof(result.get_audio_frame()))

        if result.get_timestamp() is not None:
            print('timestamp result:', str(result.get_timestamp()))

callback = Callback()
SpeechSynthesizer.call(model='sambert-zhichu-v1',
                       text='How is the weather today?',
                       sample_rate=48000,
                       callback=callback,
                       word_timestamp_enabled=True,
                       phoneme_timestamp_enabled=True)

Request parameters

You can set the request parameters in the call method of the SpeechSynthesizer class.

Parameter	Type	Default value	Required	Description
model	str	-	Yes	The name of the voice model to use for speech synthesis. For a complete list, see Model list.
text	str	-	Yes	The text to be synthesized. The text must be UTF-8 encoded and cannot be empty. Maximum number of characters: 10,000. Character calculation rule: One Chinese character, one English letter, one punctuation mark, or one space between sentences is counted as one character. The SSML format is supported. For more information, see Introduction to SSML.
format	str	wav	No	The encoding format of the synthesized audio. The `pcm`, `wav`, and `mp3` formats are supported.
sample_rate	int	16000	No	The sample rate of the synthesized audio in Hz. We recommend that you use the model's default sample rate (see the Model list). If there is a mismatch, the service performs the necessary upsampling or downsampling.
volume	int	50	No	The volume of the synthesized audio. Valid values: 0 to 100.
rate	float	1.0	No	The speech rate of the synthesized audio. Valid values: 0.5 to 2. 0.5: indicates 0.5 times the default speech rate. 1: indicates the default speech rate. The default speech rate is the synthesis rate of the default output of the model. The speech rate may vary slightly depending on the speaker. The rate is about four characters per second. 2: indicates twice the default speech rate.
pitch	float	1.0	No	The pitch of the synthesized audio. Valid values: 0.5 to 2.
word_timestamp_enabled	bool	False	No	Specifies whether to enable word-level timestamps. The default value is false.
phoneme_timestamp_enabled	bool	False	No	Specifies whether to generate phoneme-level timestamp information based on word-level timestamps (`word_timestamp_enabled`). The default value is false.
callback	ResultCallback	-	No	If you set the callback parameter, the streaming call mode is used. If the callback parameter is not set, the invocation runs in non-streaming mode. For the callback implementation, see Callback interface (ResultCallback).

Key interfaces

`SpeechSynthesizer` class

You can import the SpeechSynthesizer class with from dashscope.audio.tts import SpeechSynthesizer. The key methods are as follows:

Interface/Method

Parameters

Return value

Description

@classmethod
def call(cls,
         model: str,
         text: str,
         callback: ResultCallback = None,
         workspace: str = None,
         **kwargs) -> SpeechSynthesisResult:

model: The name of the model. For more information, see Model list.
text: The text to be synthesized.
callback: Callback interface (ResultCallback).
workspace: The DashScope workspace ID. You can ignore this parameter.
kwargs: Request parameters, such as format.

The synthesis result is SpeechSynthesisResult. You must handle this result for non-streaming invocations, but not for asynchronous invocations.

Starts a speech synthesis task. The behavior depends on whether you pass the callback parameter:

If you do not pass the callback parameter, the call function returns all speech synthesis results after the synthesis is complete.
If you pass the callback parameter, the server calls the corresponding function in callback to stream the speech synthesis results during the synthesis process.

You can use the call method to set the request parameters.

Callback interface (`ResultCallback`)

When you make a streaming call, the server returns key information and data to the client through a callback. You must implement a callback method to process the information and data that is returned by the server.

Click to view the example

class Callback(ResultCallback):
    def on_open(self):
        print('Successfully connected to the server')

    def on_complete(self):
        print('Task completed')

    def on_error(self, response: SpeechSynthesisResponse):
        print('Error: %s' % (str(response)))

    def on_close(self):
        print('The connection to the server is closed.')

    def on_event(self, result: SpeechSynthesisResult):
        if result.get_audio_frame() is not None:
            print('Received binary audio data:', result.get_audio_frame())

        if result.get_timestamp() is not None:
            print('Received timestamp data:', str(result.get_timestamp()))

callback = Callback()

Interface/Method	Parameters	Return value	Description
`def on_open(self) -> None`	None	None	This method is called immediately after a connection is established with the service.
`def on_event(self, result: SpeechSynthesisResult) -> None`	`result`: Audio data and timestamp information (SpeechSynthesisResult)	None	This method is called when the server returns synthetic data.
`def on_complete(self) -> None`	None	None	This method is called after all synthetic data is returned.
`def on_error(self, response: SpeechSynthesisResponse)`	`response`: Exception information	None	This method is called when an exception occurs during the call or the service returns an error.
`def on_close(self) -> None`	None	None	This method is called after the service closes the connection.

Response results

Audio data and timestamp information (`SpeechSynthesisResult`)

SpeechSynthesisResult encapsulates the speech synthesis result. The commonly used methods are get_audio_frame, get_timestamp, get_audio_data, and get_timestamps.

Interface/Method	Parameters	Return value	Description
`def get_audio_frame(self) -> bytes`	None	The current synthesized binary audio data segment	In streaming synthesis, obtains the current synthesized audio frame data. Important This function must be used in the `event` callback method during streaming synthesis.
`def get_timestamp(self) -> Dict[str, str]`	None	The currently synthesized sentence's timestamp information	In streaming synthesis, obtains the timestamp information corresponding to the currently synthesized sentence. Important This function must be used in the `event` callback method during streaming synthesis.
`def get_audio_data(self) -> bytes`	None	The complete binary audio data	Obtains the complete binary audio data.
`def get_timestamps(self) -> List[Dict[str, str]]`	None	Sentence-level timestamp information	Obtains the timestamp information corresponding to all sentences.

Timestamp information

The get_timestamp method of SpeechSynthesisResult retrieves the timestamp information for the currently synthesized sentence. The get_timestamps method retrieves the timestamp information for all sentences.

The following code provides an example of the timestamp information for a single sentence. words corresponds to word-level timestamp information, and phonemes corresponds to phoneme-level timestamp information:

{
    "begin_time":0,
    "end_time":1412,
    "words":[
        {
            "text":"jin",
            "begin_time":0,
            "end_time":200,
            "phonemes":[
                {
                    "begin_time":0,
                    "end_time":82,
                    "text":"j_c",
                    "tone":1
                },
                {
                    "begin_time":82,
                    "end_time":200,
                    "text":"in_c",
                    "tone":1
                }
            ]
        }
    ]
}

The parameters are as follows:

Parameter	Type	Description
begin_time	int	The start time of a sentence, word, or phoneme in milliseconds.
end_time	int	The end time of a sentence, word, or phoneme in milliseconds.
words	list	The word-level timestamp information. The `word_timestamp_enabled` parameter in the request must also be set to true.
text	str	The text information.
phonemes	list	The phoneme-level timestamp information. The `phoneme_timestamp_enabled` parameter in the request must also be set to true.
tone	str	The tone. In English, 0, 1, and 2 represent unstressed, primary stress, and secondary stress, respectively. In Pinyin, 1, 2, 3, 4, and 5 represent the first, second, third, fourth, and neutral tones, respectively.

Error codes

If an API call fails and returns an error message, see Error codes to resolve the issue.

More examples

For more examples, see GitHub.

FAQ

For more information, see the QA on GitHub.

Model list

Note

The default sample rate is the optimal sample rate for the current model. By default, the output is based on this sample rate. Downsampling and upsampling are also supported. For example, for the Zhimiao voice, the default sample rate is 16 kHz. You can downsample it to 8 kHz, but upsampling it to 48 kHz does not provide additional benefits.

Timbre	Audio sample (Right-click to save the audio)	model parameter	Timestamp support	Scenarios	Features	Language	Default sample rate (Hz)
Zhinan		sambert-zhinan-v1	Yes	General scenarios	Advertising male voice	Chinese and English	48 kHz
Zhiqi		sambert-zhiqi-v1	Yes	General scenarios	Gentle female voice	Chinese and English	48 kHz
Zhichu		sambert-zhichu-v1	Yes	News broadcasting	A Bite of China male voice	Chinese and English	48 kHz
Zhide		sambert-zhide-v1	Yes	News broadcasting	News male voice	Chinese and English	48 kHz
Zhijia		sambert-zhijia-v1	Yes	News broadcasting	Standard female voice	Chinese and English	48 kHz
Zhiru		sambert-zhiru-v1	Yes	News broadcasting	News female voice	Chinese and English	48 kHz
Zhiqian		sambert-zhiqian-v1	Yes	Dubbing and news broadcasting	Information female voice	Chinese and English	48 kHz
Zhixiang		sambert-zhixiang-v1	Yes	Dubbing	Magnetic male voice	Chinese and English	48 kHz
Zhiwei		sambert-zhiwei-v1	Yes	Reading product introductions	Lolita female voice	Chinese and English	48 kHz
Zhihao		sambert-zhihao-v1	Yes	General scenarios	Consultation male voice	Chinese and English	16 KB
Zhijing		sambert-zhijing-v1	Yes	General scenarios	Strict female voice	Chinese and English	16 k
Zhiming		sambert-zhiming-v1	Yes	General scenarios	Humorous male voice	Chinese and English	16 KB
Zhimo		sambert-zhimo-v1	Yes	General scenarios	Emotional male voice	Chinese and English	16 kHz
Zhina		sambert-zhina-v1	Yes	General scenarios	Zhejiang Mandarin female voice	Chinese and English	16 kHz
Zhishu		sambert-zhishu-v1	Yes	General scenarios	Information male voice	Chinese and English	16 KB
Zhisha		sambert-zhistella-v1	Yes	General scenarios	Intellectual female voice	Chinese and English	16 kHz
Zhiting		sambert-zhiting-v1	Yes	General scenarios	Radio female voice	Chinese and English	16 kHz
Zhixiao		sambert-zhixiao-v1	Yes	General scenarios	Information female voice	Chinese and English	16 kHz
Zhiya		sambert-zhiya-v1	Yes	General scenarios	Strict female voice	Chinese and English	16 kHz
Zhiye		sambert-zhiye-v1	Yes	General scenarios	Young male voice	Chinese and English	16 kHz
Zhiying		sambert-zhiying-v1	Yes	General scenarios	Cute child voice	Chinese and English	16 kHz
Zhiyuan		sambert-zhiyuan-v1	Yes	General scenarios	Caring sister voice	Chinese and English	16 kHz
Zhiyue		sambert-zhiyue-v1	Yes	Customer service	Gentle female voice	Chinese and English	16 kHz
Zhigui		sambert-zhigui-v1	Yes	Reading product introductions	Livestreaming female voice	Chinese and English	16 kHz
Zhishuo		sambert-zhishuo-v1	Yes	Digital human	Natural male voice	Chinese and English	16 kHz
Zhimiao (multi-emotional)		sambert-zhimiao-emo-v1	Yes	Reading product introductions, digital humans, and livestreaming	Multi-emotional female voice	Chinese and English	16 kHz
Zhimao		sambert-zhimao-v1	Yes	Reading product introductions, dubbing, digital humans, and livestreaming	Livestreaming female voice	Chinese and English	16 kHz
Zhilun		sambert-zhilun-v1	Yes	Dubbing	Suspense narration	Chinese and English	16 kHz
Zhifei		sambert-zhifei-v1	Yes	Dubbing	Passionate narration	Chinese and English	16 KB
Zhida		sambert-zhida-v1	Yes	News broadcasting	Standard male voice	Chinese and English	16 KB
Camila		sambert-camila-v1	No	General scenarios	Spanish female voice	Spanish	16 kHz
Perla		sambert-perla-v1	No	General scenarios	Italian female voice	Italian	16 k
Indah		sambert-indah-v1	No	General scenarios	Indonesian female voice	Indonesian	16 kHz
Clara		sambert-clara-v1	No	General scenarios	French female voice	French	16 kHz
Hanna		sambert-hanna-v1	No	General scenarios	German female voice	German	16 kHz
Beth		sambert-beth-v1	Yes	General scenarios	Consultation female voice	American English	16 kHz
Betty		sambert-betty-v1	Yes	General scenarios	Customer service female voice	American English	16 kHz
Cally		sambert-cally-v1	Yes	General scenarios	Natural female voice	American English	16 kHz
Cindy		sambert-cindy-v1	Yes	General scenarios	Conversational female voice	American English	16 kHz
Eva		sambert-eva-v1	Yes	General scenarios	Companion female voice	American English	16 KB
Donna		sambert-donna-v1	Yes	General scenarios	Education female voice	American English	16 kHz
Brian		sambert-brian-v1	Yes	General scenarios	Customer service male voice	American English	16 kHz
Waan		sambert-waan-v1	No	General scenarios	Thai female voice	Thai	16 kHz