Sambert Speech Synthesis iOS SDK

更新时间:
复制 MD 格式

This document provides a detailed guide on using the Sambert speech synthesis software development kit (SDK) for iOS to convert text into high-quality, expressive speech.

Important

Model Studio has released a workspace-specific domain for the China (Beijing) region: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com. The new dedicated domain delivers superior performance and higher stability for inference requests. We recommend migrating from https://dashscope.aliyuncs.com to the new domain.

{WorkspaceId} is your workspace ID, which can be found on the Workspace Details page in the Model Studio console. The existing domain remains fully functional.

User guide: For more information about the available models and selection suggestions, see Sambert speech synthesis.

Online experience: Not supported.

Getting started

  1. Obtain an API key: Obtain an API key

    Note

    To grant temporary access permissions to third-party applications or users, or to strictly control access to sensitive data and risky operations such as deletion, use a temporary API key. A temporary API key is valid for 60 seconds and you must obtain a new one after it expires.

  2. Download the SDK and run the sample code:

    • Download the latest SDK package.

    • Decompress the ZIP package and add nuisdk.framework to your project.

    • In Build Phases → Link Binary With Libraries, add nuisdk.framework.

    • In General → Frameworks, Libraries, and Embedded Content, set nuisdk.framework to Embed & Sign.

    • Open the sample project in Xcode. The sample code is in the DashSambertTTSViewController class. You can replace the API key to test the feature.

Call procedure

  1. Initialize the SDK.

  2. Set the required parameters. Use the parameters parameter of the nui_tts_initialize method to set connection and control parameters. Use the nui_tts_set_param method to set speech synthesis effect parameters.

  3. Call <a baseurl="t3161921_v1_0_0.xdita" data-node="6176237" data-root="85177" data-tag="xref" href="#8fe6ea298apzu" id="2189d751f7k5z">nui_tts_play</a> to start speech synthesis.

  4. In the onNuiTtsUserdataCallback callback, you can obtain the audio data and use stream playback. To save the audio locally, you can write the audio to the same file in append mode until the synthesis is complete.

  5. After the task is complete, call <a baseurl="t3161921_v1_0_0.xdita" data-node="6176237" data-root="85177" data-tag="xref" href="#6c2931e9ae3eq" id="c27bf56de7sde">nui_tts_release</a> to release the SDK resources.

Request parameters

Connection and control parameters

You can configure the parameters by passing a JSON string in the parameters parameter of the nui_tts_initialize method.

  • Parameter example: The following code provides an example of a JSON string. Not all parameters are listed. You can add parameters as needed during encoding:

    {
        "url": "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference",
        "apikey": "st-****",
        "device_id": "my_device_id"
    }
  • Parameter descriptions

    Parameter

    Type

    Required

    Description

    url

    String

    Yes

    The service endpoint. Set to wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference. Replace WorkspaceId with your actual workspace ID.

    apikey

    String

    Yes

    The API key. Use a temporary API key for higher security and a shorter validity period. This reduces the risk of long-term key leakage.

    device_id

    String

    Yes

    A unique string that identifies the end user. You can set this to the in-app user ID or a unique device identifier generated by the client. This ID is mainly used for log tracking and troubleshooting.

    debug_path

    String

    No

    The storage path for log files.

    This parameter takes effect only when save_log is set to YES in the nui_tts_initialize interface call. You must set a log file path. Otherwise, an error is reported.

    A maximum of two log files are retained locally.

    max_log_file_size

    int

    No

    The maximum size of a log file in bytes.

    This parameter takes effect only when save_log is set to YES in the nui_tts_initialize interface call.

    Default value: 104857600 (100 × 1024 × 1024 bytes, which is 100 MiB).

    log_track_level

    int

    No

    The filtering level for log content sent through the log callback (onNuiTtsLogTrackCallback).

    Default value: 2.

    Valid values:

    • 0: LOG_LEVEL_VERBOSE

    • 1: LOG_LEVEL_DEBUG

    • 2: LOG_LEVEL_INFO

    • 3: LOG_LEVEL_WARNING

    • 4: LOG_LEVEL_ERROR

    • 5: LOG_LEVEL_NONE (disables this feature)

    Note: log_track_level and level (set through the nui_tts_initialize interface) together determine the final logs for the callback. A log is sent to the callback only if its level value is greater than or equal to both the log_track_level and level values. For example, if log_track_level is set to 2 (INFO) and level is set to 3 (WARNING), only logs at the WARNING level and higher (value >= 3) are sent to the callback.

Speech synthesis effect parameters

You can set these parameters using the nui_tts_set_param method.

Parameter

Type

Required

Description

model

String

Yes

The speech synthesis model.

format

String

No

The audio coding format. Supported formats are pcm, wav, and mp3.

Default value: pcm.

volume

String

No

The volume.

Default value: 50.

Valid values: [0, 100]. A value of 50 represents the standard volume. The volume has a linear relationship with this value. A value of 0 means mute, and 100 means the maximum volume.

sample_rate

String

No

The sample rate in Hz.

Default value: The default sample rate of the model.

Use the default value of the model. If the sample rate does not match, the server-side performs resampling.

rate

String

No

The speech rate.

Default value: 1.0.

Valid values: [0.5, 2.0]. A value of 1.0 is the standard speech rate. A value less than 1.0 slows down the speech, and a value greater than 1.0 speeds it up.

pitch

String

No

The pitch. This value is a multiplier for pitch adjustment, but its relationship with the perceived pitch change is not strictly linear or logarithmic. Test to find a suitable value.

Default value: 1.0.

Valid values: [0.5, 2.0]. A value of 1.0 is the natural pitch for the timbre. A value greater than 1.0 raises the pitch, and a value less than 1.0 lowers it.

word_timestamp_enabled

String

No

Specifies whether to enable word-level timestamps.

Default value: 0.

Valid values:

  • 1: Enable.

  • 0: Disable.

phoneme_timestamp_enabled

String

No

Specifies whether to enable phoneme-level timestamps. This parameter takes effect only when word_timestamp_enabled is set to 1 (enabled).

Default value: 0.

Valid values:

  • 1: Enable.

  • 0: Disable.

enable_audio_decoder

String

No

Specifies whether to enable the built-in audio decoder.

Default value: 0.

Valid values:

  • 1: Enable. If format is set to mp3, set this parameter to "1" to enable the built-in decoder of the SDK. In this case, onTtsDataCallback returns the decoded PCM data.

  • 0: Disable.

Key interfaces

NeoNuiTts

nui_tts_initialize

Initializes the speech synthesis SDK instance. The SDK is a singleton. Do not initialize the instance again before you call <a baseurl="t3161921_v1_0_0.xdita" data-node="6176237" data-root="85177" data-tag="xref" href="#6c2931e9ae3eq" id="6161b501fajbh">nui_tts_release</a>.

  • Method signature

    -(int) nui_tts_initialize:(const char *)parameters
                     logLevel:(NuiSdkLogLevel)level
                      saveLog:(BOOL)save_log;
  • Parameter descriptions

    Parameter

    Type

    Description

    parameters

    char*

    A JSON string that contains authentication, connection, and debugging parameters. For more information, see Connection and control parameters.

    level

    NuiSdkLogLevel

    Controls the logging level of the SDK itself.

    save_log

    BOOL

    Specifies whether to save binary logs. If this parameter is set to YES, you must use the debug_path parameter to specify a path in Connection and control parameters. You can also use max_log_file_size to set the file size.

  • Return value description

    Returns an error code. For more information, see Query error codes.

nui_tts_set_param

Before calling <a baseurl="t3161921_v1_0_0.xdita" data-node="6176237" data-root="85177" data-tag="xref" href="#8fe6ea298apzu" id="97393f5763ycp">nui_tts_play</a>, you must set the speech synthesis effect parameters as key-value pairs.

nui_tts_get_param

Retrieves a parameter value. This method is mainly used for troubleshooting.

  • Method signature

    -(const char *) nui_tts_get_param:(const char *)param;
  • Parameter descriptions

    Parameter

    Type

    Description

    param

    char*

    The parameter. Currently, only "error_msg" is supported.

  • Return value description

    Returns the parameter value.

nui_tts_play

Starts a speech synthesis task.

  • Method signature

    -(int) nui_tts_play:(const char *)priority
                 taskId:(const char *)taskid
                  text:(const char *)text;
  • Parameter descriptions

    Parameter

    Type

    Description

    priority

    char*

    The task priority. Set this parameter to 1.

    taskid

    char*

    The task ID. If you pass null, the SDK automatically generates a task ID.

    text

    char*

    The text to be synthesized.

  • Return value description

    Returns an error code. For more information, see Query error codes.

nui_tts_pause

Pauses the current speech synthesis task. After the task is paused, you can resume it by calling <a baseurl="t3161921_v1_0_0.xdita" data-node="6176237" data-root="85177" data-tag="xref" href="#448e8ffafd72e" id="6cb9a76d37uvv">nui_tts_resume</a> or cancel it by calling <a baseurl="t3161921_v1_0_0.xdita" data-node="6176237" data-root="85177" data-tag="xref" href="#156934a01bzjc" id="0cb744598acd8">nui_tts_cancel</a>. The SDK does not support starting new synthesis tasks while a task is paused.

Note: This operation only pauses data from being pulled from the server-side. Audio data that is cached in the player continues to play.

  • Method signature

    -(int) nui_tts_pause;
  • Return value description

    Returns an error code. For more information, see Query error codes.

nui_tts_resume

Resumes a paused speech synthesis task.

  • Method signature

    -(int) nui_tts_resume;
  • Return value description

    Returns an error code. For more information, see Query error codes.

nui_tts_cancel

Cancels a synthesis task.

Note: This operation only cancels data from being pulled from the server-side. Audio data that is cached in the player continues to play.

  • Method signature

    -(int) nui_tts_cancel:(const char *)taskid;
  • Parameter descriptions

    Parameter

    Type

    Description

    taskid

    char*

    The ID of the task to cancel. If you pass null, all paused or ongoing synthesis tasks are canceled.

  • Return value description

    Returns an error code. For more information, see Query error codes.

nui_tts_release

Releases all internal SDK resources and forcibly terminates all ongoing synthesis tasks. After this method is called, the SDK instance becomes unavailable. To use the instance again, you must call <a baseurl="t3161921_v1_0_0.xdita" data-node="6176237" data-root="85177" data-tag="xref" href="#05eab5125e2pm" id="0086f97fefeuw">nui_tts_initialize</a> to re-initialize it.

  • Method signature

    -(int) nui_tts_release;
  • Return value description

    Returns an error code. For more information, see Query error codes.

NeoNuiTtsDelegate: Listener callbacks

onNuiTtsEventCallback: Listen for events

  • Method signature

    - (void)onNuiTtsEventCallback:(NuiSdkTtsEvent)event taskId:(char*)taskid code:(int)code;
  • Parameter descriptions

    Parameter

    Type

    Description

    event

    NuiSdkTtsEvent

    The callback event.

    taskid

    char*

    The speech synthesis task ID.

    code

    int

    The error code. This is valid only in the TTS_EVENT_ERROR event. For more information, see Query error codes.

onNuiTtsUserdataCallback: Listen for audio data and timestamp information

  • Method signature

    - (void)onNuiTtsUserdataCallback:(char*)info infoLen:(int)info_len buffer:(char*)buffer len:(int)len taskId:(char*)task_id;
  • Parameter descriptions

    Parameter

    Type

    Description

    info

    char*

    The timestamp result in JSON format. This takes effect when the speech synthesis effect parameter word_timestamp_enabled is set to "1".

    info_len

    int

    The data length of the info field. You can ignore this.

    buffer

    char*

    Returns the audio data of the current segment.

    len

    int

    The length of the audio data in bytes.

    task_id

    char*

    The speech synthesis task ID.

onNuiTtsLogTrackCallback: Listen for tracking logs

This callback is used to receive detailed internal logs from the SDK for troubleshooting and debugging.

- (void)onNuiTtsLogTrackCallback:(NuiSdkLogLevel)level
                      logMessage:(const char *)log;

NuiSdkTtsEvent: Event types

Event

Description

TTS_EVENT_START

The synthesis task starts. Audio data will be returned soon.

TTS_EVENT_END

The synthesis task ends normally. All audio data has been sent through the callback.

TTS_EVENT_CANCEL

The synthesis task is canceled.

TTS_EVENT_PAUSE

The synthesis task is paused.

TTS_EVENT_RESUME

The synthesis task is resumed.

TTS_EVENT_ERROR

An error occurred during synthesis. You can call nui_tts_get_param: "error_msg" to get detailed error information.

{
  "header": {
    "task_id": "xxxxxxxxx",
    "event": "task-failed",
    "error_code": "InvalidParameter",
    "error_message": "Please ensure input text is valid.",
    "attributes": {}
  },
  "payload": {}
}

Model list

Note

The default sample rate is the optimal sample rate for the current model. By default, the output is based on this sample rate. Downsampling and upsampling are also supported. For example, for the Zhimiao voice, the default sample rate is 16 kHz. You can downsample it to 8 kHz, but upsampling it to 48 kHz does not provide additional benefits.

Timbre

Audio sample (Right-click to save the audio)

model parameter

Timestamp support

Scenarios

Features

Language

Default sample rate (Hz)

Zhinan

sambert-zhinan-v1

Yes

General scenarios

Advertising male voice

Chinese and English

48 kHz

Zhiqi

sambert-zhiqi-v1

Yes

General scenarios

Gentle female voice

Chinese and English

48 kHz

Zhichu

sambert-zhichu-v1

Yes

News broadcasting

A Bite of China male voice

Chinese and English

48 kHz

Zhide

sambert-zhide-v1

Yes

News broadcasting

News male voice

Chinese and English

48 kHz

Zhijia

sambert-zhijia-v1

Yes

News broadcasting

Standard female voice

Chinese and English

48 kHz

Zhiru

sambert-zhiru-v1

Yes

News broadcasting

News female voice

Chinese and English

48 kHz

Zhiqian

sambert-zhiqian-v1

Yes

Dubbing and news broadcasting

Information female voice

Chinese and English

48 kHz

Zhixiang

sambert-zhixiang-v1

Yes

Dubbing

Magnetic male voice

Chinese and English

48 kHz

Zhiwei

sambert-zhiwei-v1

Yes

Reading product introductions

Lolita female voice

Chinese and English

48 kHz

Zhihao

sambert-zhihao-v1

Yes

General scenarios

Consultation male voice

Chinese and English

16 KB

Zhijing

sambert-zhijing-v1

Yes

General scenarios

Strict female voice

Chinese and English

16 k

Zhiming

sambert-zhiming-v1

Yes

General scenarios

Humorous male voice

Chinese and English

16 KB

Zhimo

sambert-zhimo-v1

Yes

General scenarios

Emotional male voice

Chinese and English

16 kHz

Zhina

sambert-zhina-v1

Yes

General scenarios

Zhejiang Mandarin female voice

Chinese and English

16 kHz

Zhishu

sambert-zhishu-v1

Yes

General scenarios

Information male voice

Chinese and English

16 KB

Zhisha

sambert-zhistella-v1

Yes

General scenarios

Intellectual female voice

Chinese and English

16 kHz

Zhiting

sambert-zhiting-v1

Yes

General scenarios

Radio female voice

Chinese and English

16 kHz

Zhixiao

sambert-zhixiao-v1

Yes

General scenarios

Information female voice

Chinese and English

16 kHz

Zhiya

sambert-zhiya-v1

Yes

General scenarios

Strict female voice

Chinese and English

16 kHz

Zhiye

sambert-zhiye-v1

Yes

General scenarios

Young male voice

Chinese and English

16 kHz

Zhiying

sambert-zhiying-v1

Yes

General scenarios

Cute child voice

Chinese and English

16 kHz

Zhiyuan

sambert-zhiyuan-v1

Yes

General scenarios

Caring sister voice

Chinese and English

16 kHz

Zhiyue

sambert-zhiyue-v1

Yes

Customer service

Gentle female voice

Chinese and English

16 kHz

Zhigui

sambert-zhigui-v1

Yes

Reading product introductions

Livestreaming female voice

Chinese and English

16 kHz

Zhishuo

sambert-zhishuo-v1

Yes

Digital human

Natural male voice

Chinese and English

16 kHz

Zhimiao (multi-emotional)

sambert-zhimiao-emo-v1

Yes

Reading product introductions, digital humans, and livestreaming

Multi-emotional female voice

Chinese and English

16 kHz

Zhimao

sambert-zhimao-v1

Yes

Reading product introductions, dubbing, digital humans, and livestreaming

Livestreaming female voice

Chinese and English

16 kHz

Zhilun

sambert-zhilun-v1

Yes

Dubbing

Suspense narration

Chinese and English

16 kHz

Zhifei

sambert-zhifei-v1

Yes

Dubbing

Passionate narration

Chinese and English

16 KB

Zhida

sambert-zhida-v1

Yes

News broadcasting

Standard male voice

Chinese and English

16 KB

Camila

sambert-camila-v1

No

General scenarios

Spanish female voice

Spanish

16 kHz

Perla

sambert-perla-v1

No

General scenarios

Italian female voice

Italian

16 k

Indah

sambert-indah-v1

No

General scenarios

Indonesian female voice

Indonesian

16 kHz

Clara

sambert-clara-v1

No

General scenarios

French female voice

French

16 kHz

Hanna

sambert-hanna-v1

No

General scenarios

German female voice

German

16 kHz

Beth

sambert-beth-v1

Yes

General scenarios

Consultation female voice

American English

16 kHz

Betty

sambert-betty-v1

Yes

General scenarios

Customer service female voice

American English

16 kHz

Cally

sambert-cally-v1

Yes

General scenarios

Natural female voice

American English

16 kHz

Cindy

sambert-cindy-v1

Yes

General scenarios

Conversational female voice

American English

16 kHz

Eva

sambert-eva-v1

Yes

General scenarios

Companion female voice

American English

16 KB

Donna

sambert-donna-v1

Yes

General scenarios

Education female voice

American English

16 kHz

Brian

sambert-brian-v1

Yes

General scenarios

Customer service male voice

American English

16 kHz

Waan

sambert-waan-v1

No

General scenarios

Thai female voice

Thai

16 kHz