Sambert Speech Synthesis iOS SDK

Updated at:

This document provides a detailed guide on using the Sambert speech synthesis software development kit (SDK) for iOS to convert text into high-quality, expressive speech.

Online experience: Not supported.

ImportantAlibaba Cloud Model Studio has launched a workspace-specific domain for the China (Beijing) region, which delivers superior performance and higher stability for inference requests. We recommend that you migrate from dashscope.aliyuncs.com to {WorkspaceId}.cn-beijing.maas.aliyuncs.com.

{WorkspaceId} must be replaced with the actual Workspace ID. The existing domain remains fully functional.

NeoNui

This SDK is built on the NeoNui singleton architecture and handles speech synthesis events through the callback mechanism.

Architecture features:

Usage procedure

  1. nui_tts_initialize() - Initializes the SDK and sets the callback interface and connection parameters
  2. nui_tts_set_param() - Sets the speech synthesis effect parameters (model, timbre, volume, and more)
  3. nui_tts_play() - Starts the speech synthesis task
  4. onNuiTtsUserdataCallback() - Receives the audio data
  5. nui_tts_release() - Releases the SDK resources

Sambert methods

nui_tts_initialize

Initializes the speech synthesis SDK instance. The SDK uses the singleton pattern. Do not initialize the SDK again before you call nui_tts_release.

Method signature:
-(int) nui_tts_initialize:(const char *)parameters
                 logLevel:(NuiSdkLogLevel)level
                  saveLog:(BOOL)save_log;
Parameter descriptions:

Parameter

Type

Description

parameters

char*

A JSON string that contains the authentication, connection, and debug parameters. For more information, see the parameters parameter description below.

level

NuiSdkLogLevel

Controls the print level of the SDK logs.

save_log

BOOL

Specifies whether to save local logs. If this parameter is set to YES, you must specify a path through debug_path in parameters, and you can set the file size through max_log_file_size.

parameters JSON example: The following code shows an example of a JSON string, in which not all parameters are listed. Add parameters based on your actual needs during encoding:

{
    "url": "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference",
    "apikey": "st-****",
    "device_id": "my_device_id"
}
parameters parameter descriptions:

Parameter

Type

Required

Description

url

String

Yes

The service endpoint, which is fixed as wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference.

When you call the interface, replace {WorkspaceId} with the actual Workspace ID.

apikey

String

Yes

The API key. We recommend that you use a temporary API key that has a shorter validity period and higher security to reduce the risk of long-term key leakage.

device_id

String

Yes

A unique string that identifies the end user. You can set it to the in-app user ID or a unique device identifier generated on the client. This ID is mainly used for log tracking and troubleshooting.

debug_path

String

No

The storage path of log files.

This parameter takes effect only when save_log is set to YES when you call the nui_tts_initialize interface. In this case, you must set the log file path. Otherwise, an error is reported.

A maximum of two log files are retained locally.

max_log_file_size

int

No

The maximum number of bytes of a log file.

This parameter takes effect only when save_log is set to YES when you call the nui_tts_initialize interface.

Default value: 104857600 (100 * 1024 * 1024 bytes, i.e., 100MiB).

log_track_level

int

No

The filter level of the log content that is sent through the log callback (onNuiTtsLogTrackCallback).

Default value: 2.

Valid values:

  • 0: LOG_LEVEL_VERBOSE

  • 1: LOG_LEVEL_DEBUG

  • 2: LOG_LEVEL_INFO

  • 3: LOG_LEVEL_WARNING

  • 4: LOG_LEVEL_ERROR

  • 5: LOG_LEVEL_NONE (disables this feature)

Note: log_track_level and level (set through the nui_tts_initialize interface) together determine the logs that are finally sent to the callback. A log is sent to the callback only when its level value is greater than or equal to both the log_track_level value and the level value. For example, if log_track_level is set to 2 (INFO) and level is set to 3 (WARNING), only logs at the WARNING level and higher (value >= 3) are sent to the callback.

nui_tts_set_param

Sets parameters in key-value pairs. Call this method before you call nui_tts_play.

Method signature:
-(int) nui_tts_set_param:(const char *)param
                   value:(const char *)value;
Parameter descriptions:

Parameter

Type

Description

param

char*

The parameter name.

value

char*

The parameter value.

Available parameter descriptions:

Parameter

Type

Required

Description

model

String

Yes

The model name, such as sambert-zhichu-v1.

format

String

No

The audio coding format.

Valid values:

  • pcm

  • wav (default)

  • mp3

volume

String

No

The volume.

Default value: 50.

Valid values: [0, 100].

sample_rate

String

No

The audio sample rate (Hz).

Valid values: 8000, 16000 (default), 22050, 24000.

rate

String

No

The speech rate.

Default value: 1.0.

Valid values: [0.5, 2.0].

pitch

String

No

The pitch.

Default value: 1.0.

Valid values: [0.5, 2.0].

word_timestamp_enabled

String

No

Specifies whether to enable word-level timestamps.

Default value: false.

Scope: all Sambert models.

phoneme_timestamp_enabled

String

No

Specifies whether to enable phoneme-level timestamps.

Default value: false.

You must enable word_timestamp_enabled first.

enable_audio_decoder

String

No

Specifies whether to enable the built-in audio decoder.

Default value: 0.

Valid values:

  • 1: Enable. When format is mp3, set this parameter to "1" to enable the built-in decoder of the SDK. In this case, onNuiTtsUserdataCallback returns the decoded PCM data.

  • 0: Disable.

nui_tts_get_param

Obtains parameter values. This method is mainly used for troubleshooting.

Method signature:
-(const char *) nui_tts_get_param:(const char *)param;
Parameter descriptions:

Parameter

Type

Description

param

char*

The parameter. Currently, only "error_msg" is supported.

Return value description:

Returns the parameter value.

nui_tts_play

Starts a speech synthesis task.

Method signature:
-(int) nui_tts_play:(const char *)priority
             taskId:(const char *)taskid
              text:(const char *)text;
Parameter descriptions:

Parameter

Type

Description

priority

char*

The task priority. Set it to 1.

taskid

char*

The task ID. If you pass null, the SDK automatically generates one.

text

char*

The text to synthesize.

nui_tts_pause

Pauses the current speech synthesis task. After the task is paused, you can resume it through nui_tts_resume or cancel it through nui_tts_cancel. While a task is paused, the SDK does not support starting a new synthesis task.

Note: This operation only pauses data pulling from the server. The audio data already cached in the player continues to play.

Method signature:
-(int) nui_tts_pause;

nui_tts_resume

Resumes a paused speech synthesis task.

Method signature:
-(int) nui_tts_resume;

nui_tts_cancel

Cancels a synthesis task.

Note: This operation only cancels data pulling from the server. The audio data already cached in the player continues to play.

Method signature:
-(int) nui_tts_cancel:(const char *)taskid;
Parameter descriptions:

Parameter

Type

Description

taskid

char*

The ID of the task to cancel. If you pass null, all paused/in-progress synthesis tasks are canceled.

nui_tts_release

Releases all internal SDK resources and forcibly terminates all in-progress synthesis tasks. After this method is called, the SDK instance becomes unavailable. To use the SDK again, you must call nui_tts_initialize to initialize it again.

Method signature:
-(int) nui_tts_release;

NeoNuiTtsDelegate

The Sambert speech synthesis callback protocol, which is used to receive synthesis events, audio data, and logs.

Methods

onNuiTtsEventCallback: listen to events

Method signature:
- (void)onNuiTtsEventCallback:(NuiSdkTtsEvent)event taskId:(char*)taskid code:(int)code;
Parameter descriptions:

Parameter

Type

Description

event

NuiSdkTtsEvent

The callback event.

taskid

char*

The speech synthesis task ID.

code

int

Valid only when the TTS_EVENT_ERROR event occurs.

onNuiTtsUserdataCallback: listen to audio data and timestamp information

Method signature:
- (void)onNuiTtsUserdataCallback:(char*)info infoLen:(int)info_len buffer:(char*)buffer len:(int)len taskId:(char*)task_id;
Parameter descriptions:

Parameter

Type

Description

info

char*

The timestamp result in JSON format. This parameter takes effect when word_timestamp_enabled is set to "1".

info_len

int

The data length of the info field, which can be ignored.

buffer

char*

The audio data of the current segment.

len

int

The length of the audio data in bytes.

task_id

char*

The speech synthesis task ID.

onNuiTtsLogTrackCallback: listen to trace logs

This callback is used to receive the detailed logs inside the SDK, which facilitates troubleshooting and debugging.

Method signature:
- (void)onNuiTtsLogTrackCallback:(NuiSdkLogLevel)level
                      logMessage:(const char *)log;
Parameter descriptions:

Parameter

Type

Description

level

NuiSdkLogLevel

The log level.

log

char*

The log content.

NuiSdkTtsEvent

The Sambert speech synthesis event type enumeration.

EventDescription

TTS_EVENT_START

The synthesis task starts, and audio data is about to be returned.

TTS_EVENT_END

The synthesis task ends normally, and all audio data has been sent through the callbacks.

TTS_EVENT_CANCEL

The synthesis task is canceled.

TTS_EVENT_PAUSE

The synthesis task is paused.

TTS_EVENT_RESUME

The synthesis task is resumed.

TTS_EVENT_ERROR

An error occurs during synthesis. In this case, you can use nui_tts_get_param: "error_msg" to obtain the detailed error information.

{
  "header": {
    "task_id": "xxxxxxxxx",
    "event": "task-failed",
    "error_code": "InvalidParameter",
    "error_message": "Please ensure input text is valid.",
    "attributes": {}
  },
  "payload": {}
}

NuiSdkLogLevel

The SDK log level enumeration, which is used to control log output.

Level

Description

0: LOG_LEVEL_VERBOSE

The most detailed logs, including all debugging information.

1: LOG_LEVEL_DEBUG

Debug-level logs.

2: LOG_LEVEL_INFO

General information-level logs (default value).

3: LOG_LEVEL_WARNING

Warning-level logs.

4: LOG_LEVEL_ERROR

Error-level logs.

5: LOG_LEVEL_NONE

Disables log output.

Sample code

  1. Obtain an API key: Obtain and configure an API key

    NoteWhen you need to grant temporary access permissions to third-party applications or users, or want to strictly control high-risk operations such as sensitive data access and deletion, we recommend that you use a temporary API key. A temporary API key has a fixed validity period of 60 seconds. You must obtain a new one after it expires.

  2. Download the SDK and run the sample code:
    • Download the latest SDK package.
    • Decompress the ZIP package and add nuisdk.framework in it to your project.
    • In Build Phases → Link Binary With Libraries, add nuisdk.framework.
    • In General → Frameworks, Libraries, and Embedded Content, set nuisdk.framework to Embed & Sign.
    • Open the sample project in Xcode. The sample code is in DashSambertTTSViewController. Replace the API key and try the feature.

Call procedure

  1. Initialize the SDK.
  2. Set the parameters based on your business needs: set the connection parameters through the parameters parameter of the nui_tts_initialize interface, and set the speech synthesis effect parameters through the nui_tts_set_param interface.
  3. Call nui_tts_play to start speech synthesis.
  4. Obtain the audio data in the onNuiTtsUserdataCallback callback. We recommend that you use stream playback. To save the audio locally, write the audio to the same file in append mode until the synthesis is complete.
  5. After the task is complete, call nui_tts_release to release the SDK resources.