iOS SDK

Prerequisites

• Read the API reference. For more information, see API reference.

• Create a project and obtain an app key. For more information, see Create a project.

• Obtain an access token. For more information, see Get an access token.

Download and install

1. Select and download the mobile SDK.

   Important

   After downloading, replace the placeholder credentials in the sample initialization code with your Alibaba Cloud account information, app key, and access token.

   Starting from version 2.5.14, the iOS SDK uses a pure Objective-C API instead of a C++ hybrid API to simplify integration.

2. Unzip the package and add nuisdk.framework to your project. In the project's Build Phases, add nuisdk.framework to Link Binary With Libraries. Then, in the General tab under Frameworks, Libraries, and Embedded Content, set the Embed option for nuisdk.framework to Embed & Sign.

3. Open the project in Xcode. It contains reference code and reusable utility classes for tasks such as audio playback, recording, and file operations, which you can copy directly into your project. The sample code for Text-to-Speech (TTS) is in the TTSViewController class.

Key SDK APIs

• tts_initialize: Initializes the SDK.

  /**
   * Initializes the SDK. The SDK is a singleton. You must release the existing instance before re-initializing. Do not call this method on the UI thread, as it may cause blocking.
   * @param parameters: The initialization parameters. For more information, see the parameter description below or the API reference at: https://help.aliyun.com/document_detail/173642.html
   * @param level: The log level. A lower value produces more detailed logs.
   * @param save_log: Specifies whether to save logs to a file. The log file is stored in the directory specified by the debug_path field in the parameters. Note: Log files are not automatically rotated or capped, which can lead to excessive disk usage.
   * @return For a list of possible values, see the error codes at: https://help.aliyun.com/document_detail/459864.html
   */
  -(int) nui_tts_initialize:(const char *)parameters
                   logLevel:(NuiSdkLogLevel)level
                    saveLog:(BOOL)save_log;

• nui_tts_play: Starts playback.

  /**
   * Starts playback.
   * @param priority: The task priority. Set this to "1".
   * @param taskid: The task ID. You can pass a 32-byte UUID or pass a null value to let the SDK generate one automatically.
   * @param text: The text to be synthesized.
   * @return For a list of possible values, see the error codes at: https://help.aliyun.com/document_detail/459864.html
   */
  -(int) nui_tts_play:(const char *)priority
               taskId:(const char *)taskid
                text:(const char *)text;

• nui_tts_cancel: Cancels a synthesis task.

  /**
   * Cancels a synthesis task.
   * @param taskid: The ID of the task to cancel. If this is null, all tasks are canceled.
   * @return For a list of possible values, see the error codes at: https://help.aliyun.com/document_detail/459864.html
   */
  -(int) nui_tts_cancel:(const char *)taskid;

• nui_tts_pause: Pauses playback.

  /**
   * Pauses playback.
   * @return For a list of possible values, see the error codes.
   */
  -(int) nui_tts_pause;

• nui_tts_resume: Resumes playback.

  /**
   * Resumes a paused task.
   * @return For a list of possible values, see the error codes.
   */
  -(int) nui_tts_resume;

• nui_tts_set_param: Sets Text-to-Speech (TTS) parameters.

  /**
   * Sets a parameter as a key-value pair.
   * @param param: The parameter name. For more information, see the API reference.
   * @param value: The parameter value. For more information, see the API reference.
   * @return For a list of possible values, see the error codes.
   */
  -(int) nui_tts_set_param:(const char *)param
                     value:(const char *)value;

• nui_tts_get_param: Gets a parameter value.

  /**
   * Gets a parameter value.
   * @param param: The parameter name. For more information, see the API reference.
   * @return The parameter value.
   */
  -(const char *) nui_tts_get_param:(const char *)param;

• nui_tts_release: Releases SDK resources.

  /**
   * Releases the SDK resources.
   * @return For a list of possible values, see the error codes.
   */
  -(int) nui_tts_release;

• NeoNuiTtsDelegate: Event delegate.

  • onNuiTtsUserdataCallback: Provides audio data in a callback.

    /**
     * This callback is invoked repeatedly to deliver chunks of synthesized audio data.
     * @param info: Returns timestamp results in JSON format when the timestamp feature is enabled.
     * @param info_len: The length of the data in the info field.
     * @param buffer: A pointer to the buffer containing the synthesized audio data.
     * @param len: The length of the synthesized audio data in the buffer.
     * @param taskid: The ID of the current synthesis task.
     */
    - (void)onNuiTtsUserdataCallback:(char*)info infoLen:(int)info_len buffer:(char*)buffer len:(int)len taskId:(char*)task_id;

• onNuiTtsEventCallback: The main event callback for the SDK.

  /**
   * The main event callback for the SDK.
   * @param event: The callback event. For more information, see the API reference: https://help.aliyun.com/document_detail/459864.html
   * @param taskid: The ID of the current synthesis task.
   * @param code: The error code. This is valid only when the event is TTS_EVENT_ERROR.
   */
  - (void)onNuiTtsEventCallback:(NuiSdkTtsEvent)event taskId:(char*)taskid code:(int)code;

• NuiSdkTtsEvent events:

  Parameter	Description
  TTS_EVENT_START	TTS synthesis has started and is ready for playback.
  TTS_EVENT_END	TTS synthesis has finished and all audio data has been delivered. This event does not mean that playback is also complete.
  TTS_EVENT_CANCEL	TTS synthesis was canceled.
  TTS_EVENT_PAUSE	TTS synthesis was paused.
  TTS_EVENT_RESUME	TTS synthesis was resumed.
  TTS_EVENT_ERROR	An error occurred during TTS synthesis.

Procedure

1. Initialize the SDK and the audio player component.

2. Set the synthesis parameters based on your requirements.

3. Call nui_tts_play to start synthesis.

4. In the data callback, write the synthesized audio data to the player for playback. Streaming playback is recommended.

   If you want to save the complete synthesized audio to a local file, append each chunk of audio data to a file.

5. Receive the callback indicating that synthesis has finished.

Code examples

Initialize TTS

NSString * initParam = [self genInitParams];
[_nui nui_tts_initialize:[initParam UTF8String] logLevel:LOG_LEVEL_VERBOSE saveLog:true];

The genInitParams method generates a JSON string that contains the resource directory and user information. The user information includes the following fields.

-(NSString *)genInitParams {
    NSString *strResourcesBundle = [[NSBundle mainBundle] pathForResource:@"Resources" ofType:@"bundle"];
    NSString *bundlePath = [[NSBundle bundleWithPath:strResourcesBundle] resourcePath];
    NSString *debug_path = [_utils createDir];
    NSMutableDictionary *ticketJsonDict = [NSMutableDictionary dictionary];
    // Note:
    //   Before you use Intelligent Speech Interaction, you must have an account and enable the required services. For more information, see:
    //   https://help.aliyun.com/zh/isi/getting-started/start-here
    //
    // Alibaba Cloud account:
    //   Your account credentials include an AccessKey ID (ak_id) and an AccessKey Secret (ak_secret).
    //   To prevent security risks and unexpected charges, do not hardcode your AccessKey ID and AccessKey Secret in your application code or store them on the client side.
    //
    // STS temporary credentials:
    //   Because distributing account credentials to clients poses a security risk, Alibaba Cloud provides Security Token Service (STS) for temporary access management.
    //   You can use your AccessKey ID and AccessKey Secret to request temporary credentials from STS, which consist of an STS AccessKey ID (sts_ak_id), an STS AccessKey Secret (sts_ak_secret), and a security token (sts_token).
    //   (The "sts_" prefix is used to distinguish temporary credentials from permanent account credentials.)
    // What is STS: https://help.aliyun.com/zh/ram/product-overview/what-is-sts
    // STS SDK overview: https://help.aliyun.com/zh/ram/developer-reference/sts-sdk-overview
    // STS Python SDK example: https://help.aliyun.com/zh/ram/developer-reference/use-the-sts-openapi-example
    //
    // Credential requirements:
    //   To use offline features (such as offline TTS and wake-word), you must provide either an app_key, ak_id, and ak_secret, or an app_key, sts_ak_id, sts_ak_secret, and sts_token.
    //   To use online features (such as TTS, Real-time Speech Recognition, Short Speech Recognition, and audio file transcription), you only need an app_key and an access token.
    [_utils getTicket:ticketJsonDict Type:get_token_from_server_for_online_features];
    if ([ticketJsonDict objectForKey:@"token"] != nil) {
        NSString *tokenValue = [ticketJsonDict objectForKey:@"token"];
        if ([tokenValue length] == 0) {
            TLog(@"The 'token' key exists but the value is empty.");
        }
    } else {
        TLog(@"The 'token' key does not exist.");
    }
    // The working directory path from which the SDK reads configuration files.
    [ticketJsonDict setObject:bundlePath forKey:@"workspace"]; // Required. The path must have read and write permissions.
    TLog(@"workspace:%@", bundlePath);
    [ticketJsonDict setObject:debug_path forKey:@"debug_path"];
    TLog(@"debug_path:%@", debug_path);
    [ticketJsonDict setObject:@"wss://nls-gateway.cn-shanghai.aliyuncs.com:443/ws/v1" forKey:@"url"]; // Default
    // Set the mode to online TTS. This setting is critical. If omitted, the service will not run.
    [ticketJsonDict setObject:@"2" forKey:@"mode_type"]; // Required
    NSString *id_string = [[[ASIdentifierManager sharedManager] advertisingIdentifier] UUIDString];
    TLog(@"id: %s", [id_string UTF8String]);
    [ticketJsonDict setObject:id_string forKey:@"device_id"]; // Required. We recommend using a unique ID to help troubleshoot issues.
    NSData *data = [NSJSONSerialization dataWithJSONObject:ticketJsonDict options:NSJSONWritingPrettyPrinted error:nil];
    NSString * jsonStr = [[NSString alloc]initWithData:data encoding:NSUTF8StringEncoding];
    return jsonStr;
}

Set parameters

// You can set parameters such as the voice, intonation, and speech rate. For a full list of parameters, see the API reference.
[self.nui nui_tts_set_param:"font_name" value:"xiaoyun"];

Start TTS synthesis

// We recommend running only one synthesis task at a time per instance. Running multiple tasks on a single instance may cause exceptions.
[self.nui nui_tts_play:"1" taskId:"" text:[content UTF8String]];

Handle callbacks

• onNuiTtsEventCallback: The TTS event callback. Use this callback to control the audio player based on the synthesis state.

  - (void)onNuiTtsEventCallback:(NuiSdkTtsEvent)event taskId:(char*)taskid code:(int)code {
      TLog(@"onNuiTtsEventCallback event[%d]", event);
      if (event == TTS_EVENT_START) {
          TLog(@"onNuiTtsEventCallback TTS_EVENT_START");
          loop_in = TTS_EVENT_START;
          // The player provided in legacy sample projects is for reference only. You can implement your own player based on your business needs.
          // [self->_voicePlayer play];
          // The new player provided in recent sample projects is for reference only. You can implement your own player based on your business needs.
          [_audioController startPlayer];
      } else if (event == TTS_EVENT_END || event == TTS_EVENT_CANCEL || event == TTS_EVENT_ERROR) {
          loop_in = event;
          if (event == TTS_EVENT_END) {
              TLog(@"onNuiTtsEventCallback TTS_EVENT_END");
              // The player provided in legacy sample projects.
              // Note: This event indicates that synthesis is complete, not that playback is complete. The voicePlayer object notifies you when playback is finished.
              // [self->_voicePlayer drain];
              // The new player provided in recent sample projects.
              // Note: This event indicates that synthesis is complete, not that playback is complete. The audioController object notifies you when playback is finished.
              [_audioController drain];
          } else {
              // The player provided in legacy sample projects.
              // Stop playback if the task is canceled or an error occurs.
              // [self->_voicePlayer stop];
              // The new player provided in recent sample projects.
              // Stop playback if the task is canceled or an error occurs.
              [_audioController stopPlayer];
          }
          if (event == TTS_EVENT_ERROR) {
              const char *errmsg = [_nui nui_tts_get_param: "error_msg"];
              TLog(@"tts get errmsg:%s", errmsg);
          }
      }
  }

• onNuiTtsUserdataCallback: The TTS data callback. Write the synthesized data from this callback to the player for playback.

  - (void)onNuiTtsUserdataCallback:(char*)info infoLen:(int)info_len buffer:(char*)buffer len:(int)len taskId:(char*)task_id {
      TLog(@"onNuiTtsUserdataCallback info ...");
      if (info_len > 0) {
          TLog(@"onNuiTtsUserdataCallback info text %s. index %d.", info, info_len);
      }
      if (len > 0) {
          // The player provided in legacy sample projects is for reference only. You can implement your own player based on your business needs.
          // [_voicePlayer write:(char*)buffer Length:(unsigned int)len];
          // The new player provided in recent sample projects is for reference only. You can implement your own player based on your business needs.
          [_audioController write:(char*)buffer Length:(unsigned int)len];
      }
  }

Stop TTS synthesis

[self.nui nui_tts_cancel:NULL];

- (void)onNuiTtsUserdataCallback:(char*)info infoLen:(int)info_len buffer:(char*)buffer
    len:(int)len taskId:(char*)task_id {
    TLog(@"onnuiTtsUserdataCallback info text %s index %d", info, info_len);
    if (len > 0) {
        [_voicePlayer write:(char*)buffer Length:(unsigned int)len];
        fwrite(buffer, 1, len, fp);
    }
}

# Other code is omitted.
post_install do |installer|
  installer.pods_project.targets.each do |target|
    flutter_additional_ios_build_settings(target)
    target.build_configurations.each do |config|
      config.build_settings["EXCLUDED_ARCHS[sdk=iphonesimulator*]"] = "arm64"
    end
  end
end