iOS SDK

Prerequisites

• Read the API reference. For more information, see API reference.

• Obtain a project AppKey. For more information, see Create a project.

• Obtain an AccessKey ID and AccessKey secret. For more information, see Activate the service.

Download and installation

1. Select and download the mobile SDK.

   Important

   After you download the SDK, you must replace the Alibaba Cloud account information and AppKey in the sample initialization code to run the demo. For easier integration, iOS interfaces from version 2.5.14 and later use pure Objective-C interfaces instead of mixed C++ interfaces.

2. Download a voice package. For more information, see the Voice package list in API reference.

   Important

   The SDK and voice packages are separate. After you download the SDK, you must also download a voice package and set its storage path to use the service.

3. Decompress the ZIP package.

   Add nuisdk.framework from the ZIP package to your project. In your project's Build Phases, add nuisdk.framework to the Link Binary With Libraries section.

4. Open the project in Xcode.

   The project provides reference code and ready-to-use utilities for tasks such as audio playback, recording, and file operations. You can copy the source code directly into your project. The sample code for speech synthesis is in the LocalTTSViewController class. You can run the code directly after you replace the AppKey and token.

Key SDK interfaces

• nui_tts_initialize: Initializes the SDK.

  /**
   * Initializes the SDK. Offline synthesis does not support multiple instances. Release the current instance before re-initializing. Do not call this interface in the UI thread, as it may cause blocking.
   * Initialization is a time-consuming operation. Do not perform it for every synthesis task. Perform it once at startup and release it at exit.
   * @param parameters: Initialization parameters. For more information, see the API reference:
  https://help.aliyun.com/en/isi/developer-reference/sdk-reference-11
   * @param level: The log level. A smaller value means more logs are printed.
   * @param save_log: Specifies whether to save logs to a file. The storage directory is the value of the debug_path field in the parameter. Note that log files have no size limit. Continuous storage may fill up the disk.
   * @return For more information about error codes, see https://help.aliyun.com/document_detail/459864.html
   */
  -(int) nui_tts_initialize:(const char *)parameters
                   logLevel:(NuiSdkLogLevel)level
                    saveLog:(BOOL)save_log;

• nui_tts_play: Starts playback.

  /**
   * Starts playback. This interface is asynchronous.
   * @param priority: The task priority. Use "1".
   * @param taskid: The task ID. You can pass a 32-byte UUID or leave it empty for the SDK to generate one automatically.
   * @param text: The text to be played.
   * @return For more information, see error codes.
   */
  -(int) nui_tts_play:(const char *)priority
               taskId:(const char *)taskid
                 text:(const char *)text;

• nui_tts_cancel: Cancels playback.

  /**
   * Cancels a synthesis task.
   * @param taskid: The ID of the task to stop. If empty, all tasks are canceled.
   * @return For more information, see error codes.
   */
  -(int) nui_tts_cancel:(const char *)taskid;

• nui_tts_pause: Pauses playback.

  /**
   * Pauses playback.
   * @return For more information, see error codes.
   */
  -(int) nui_tts_pause;

• nui_tts_resume: Resumes playback.

  /**
   * Resumes a paused task.
   * @return For more information, see error codes.
   */
  -(int) nui_tts_resume;

• nui_tts_set_param: Sets speech synthesis parameters.

  /**
   * Sets parameters as key-value pairs.
   * @param param: The parameter name. For more information, see the API reference: https://help.aliyun.com/en/isi/developer-reference/sdk-reference-11
   * @param value: The parameter value. For more information, see the API reference: https://help.aliyun.com/en/isi/developer-reference/sdk-reference-11
   * @return For more information, see error codes.
   */
  -(int) nui_tts_set_param:(const char *)param
                     value:(const char *)value;

• nui_tts_get_param: Retrieves parameters.

  /**
   * Gets a parameter value.
   * @param param: The parameter name. For more information, see the API reference: https://help.aliyun.com/en/isi/developer-reference/sdk-reference-11
   * @return The parameter value.
   */
  -(const char *) nui_tts_get_param:(const char *)param;

• nui_tts_release: Releases SDK resources.

  /**
   * Releases the SDK.
   * @return For more information, see error codes.
   */
  -(int) nui_tts_release;

• NeoNuiTtsDelegate: Event delegate.

  • onNuiTtsUserdataCallback: Provides audio data in the callback.

    /**
     * This callback is called continuously when recognition starts. The app needs to fill in the voice data in the callback.
     * @param info: Returns the timestamp result in JSON format when the timestamp feature is used.
     * @param info_len: The data length of the info field.
     * @param buffer: The synthesized voice data.
     * @param len: The length of the synthesized voice.
     * @param taskid: The task ID for this synthesis.
     */
    - (void)onNuiTtsUserdataCallback:(char*)info infoLen:(int)info_len buffer:(char*)buffer len:(int)len taskId:(char*)task_id;

  • onNuiTtsEventCallback: Event callback.

    /**
     * The main event callback for the SDK.
     * @param event: The callback event. For more information, see the API reference: https://help.aliyun.com/en/isi/developer-reference/sdk-reference-11
     * @param taskid: The task ID for this synthesis.
     * @param code: The error code, valid when the event is TTS_EVENT_ERROR.
     */
    - (void)onNuiTtsEventCallback:(NuiSdkTtsEvent)event taskId:(char*)taskid code:(int)code;

    The following table lists the NuiSdkTtsEvent events:

    Name	Description
    TTS_EVENT_START	Speech synthesis starts, preparing for playback.
    TTS_EVENT_END	Speech synthesis ends. All synthesized data has been sent, but this does not mean playback has finished.
    TTS_EVENT_CANCEL	Speech synthesis is canceled.
    TTS_EVENT_PAUSE	Speech synthesis is paused.
    TTS_EVENT_RESUME	Speech synthesis is resumed.
    TTS_EVENT_ERROR	An error occurred during speech synthesis.

  • onNuiTtsLogTrackCallback: SDK internal log callback (added in version 2.6.4).

    /**
     * SDK internal log callback.
     * @param level: Internal SDK logs with a level higher than this will be sent through this callback.
     * @param log: The specific log content.
     */
    -(void) onNuiTtsLogTrackCallback:(NuiSdkLogLevel)level
                          logMessage:(const char *)log;

Invocation steps

1. Initialize the SDK and the playback component.

2. Set parameters as needed.

3. Call nui_tts_play to start playback.

4. In the synthesis data callback, write the data to the player for playback. We recommend that you use streaming playback.

5. Receive the callback that indicates the end of speech synthesis.

Code examples

1. Initialize speech synthesis.

   // After a successful initialization, you can repeatedly call synthesis and parameter setting interfaces. Frequent initialization and release are not required, which saves time.
   NSString * initParam = [self genInitParams];
   [_nui nui_tts_initialize:[initParam UTF8String] logLevel:NUI_LOG_LEVEL_VERBOSE saveLog:YES];
   if (retcode != 0) {
       // Initialization failed. Check "error_msg" for detailed error information. Common errors are listed in the offline speech synthesis FAQ document.
       const char *errmsg = [_nui nui_tts_get_param: "error_msg"];
       TLog(@"init failed. retcode:%d. errmsg:%s", retcode, errmsg);
       // If initialization fails, do not call parameter setting and synthesis interfaces.
       return;
   }

   The genInitParams method generates a JSON string that contains the resource directory and user information. The user information includes the following fields.

   -(NSString *)genInitParams {
       NSString *strResourcesBundle = [[NSBundle mainBundle] pathForResource:@"Resources" ofType:@"bundle"];
       NSString *bundlePath = [[NSBundle bundleWithPath:strResourcesBundle] resourcePath];
       NSString *debug_path = [_utils createDir];
       NSMutableDictionary *dictM = [NSMutableDictionary dictionary];
       // Important notice:
       //  To use the interactive voice response service, you must prepare an account and activate the relevant services. For more information, see:
       //    https://help.aliyun.com/en/isi/getting-started/start-here
       //
       // Primary account:
       //  Account (RAM user) information mainly includes the AccessKey ID (ak_id) and AccessKey secret (ak_secret).
       //  Do not store this account information in the app code or on the client side to prevent leaks and financial loss.
       //
       // STS temporary credential:
       //  Because sending account information to the client poses a security risk, Alibaba Cloud provides the Security Token Service (STS) for temporary access permission management.
       //  STS generates a temporary sts_ak_id, sts_ak_secret, and sts_token from the primary ak_id and ak_secret.
       //  (The "sts_" prefix distinguishes temporary credentials generated by STS from the primary account information.)
       // What is STS: https://help.aliyun.com/en/ram/product-overview/what-is-sts
       // STS SDK overview: https://help.aliyun.com/en/ram/developer-reference/sts-sdk-overview
       // STS Python SDK call example: https://help.aliyun.com/en/ram/developer-reference/use-the-sts-openapi-example
       //
       // Account requirements:
       //  To use offline features (offline speech synthesis, wake-word), you must provide either app_key, ak_id, and ak_secret, or app_key, sts_ak_id, sts_ak_secret, and sts_token.
       //  To use online features (speech synthesis, real-time transcription, short sentence recognition, audio file transcription), you only need the app_key and token.
       // Obtain access credentials:
       [_utils getTicket:dictM Type:get_sts_access_from_server_for_offline_features];
       // For information about how to obtain authentication information and quotas, see the Alibaba Cloud official documentation:
       //  https://help.aliyun.com/document_detail/251488.html?spm=a2c4g.11186623.6.638.1f0d530eut95Jn
       //  If your quota is exhausted, contact your customer manager to increase it.
       //  If synthesis fails, it is usually due to an authentication failure. Refer to the Q&A section on the Alibaba Cloud official website to identify the cause based on the error log:
       //  https://help.aliyun.com/document_detail/204191.html?spm=a2c4g.11186623.6.657.3cde7340qMll1h
       // The working directory path from which the SDK reads the configuration file.
       [dictM setObject:bundlePath forKey:@"workspace"];
       // To save debug logs to a file, include this field during initialization. To disable logging, remove this field.
       // Log files are appended. Old logs are not overwritten during the next initialization.
       // Additionally, when saving log files is enabled, you can dynamically set the log level to the highest value using the nui_tts_set_param interface to prevent logs from being written to the file. You can then dynamically set it to a lower level when you need to write logs.
       [dictM setObject:debug_path forKey:@"debug_path"];
       TLog(@"debug_path:%@", debug_path);
       TLog(@"workspace:%@", bundlePath);
       // Filter internal SDK logs to be sent back to the user layer through callbacks.
       [dictM setObject:[NSString stringWithFormat:@"%d", NUI_LOG_LEVEL_INFO] forKey:@"log_track_level"];
       // Set the maximum size in bytes for local log files. A maximum of two log files of the set size will be stored locally.
       [dictM setObject:@(50 * 1024 * 1024) forKey:@"max_log_file_size"];
       [dictM setObject:@"wss://nls-gateway.cn-shanghai.aliyuncs.com:443/ws/v1" forKey:@"url"];
       // Set to local speech synthesis mode. This setting is important. Omitting it will prevent the program from running.
       [dictM setObject:@"0" forKey:@"mode_type"]; // Required
       // Note: The ID used for authentication is generated by encrypting a combination of the following device_id and some unique codes from the mobile phone.
       //   Changing the mobile phone or the device_id will trigger re-authentication and billing.
       //   In addition, set a meaningful and unique device_id, such as a user account (phone number, IMEI).
       //   Passing the same or a randomly changing device_id will cause authentication failure or repeated charges.
       //   NSString *id_string = [[[ASIdentifierManager sharedManager] advertisingIdentifier] UUIDString]; does not guarantee a constant device_id. Do not use it.
       [dictM setObject:@"empty_device_id" forKey:@"device_id"]; // Required
       NSData *data = [NSJSONSerialization dataWithJSONObject:dictM options:NSJSONWritingPrettyPrinted error:nil];
       NSString * jsonStr = [[NSString alloc]initWithData:data encoding:NSUTF8StringEncoding];
       return jsonStr;
   }

2. Set parameters as needed.

   // Load the voice package: The purchased voice package can be placed anywhere. For example, if the aijia voice package is located in Documents/voices/, set the command to "Documents/voices/aijia".
   NSString *cmd = [NSString stringWithFormat:@"%@/aijia", myvoicedir];
   [self.nui nui_tts_set_param:"extend_font_name" value:[cmd UTF8String]];

3. Start speech synthesis.

   // We recommend starting one task at a time for speech synthesis in a single instance. Multiple tasks in a single instance can easily cause exceptions.
   [self.nui nui_tts_play:"1" taskId:"" text:[content UTF8String]];

4. Cancel speech synthesis.

   // If the previous task is not complete, cancel it manually before starting a new one.
   // We recommend starting one task at a time for speech synthesis in a single instance. Multiple tasks in a single instance can easily cause exceptions during cancellation.
   [self.nui nui_tts_cancel:NULL];

5. Handle callbacks.

   • onNuiTtsEventCallback: This is the speech synthesis event callback. You can use this callback to control the player based on the synthesis status.

     - (void)onNuiTtsEventCallback:(NuiSdkTtsEvent)event taskId:(char*)taskid code:(int)code {
         TLog(@"onNuiTtsEventCallback event[%d]", event);
         if (event == TTS_EVENT_START) {
             TLog(@"onNuiTtsEventCallback TTS_EVENT_START");
             loop_in = TTS_EVENT_START;
             // The player provided in the old version of the sample project is for reference only. You can rewrite the player based on your business needs.
             // [self->_voicePlayer play];
             // The new version of the sample project provides a new player for reference only. You can rewrite the player based on your business needs.
             [_audioController startPlayer];
         } else if (event == TTS_EVENT_END || event == TTS_EVENT_CANCEL || event == TTS_EVENT_ERROR) {
             loop_in = event;
             if (event == TTS_EVENT_END) {
                 TLog(@"onNuiTtsEventCallback TTS_EVENT_END");
                 // The player provided in the old version of the sample project is for reference only. You can rewrite the player based on your business needs.
                 // Note that this event indicates the completion of speech synthesis, not the completion of playback. Playback completion is notified by the voicePlayer object.
                 // [self->_voicePlayer drain];
                 // The new version of the sample project provides a new player for reference only. You can rewrite the player based on your business needs.
                 // Note that this event indicates the completion of speech synthesis, not the completion of playback. Playback completion is notified by the audioController object.
                 [_audioController drain];
             } else {
                 // The player provided in the old version of the sample project is for reference only. You can rewrite the player based on your business needs.
                 // Stop playback when the broadcast is canceled or an exception occurs.
                 // [self->_voicePlayer stop];
                 // The new version of the sample project provides a new player for reference only. You can rewrite the player based on your business needs.
                 // Stop playback when the broadcast is canceled or an exception occurs.
                 [_audioController stopPlayer];
             }
             if (event == TTS_EVENT_ERROR) {
                 const char *errmsg = [_nui nui_tts_get_param: "error_msg"];
                 TLog(@"tts get errmsg:%s", errmsg);
             }
         }
     }

   • onNuiTtsUserdataCallback: This is the speech synthesis data callback. You can use this callback to write the synthesized data to the player for playback.

     - (void)onNuiTtsUserdataCallback:(char*)info infoLen:(int)info_len buffer:(char*)buffer len:(int)len taskId:(char*)task_id {
         TLog(@"onNuiTtsUserdataCallback info ...");
         if (info_len > 0) {
             TLog(@"onNuiTtsUserdataCallback info text %s. index %d.", info, info_len);
         }
         if (len > 0) {
             // The player provided in the old version of the sample project is for reference only. You can rewrite the player based on your business needs.
             // [_voicePlayer write:(char*)buffer Length:(unsigned int)len];
             // The new version of the sample project provides a new player for reference only. You can rewrite the player based on your business needs.
             [_audioController write:(char*)buffer Length:(unsigned int)len];
         }
     }

   • onNuiTtsLogTrackCallback: SDK internal log callback (added in version 2.6.4).

     -(void)onNuiTtsLogTrackCallback:(NuiSdkLogLevel)level
                          logMessage:(const char *)log {
         TLog(@"onNuiTtsLogTrackCallback log level:%d, message -> %s", level, log);
     }

FAQ

# Other code is omitted
post_install do |installer|
  installer.pods_project.targets.each do |target|
    flutter_additional_ios_build_settings(target)
    target.build_configurations.each do |config|
      config.build_settings["EXCLUDED_ARCHS[sdk=iphonesimulator*]"] = "arm64"
    end
  end
end