Android SDK

Prerequisites

• Read the API reference. For more information, see API reference.

• Obtain a project Appkey. For more information, see Create a project.

• Obtain an AccessKey ID and AccessKey secret. For more information, see Activate the service.

Download and installation

1. Select and download the mobile SDK.

   Important

   After you download the SDK, replace the sample initialization code with your Alibaba Cloud account information and Appkey to run the application.

2. Download a voice package. For more information, see the voice package list in API reference.

   Important

   The SDK and voice packages are separate. You cannot use the SDK directly after you download it. You must also download a voice package and set its storage path.

3. Unzip the package. In the app/libs directory, find the SDK package in AAR format. If you want to use the Android C++ connection type, you can find the dynamic library and header files in the android_libs and android_include directories of the ZIP package.

4. Use Android Studio to open the project. The sample code for speech synthesis is in the TtsLocalActivity.java file.

Key SDK interfaces

• tts_initialize: Initializes the SDK.

  /**
   * Initializes the SDK. Offline synthesis does not support multiple instances. Release the current instance before initializing a new one. Do not call this interface in the UI thread, as it may cause blocking.
   * Initialization is a time-consuming operation. You do not need to perform this operation for every synthesis task. Perform it once at startup and once at exit.
   * @param callback: The event listener callback. See the specific callbacks below.
   * @param ticket: The initialization parameters in a JSON string. See the description below or the API reference at https://help.aliyun.com/document_detail/204185.html.
   * @param level: The log printing level. A smaller value prints more logs.
   * @param save_log: Specifies whether to save logs to a file. The storage directory is specified by the debug_path field in the ticket. Note that log files have no size limit. Storing logs continuously may fill up your disk.
   * @return: See the error codes at https://help.aliyun.com/document_detail/459864.html.
   */
  public synchronized int tts_initialize(INativeTtsCallback callback,
                                         String ticket,
                                         final Constants.LogLevel level,
                                         boolean save_log)；

  The INativeTtsCallback type includes the following callbacks:

  • onTtsEventCallback: The SDK event callback.

    /**
     * Event callback
     * @param event: The callback event. See the event list below.
     * @param task_id: The ID of the request task.
     * @param ret_code: See the error codes. This is valid when a TTS_EVENT_ERROR event occurs. For more information, see https://help.aliyun.com/document_detail/459864.html.
     */
    void onTtsEventCallback(TtsEvent event, String task_id, int ret_code);

    Event list:

    Name	Description
    TTS_EVENT_START	Speech synthesis starts. Ready for playback.
    TTS_EVENT_END	Speech synthesis ends. All synthetic data has been sent. This does not mean playback has finished.
    TTS_EVENT_CANCEL	Speech synthesis is canceled.
    TTS_EVENT_PAUSE	Speech synthesis is paused.
    TTS_EVENT_RESUME	Speech synthesis is resumed.
    TTS_EVENT_ERROR	An error occurred during speech synthesis. You can get detailed error messages using getparamTts("error_msg").

  • onTtsDataCallback: The synthetic data callback.

    /**
     * @param info: When the timestamp feature is enabled, this returns the timestamp result in JSON format.
     * @param info_len: The data length of the info field. Not currently in use.
     * @param data: The synthetic audio data. Write this to the player.
     */
    void onTtsDataCallback(String info, int info_len, byte[] data);

  • onTtsLogTrackCallback: The SDK internal log callback (added in version 2.6.4).

    /**
     * SDK internal log callback. If you override this callback, internal SDK logs that meet the log level are sent through this callback. If multiple instances override this callback, all callbacks receive the SDK's log information.
     * @param level: Internal SDK logs with a level higher than this are sent through this callback.
     * @param log: The specific log content.
     */
    void onTtsLogTrackCallback(Constants.LogLevel level, String log);

  For a description of the ticket content parameters and a generation example, see the following code example. For more information about parameter settings, see the API reference.

  Parameter	Type	Required	Description
  workspace	String	Yes	The path of the working directory. The SDK reads configuration files from this path. Read and write permissions are required.
  app_key	String	Yes	The Appkey of the project created in the console.
  ak_id	String	Yes	The AccessKey ID of your Alibaba Cloud account, which identifies the user. It can also be the AccessKey ID from Alibaba Cloud Security Token Service (STS). For details, see the account authentication description in the code examples.
  ak_secret	String	Yes	The AccessKey secret of your Alibaba Cloud account, which is used to authenticate the user's key. The AccessKey secret must be kept confidential. It can also be the AccessKey secret from Alibaba Cloud STS. For details, see the account authentication description in the code examples.
  token	String	No	The access token. Use this parameter if you are using the AccessKey of an Alibaba Cloud account. If you are using an AccessKey from Alibaba Cloud STS, use sts_token. For details, see the account authentication description in the code examples.
  sts_token	String	No	The Security Token Service (STS) token. Use this parameter if you are using an AccessKey from Alibaba Cloud STS. If you are using the AccessKey of an Alibaba Cloud account, use token. For details, see the account authentication description in the code examples.
  device_id	String	Yes	The user-level account number. This is the basis for billing. If the device_id is a duplicate, it is considered an invalid ID, and offline speech synthesis will not work. If the device_id changes randomly, duplicate billing may occur. Ensure that this device_id is unique and does not change.
  mode_type	String	Yes	Sets the mode to offline speech synthesis. For speech synthesis, this must be set to "0". This setting is important. If omitted, the service will not run.

• setparamTts: Sets parameters.

  /**
   * Sets parameters as key-value pairs. See the API reference: https://help.aliyun.com/document_detail/204185.html
   * @param param: The parameter name. See the API reference.
   * @param value: The parameter value. See the API reference.
   * @return: See the error codes: https://help.aliyun.com/document_detail/459864.html.
   */
  public synchronized int setparamTts(String param, String value);

• getparamTts: Retrieves parameters.

  /**
   * Gets a parameter value.
   * @param param: The parameter name. See the API reference: https://help.aliyun.com/document_detail/204185.html.
   * @return: The parameter value.
   */
  public String getparamTts(String param);

• startTts: Starts synthesis.

  /**
   * Starts a synthesis task.
   * @param priority: The task priority. Use "1".
   * @param taskid: The task ID. You can pass a 32-byte UUID, or pass an empty value to have the SDK generate one automatically.
   * @param text: The text content to be played.
   * @return: See the error codes: https://help.aliyun.com/document_detail/459864.html.
   */
  public synchronized int startTts(String priority, String taskid, String text);

• cancelTts: Cancels synthesis.

  /**
   * Cancels a synthesis task.
   * @param taskid: Pass the ID of the task you want to stop. If this is empty, all tasks are canceled.
   * @return: See the error codes: https://help.aliyun.com/document_detail/459864.html.
   */
  public synchronized int cancelTts(String taskid);

• pauseTts: Pauses synthesis.

  /**
   * Pauses a synthesis task.
   * @return: See the error codes: https://help.aliyun.com/document_detail/459864.html.
   */
  public synchronized int pauseTts();

• resumeTts: Resumes synthesis.

  /**
   * Resumes a paused task.
   * @return: See the error codes: https://help.aliyun.com/document_detail/459864.html.
   */
  public synchronized int resumeTts();

• tts_release: Releases SDK resources.

  /**
   * Releases the SDK.
   * @return: See the error codes: https://help.aliyun.com/document_detail/459864.html.
   */
  public synchronized int tts_release();

Procedure

1. Initialize the SDK and the player component.

2. Set parameters as needed.

3. Call startTts to start synthesis.

4. In the synthetic data callback, write the data to the player for playback. We recommend that you use stream playback.

5. Handle the callback that indicates that speech synthesis is complete.

Code examples

1. Initialize speech synthesis.

   // Set the name of the lowest-level directory in the default destination path.
   // For example, set the current path to /data/user/0/mit.alibaba.nuidemo/files/asr_my.
   // If this interface is not called, the default path is /data/user/0/mit.alibaba.nuidemo/files/asr_my.
   CommonUtils.setTargetDataDir("asr_my");
   
   // Get the default destination path where assets from the current nuisdk.aar can be moved.
   // For example, /data/user/0/mit.alibaba.nuidemo/files/asr_my.
   path = CommonUtils.getModelPath(this);
   Log.i(TAG, "workpath:" + path);
   
   // Actively call to complete the copy of SDK configuration files to the destination CommonUtils.getModelPath(this).
   // For example, the current path is /data/user/0/mit.alibaba.nuidemo/files/asr_my.
   if (CommonUtils.copyTtsAssetsData(this)) {
       Log.i(TAG, "copy assets data done");
   } else {
       Log.e(TAG, "copy assets failed");
       ToastText("Failed to copy resource files from aar assets. Check if the resource files exist. For details, check the logs.");
       return;
   }
   // Note: For example, the authentication file is /data/user/0/mit.alibaba.nuidemo/files/asr_my/tts/ttset.bin.
   // Do not overwrite or delete files in the asr_my directory.
   
   // Initialize the SDK.
   NativeNui nui_tts_instance = new NativeNui(Constants.ModeType.MODE_TTS);
   int ret = nui_tts_instance.tts_initialize(new INativeTtsCallback() {}, genInitParams(path), Constants.LogLevel.LOG_LEVEL_VERBOSE, true);

   The genInitParams method generates a JSON string that contains the resource directory and user information. The user information includes the following fields:

   Note

   • For the premium edition, sdk_code is software_nls_tts_offline.

   • For the standard edition, sdk_code is software_nls_tts_offline_standard.

   /**
    * Example of ticket generation. For details, see the code example in the demo project.
    */
   private String genInitParams(String workpath) {
       String str = "";
       try {
           // Important:
           //  Before using the interactive voice response service, you must prepare an account and activate the relevant services. For specific steps, see:
           //    https://help.aliyun.com/zh/isi/getting-started/start-here
           //
           // Primary account:
           //  Account (RAM user) information mainly includes an AccessKey ID (ak_id) and an AccessKey secret (ak_secret).
           //  This account information must not be stored in the app code or on the mobile client to prevent information leakage and financial loss.
           //
           // STS temporary credentials:
           //  Because sending account information to the client poses a risk of leakage, Alibaba Cloud provides a temporary access management service, Security Token Service (STS).
           //  STS uses the account's ak_id and ak_secret to generate a temporary sts_ak_id, sts_ak_secret, and sts_token by request.
           //  (To distinguish between the primary account information and STS temporary credentials, the prefix sts_ is used for information generated by STS.)
           // What is STS: https://help.aliyun.com/zh/ram/product-overview/what-is-sts
           // STS SDK overview: https://help.aliyun.com/zh/ram/developer-reference/sts-sdk-overview
           // STS Python SDK call example: https://help.aliyun.com/zh/ram/developer-reference/use-the-sts-openapi-example
           //
           // Account requirements:
           //  To use offline features (offline speech synthesis, wake-word), you must provide app_key, ak_id, and ak_secret, or app_key, sts_ak_id, sts_ak_secret, and sts_token.
           //  To use online features (speech synthesis, real-time transcription, short sentence recognition, audio file transcription, etc.), you only need app_key and token.
           JSONObject initObject = Auth.getTicket(Auth.GetTicketMethod.GET_STS_ACCESS_FROM_SERVER_FOR_OFFLINE_FEATURES);
           if (!object.containsKey("token")) {
               Log.e(TAG, "Cannot get token!!!");
           }
         
           object.put("url", "wss://nls-gateway.cn-shanghai.aliyuncs.com:443/ws/v1"); // Default
           // The path of the working directory. The SDK reads configuration files from this path.
           object.put("workspace", workpath); // Required. Read and write permissions are needed.
   
           initObject.put("mode_type", Constants.TtsModeTypeLocal); // Required
   
           // Special note: The ID used for authentication is generated by encrypting a combination of the device_id below and some unique codes from the mobile phone.
           //   Changing the mobile phone or the device_id will trigger re-authentication and billing.
           //   In addition, set a meaningful and unique ID for the device_id below, such as a user account (phone number, International Mobile Equipment Identity (IMEI), etc.).
           //   Passing the same or a randomly changing device_id will lead to authentication failure or duplicate charges.
           object.put("device_id", "empty_device_id"); // Required. We recommend filling in a unique ID to help locate problems.
   
           str = object.toString();
       } catch (JSONException e) {
           e.printStackTrace();
       }
       Log.i(TAG, "UserContext:" + str);
       return str;
   }

2. Set parameters as needed.

   // Set the local voice. If the downloaded aicheng voice package is in the /sdcard/idst/ directory, the command is "/sdcard/idst/aicheng".
   //  Voice packages and the SDK are separate. You must set the voice package first.
   //  To switch voices: The voice packages that the SDK can use are related to the authenticated account and are determined by the usage rights obtained when purchasing the voice package.
   //  If you have purchased aijia, calling in the following way will switch the voice to aijia.
   //  Voice package download address: https://help.aliyun.com/document_detail/204185.html
   //  Listen to voice package samples: https://www.aliyun.com/activity/intelligent/offline_tts
   //  Special note: A voice available for offline speech synthesis may not be available for online speech synthesis, and vice versa.
   
   // The resource directory in the AAR file includes a built-in voice, aijia, at /data/user/0/mit.alibaba.nuidemo/files/asr_my/tts/voices/aijia.
   String fullName = workspace + "/tts/voices/" + mFontName;
   
   // To switch voices: You must enter the full path name.
   Log.i(TAG, "use extend_font_name:" + fullName);
   ret = nui_tts_instance.setparamTts("extend_font_name", fullName);
   if (ret != Constants.NuiResultCode.SUCCESS) {
       Log.e(TAG, "setparamTts extend_font_name " + fullName + " failed, ret:" + ret);
       String errmsg =  nui_tts_instance.getparamTts("error_msg");
       return ret;
   }
   
   //  Set the speech synthesis sample rate for the voice. After setting it, also set the corresponding sample rate for the player, or the audio will not play correctly.
   nui_tts_instance.setparamTts("sample_rate", "16000");
   
   // Adjust the speech rate.
   // nui_tts_instance.setparamTts("speed_level", "1");
   // Adjust the pitch.
   // nui_tts_instance.setparamTts("pitch_level", "0");
   // Adjust the volume.
   // nui_tts_instance.setparamTts("volume", "1.0");

3. Start speech synthesis.

   nui_tts_instance.startTts("1", "", ttsText);

4. Handle callbacks.

   • onTtsEventCallback: The speech synthesis event callback. Control the player based on the speech synthesis status.

     public void onTtsEventCallback(INativeTtsCallback.TtsEvent event, String task_id, int ret_code) {
         Log.i(TAG, "tts event:" + event + " task id " + task_id + " ret " + ret_code);
         if (event == INativeTtsCallback.TtsEvent.TTS_EVENT_START) {
             mAudioTrack.play();
             Log.i(TAG, "start play");
         } else if (event == INativeTtsCallback.TtsEvent.TTS_EVENT_END) {
             /*
              * Note: The TTS_EVENT_END event indicates that TTS has finished synthesizing and has returned all audio data through the callback. It does not mean that the player has finished playing all the audio data.
              */
             Log.i(TAG, "play end");
     
             // Indicates that data push is complete. When the player finishes playing, there will be a playOver callback.
             mAudioTrack.isFinishSend(true);
         } else if (event == TtsEvent.TTS_EVENT_PAUSE) {
             mAudioTrack.pause();
             Log.i(TAG, "play pause");
         } else if (event == TtsEvent.TTS_EVENT_RESUME) {
             mAudioTrack.play();
         } else if (event == TtsEvent.TTS_EVENT_ERROR) {
             // Indicates that data push is complete. When the player finishes playing, there will be a playOver callback.
             mAudioTrack.isFinishSend(true);
     
             String error_msg =  nui_tts_instance.getparamTts("error_msg");
             Log.e(TAG, "TTS_EVENT_ERROR error_code:" + ret_code + " errmsg:" + error_msg);
             ToastText(Utils.getMsgWithErrorCode(ret_code, "error"));
             ToastText("Error code:" + ret_code + " Error message:" + error_msg);
         }
     }

   • onTtsDataCallback: The speech synthesis data callback. Write the synthetic data from the callback to the player for playback.

     public void onTtsDataCallback(String info, int info_len, byte[] data) {
         if (info.length() > 0) {
             Log.i(TAG, "info: " + info);
         }
         if (data.length > 0) {
             mAudioTrack.setAudioData(data);
             Log.i(TAG, "write:" + data.length);
         }
     }

   • onTtsLogTrackCallback: The SDK internal log callback (added in version 2.6.4).

     // SDK internal log callback. If you override this callback, internal SDK logs that meet the log level are sent through this callback.
     // If multiple instances override this callback, all callbacks receive the SDK's log information.
     @Override
     public void onTtsLogTrackCallback(Constants.LogLevel level, String log) {
         Log.i(TAG, "onTtsLogTrackCallback log level:" + level + ", message -> " + log);
     }

5. Cancel speech synthesis.

   nui_tts_instance.cancelTts("");

6. Release speech synthesis resources.

   nui_tts_instance.tts_release();

FAQ