Android SDK

Prerequisites

• Review the interface description. For more information, see Interface description.

• Obtain the Appkey for your project. For more information, see Create a project.

• Obtain an access token. For more information, see Overview of obtaining a token.

Download and installation

1. Select and download the mobile SDK.

   Important

   After you download the SDK, you must replace your Alibaba Cloud account information, Appkey, and Token in the sample initialization code to run the sample.

2. Unzip the ZIP package. In the app/libs directory, find the SDK package in AAR format and integrate it into your project as a dependency. To connect using Android C++, find the dynamic libraries and header files in the android_libs and android_include directories of the ZIP package.

3. You can use Android Studio to open the project and view the reference code. The sample code for speech synthesis is in the TtsBasicActivity.java file. After you replace the appkey and token, you can run the file directly.

Key SDK interfaces

• tts_initialize: Initializes the SDK.

  /**
   * Initializes the SDK. The SDK is a singleton. Release the SDK before you initialize it again. Do not call this method in the UI thread to prevent blocking.
   * @param callback: The event listener callback. For more information, see the callbacks described later in this topic.
   * @param ticket: The initialization parameters in a JSON string. For more information, see the description below or the interface description at https://help.aliyun.com/document_detail/173642.html.
   * @param level: The log printing level. A smaller value indicates that more logs are printed.
   * @param save_log: Specifies whether to save logs to a file. The file is stored in the directory specified by the debug_path field in the ticket. Note that log files have no size limit. Storing logs for a long time may fill up your disk.
   * @return: For more information about error codes, see https://help.aliyun.com/document_detail/459864.html.
   */
  public synchronized int tts_initialize(INativeTtsCallback callback,
                                         String ticket,
                                         final Constants.LogLevel level,
                                         boolean save_log)；

  The INativeTtsCallback type includes the following callbacks.

  • onTtsEventCallback: The SDK event callback.

    /**
     * Event callback
     * @param event: The callback event. For more information, see the event list below.
     * @param task_id: The ID of the requested task.
     * @param ret_code: The error code. This parameter is valid when a TTS_EVENT_ERROR event occurs. For more information, see https://help.aliyun.com/document_detail/459864.html.
     */
    void onTtsEventCallback(TtsEvent event, String task_id, int ret_code);

    Event list:

    Name	Description
    TTS_EVENT_START	Speech synthesis starts and is ready for playback.
    TTS_EVENT_END	Speech synthesis is complete. All synthetic data has been returned, but this does not mean that playback has finished.
    TTS_EVENT_CANCEL	Speech synthesis is canceled.
    TTS_EVENT_PAUSE	Speech synthesis is paused.
    TTS_EVENT_RESUME	Speech synthesis is resumed.
    TTS_EVENT_ERROR	An error occurred during speech synthesis. You can get detailed error messages using getparamTts("error_msg").

  • onTtsDataCallback: The synthetic data callback.

    /**
     * Synthetic data callback. If enable_subtitle is enabled, info and data are returned alternately.
     * @param info: If the timestamp feature is used, the timestamp result is returned in JSON format.
     * @param info_len: The data length of the info field. This parameter is not currently used.
     * @param data: The synthetic audio data to be written to the player.
     */
    void onTtsDataCallback(String info, int info_len, byte[] data);

    The following table describes the parameters in the ticket. For an example of how to generate a ticket, see the code example below.

    Parameter	Type	Required	Description
    workspace	String	Yes	The path of the working directory. The SDK reads configuration files from this directory. Read and write permissions are required.
    app_key	String	Yes	The Appkey of the project created in the console.
    token	String	Yes	Make sure that the token is valid and has not expired. You can set the token during initialization or update it by setting parameters.
    device_id	String	Yes	The user-level account ID. Make sure that the ID is unique.
    mode_type	String	Yes	Sets the mode to online speech synthesis. For speech synthesis, you must set this parameter to 2. Otherwise, the program cannot run.
    tts_version	String	Yes	Sets the speech synthesis mode. 1: Long text speech synthesis (more than 300 characters) 0: Short text speech synthesis (300 characters or less)
    custom_params	String	No	To set parameters that are supported by the interaction protocol but not described in the interface description, you can use this universal interface for configuration. The key is custom_params, and the value is a JSON string. For information about how to set this parameter, see the code example.

• setparamTts: Sets TTS parameters.

  /**
   * Sets parameters in key-value pairs. For more information, see https://help.aliyun.com/document_detail/173642.html.
   * @param param: The parameter name. For more information, see the interface description.
   * @param value: The parameter value. For more information, see the interface description.
   * @return: For more information about error codes, see https://help.aliyun.com/document_detail/459864.html.
   */
  public synchronized int setparamTts(String param, String value);

• getparamTts: Retrieves a parameter.

  /**
   * Gets a parameter value.
   * @param param: The parameter name. For more information, see https://help.aliyun.com/document_detail/173642.html.
   * @return: The parameter value.
   */
  public String getparamTts(String param);

• startTts: Starts playback.

  /**
   * Starts a synthesis task.
   * @param priority: The task priority. Use "1".
   * @param taskid: The task ID. You can enter a 32-byte UUID or leave it empty to have the SDK generate one automatically.
   * @param text: The text content to be played.
   * @return: For more information about error codes, see https://help.aliyun.com/document_detail/459864.html.
   */
  public synchronized int startTts(String priority, String taskid, String text);

• cancelTts: Cancels playback.

  /**
   * Cancels a synthesis task.
   * @param taskid: The ID of the task to be stopped. If this is empty, all tasks are canceled.
   * @return: For more information about error codes, see https://help.aliyun.com/document_detail/459864.html.
   */
  public synchronized int cancelTts(String taskid);

• pauseTts: Pauses playback.

  /**
   * Pauses a synthesis task.
   * @return: For more information about error codes, see https://help.aliyun.com/document_detail/459864.html.
   */
  public synchronized int pauseTts();

• resumeTts: Resumes playback.

  /**
   * Resumes the paused task.
   * @return An error code. For more information, see https://help.aliyun.com/document_detail/459864.html.
   */
  public synchronized int resumeTts();

• tts_release: Releases SDK resources.

  /**
   * Releases the SDK.
   * @return: For more information about error codes, see https://help.aliyun.com/document_detail/459864.html.
   */
  public synchronized int tts_release();

Procedure

1. Initialize the SDK and the player component.

2. Set parameters as needed.

3. Call startTts to start playback.

4. In the synthetic data callback, write the data to the player for stream playback.

   If you want to save the complete synthetic audio to a local file, save the audio data to the same file in append mode.

5. Process the callback that indicates speech synthesis is complete.

Code examples

Initialize speech synthesis

// Obtain the resource path, which is the working directory.
//  A working directory is created internally using context.getApplicationContext().getFilesDir().toString() + "/asr_my".
//  For example, /data/user/0/mit.alibaba.nuidemo/files/asr_my.
String workspace = CommonUtils.getModelPath(this);

// Copy assets from nuisdk.aar to the workspace.
CommonUtils.copyAssetsData(this);

// Initialize the SDK.
NativeNui nui_tts_instance = new NativeNui(Constants.ModeType.MODE_TTS);
int ret = nui_tts_instance.tts_initialize(new INativeTtsCallback() {}, genTicket(workspace), Constants.LogLevel.LOG_LEVEL_VERBOSE, true);

The genTicket method generates a JSON string that contains the resource directory and user information. The user information includes the following fields. For information about how to obtain the field values, refer to the interface description document.

/**
 * An example of how to generate a ticket. For more information, see the code example in the demo project.
 */
private String genTicket(String workpath) {
    String str = "";
    try {
        //Note:
        //  For interactive voice response, you must first prepare an account and enable the related services. For more information, see:
        //    https://help.aliyun.com/zh/isi/getting-started/start-here
        //
        //Primary account:
        //  Account (RAM user) information includes an AccessKey ID (ak_id) and an AccessKey secret (ak_secret).
        //  This account information must not be stored in the app code or on the mobile client to prevent financial losses caused by information leaks.
        //
        //STS temporary credentials:
        //  To prevent account information from being leaked when it is sent to the client, Alibaba Cloud provides a temporary access management service named Security Token Service (STS).
        //  STS uses the ak_id and ak_secret to generate a temporary sts_ak_id, sts_ak_secret, and sts_token.
        //  The sts_ prefix is used to distinguish STS temporary credentials from primary account information.
        //What is STS: https://help.aliyun.com/zh/ram/product-overview/what-is-sts
        //STS SDK overview: https://help.aliyun.com/zh/ram/developer-reference/sts-sdk-overview
        //STS Python SDK call example: https://help.aliyun.com/zh/ram/developer-reference/use-the-sts-openapi-example
        //
        //Account requirements:
        //  If you use offline features such as offline speech synthesis and wake-word, you must provide the app_key, ak_id, and ak_secret, or the app_key, sts_ak_id, sts_ak_secret, and sts_token.
        //  If you use online features such as speech synthesis, real-time speech recognition, short sentence recognition, and audio file transcription, you only need to provide the app_key and token.
        JSONObject object = Auth.getTicket(Auth.GetTicketMethod.GET_TOKEN_FROM_SERVER_FOR_ONLINE_FEATURES);
        if (!object.containsKey("token")) {
            Log.e(TAG, "Cannot get token!!!");
        }

        object.put("device_id", "empty_device_id"); // Required. We recommend that you enter a unique ID to help locate problems.

        object.put("url", "wss://nls-gateway.cn-shanghai.aliyuncs.com:443/ws/v1"); // Default
        // The path of the working directory. The SDK reads configuration files from this directory.
        object.put("workspace", workpath); // Required. Read and write permissions are required.

        // Set to online synthesis.
        //  Local = 0,
        //  Mix = 1,  // init local and cloud
        //  Cloud = 2,
        object.put("mode_type", Constants.TtsModeTypeCloud);
        str = object.toString();
    } catch (JSONException e) {
        e.printStackTrace();
    }
    Log.i(TAG, "UserContext:" + str);
    return str;
}

Set parameters as needed

// For more information about the parameters, see https://help.aliyun.com/document_detail/173642.html.

//  For information about online speech synthesis voices, see the Alibaba Cloud official documentation.
//  https://help.aliyun.com/document_detail/84435.html
nui_tts_instance.setparamTts("font_name", "xiaoyun");

//  Set the sample rate for the voice. After setting this, also set the corresponding sample rate for the player. Otherwise, the audio cannot be played correctly.
nui_tts_instance.setparamTts("sample_rate", "16000");

// Switch for the word-level phoneme boundary feature. This parameter is valid only for voices that support this feature. "1" enables the feature, and "0" disables it.
nui_tts_instance.setparamTts("enable_subtitle", "1");

// To set parameters that are not in the documentation, set the key to custom_params and the value to a JSON string.
// nui_tts_instance.setparamTts("custom_params",{\"enable_phoneme_timestamp\":true}");

// Adjust the speech rate.
// nui_tts_instance.setparamTts("speed_level", "1");
// Adjust the pitch.
// nui_tts_instance.setparamTts("pitch_level", "0");
// Adjust the volume.
// nui_tts_instance.setparamTts("volume", "1.0");

//  Supports synthesizing up to 300 characters at a time. A Chinese character, an English letter, or a punctuation mark is counted as one character.
//  Content exceeding 300 characters is truncated. Make sure that the input text is less than 300 characters, excluding SSML tags.
//  The fees for long-text and short-text speech synthesis are different. You must separately enable the long-text speech synthesis service.
//  If you do not need the long-text speech synthesis feature, you can ignore the following operations.
int charNum = nui_tts_instance.getUtf8CharsNum(ttsText);
if (charNum > 300) {
    // If the text exceeds 300 characters, set the mode to long-text speech synthesis.
    nui_tts_instance.setparamTts("tts_version", "1");
} else {
    // If the text does not exceed 300 characters, set the mode to short-text speech synthesis. This is the default setting.
    nui_tts_instance.setparamTts("tts_version", "0");
}

Start speech synthesis

// One task per instance. To process multiple tasks at the same time, start multiple instances.
nui_tts_instance.startTts("1", "", ttsText);

Handle callbacks

• onTtsEventCallback: The speech synthesis event callback. Use this callback to control the player based on the speech synthesis status.

  public void onTtsEventCallback(INativeTtsCallback.TtsEvent event, String task_id, int ret_code) {
      Log.i(TAG, "tts event:" + event + " task id " + task_id + " ret " + ret_code);
      if (event == INativeTtsCallback.TtsEvent.TTS_EVENT_START) {
          mAudioTrack.play();
          Log.i(TAG, "start play");
      } else if (event == INativeTtsCallback.TtsEvent.TTS_EVENT_END) {
          /*
           * Note: The TTS_EVENT_END event indicates that TTS has completed synthesis and returned all audio data through the callback. It does not indicate that the player has finished playing all the audio data.
           */
          Log.i(TAG, "play end");
  
          // Indicates that the data has been pushed. When the player finishes playing, a playOver callback is triggered.
          mAudioTrack.isFinishSend(true);
      } else if (event == TtsEvent.TTS_EVENT_PAUSE) {
          mAudioTrack.pause();
          Log.i(TAG, "play pause");
      } else if (event == TtsEvent.TTS_EVENT_RESUME) {
          mAudioTrack.play();
      } else if (event == TtsEvent.TTS_EVENT_ERROR) {
          // Indicates that the data has been pushed. When the player finishes playing, a playOver callback is triggered.
          mAudioTrack.isFinishSend(true);
  
          String error_msg =  nui_tts_instance.getparamTts("error_msg");
          Log.e(TAG, "TTS_EVENT_ERROR error_code:" + ret_code + " errmsg:" + error_msg);
          ToastText(Utils.getMsgWithErrorCode(ret_code, "error"));
          ToastText("Error code:" + ret_code + " Error message:" + error_msg);
      }
  }

• onTtsDataCallback: The speech synthesis data callback. Use this callback to write the synthetic data to the player for playback.

  If you want to save the complete synthetic audio to a local file, save the audio data to the same file in append mode.

  public void onTtsDataCallback(String info, int info_len, byte[] data) {
      if (info.length() > 0) {
          Log.i(TAG, "info: " + info);
      }
      if (data.length > 0) {
          mAudioTrack.setAudioData(data);
          Log.i(TAG, "write:" + data.length);
      }
  }

Cancel speech synthesis

nui_tts_instance.cancelTts("");

Stop speech synthesis

nui_tts_instance.tts_release();

if (CommonUtils.copyAssetsData(this)) {
Log.i(TAG, "copy assets data done");
} else {
Log.i(TAG, "copy assets failed");
return;
}