C++ SDK

SDK download

You can obtain the SDK in one of the following two ways.

• Method 1: Obtain the latest source code from GitHub. For detailed compilation and execution instructions, see the following sections or the readme.md file in the source code.

  git clone --depth 1 https://github.com/aliyun/alibabacloud-nls-cpp-sdk

• Method 2: Download the required SDK package from the table below. The SDK source code package contains the original SDK code, which you must compile to generate the library files required for integration. The other SDK packages for specific platforms contain the relevant library and header files and do not require compilation.

  Latest SDK package	Platform	MD5
  alibabacloud-nls-cpp-sdk3.3.0b-master_cbcac53.zip	SDK source code	7257c0998654e611cf2e8ca9867670ef
  NlsCppSdk_Linux-x86_64_3.3.0b_cbcac53.tar.gz	Linux x86_64	9a93df607f26f1558bc1043a425af6d1
  NlsCppSdk_Linux-aarch64_3.1.15_fa30fba.tar.gz	Linux aarch64	76c34a3ab397d7285963a139b9270ff4

  Where:

  • alibabacloud-nls-cpp-sdk<version>-master_<github commit id>.zip is the SDK source code package.

  • NlsCppSdk_<Platform>_<Version number>_<github commit id>.tar.gz is the SDK package required for development on the corresponding platform. For more information, see the readme.md file in the package.

SDK package file description

• scripts/build_linux.sh: An example compilation script for the Linux platform, located in the SDK source code.

• CMakeLists.txt: An example CMakeLists.txt file for a code-based project on the Linux platform, located in the SDK source code.

• demo folder: Contains integration code examples in the SDK package. The following table lists the examples for the Linux platform.

File name	Description
speechRecognizerDemo.cpp	Short sentence recognition demo.
speechSynthesizerDemo.cpp	Speech synthesis demo.
speechTranscriberDemo.cpp	Real-time speech recognition demo.
fileTransferDemo.cpp	Recorded audio file recognition demo.

• resource folder: Contains sample audio files for the Voice Service, located in the SDK source code. You can use these files for functional testing. The files are listed in the following table.

File name	Description
test0.wav test1.wav test2.wav test3.wav	Test audio files (16 kHz sample rate, 16-bit audio bit depth).

• include the following: Contains the SDK header files from the SDK source code, as shown in the following table.

  File name	Description
  nlsClient.h	SDK instance.
  nlsEvent.h	Callback event description.
  nlsGlobal.h	Global SDK header file.
  nlsToken.h	SDK Access Token instance.
  iNlsRequest.h	Base header file for NLS requests.
  speechRecognizerRequest.h	Short sentence recognition.
  speechSynthesizerRequest.h	Speech synthesis and long text-to-speech.
  speechTranscriberRequest.h	Real-time audio stream recognition.
  FileTrans.h	Recorded audio file recognition.

• lib: SDK library files.

• readme.md: SDK instructions.

• release.log: Version guide.

• version: Version number.

Compile and run

Key interfaces

• Basic interfaces

  • NlsClient: The voice processing client. You can use this client for short sentence recognition, real-time speech recognition, and speech synthesis tasks. This client is thread-safe. We recommend that you create only one global instance.

    Interface name	Enabled version	Description
    getInstance	2.x	Gets (creates) an NlsClient instance.
    setLogConfig	2.x	Sets the log file and storage path.
    setDirectHost	3.x	Skips domain name resolution through DNS and directly sets the server IPv4 address. If you call this interface, you must call it before you call startWorkThread.
    setAddrInFamily	3.1.12	Sets the type of the socket address structure. The default value is AF_INET, which returns only IPv4-related address information. You must call this interface before you call startWorkThread.
    setUseSysGetAddrInfo	3.1.13	If the DNS of libevent does not meet your requirements and cannot complete the DNS resolution, you can call this interface to switch to the system's interface. You must call this interface before you call startWorkThread.
    setSyncCallTimeout	3.1.17	Sets the timeout period (in ms) for the synchronous call mode. The default value is 0, which means the synchronous mode is disabled. When you use the synchronous call mode: start() is blocked until a result is received from the server. stop() is blocked until the close() callback is triggered.
    setPreconnectedPool	3.3.0	Sets a pre-connection pool for each domain name URL. Function: Creates a persistent connection pool for a domain name URL. Connections are automatically reused after requests are complete. Reduces the connection time before each request is initiated. Significantly reduces the first-packet latency. This conflicts with the long-lived connection mode and will disable any configured long-lived connection mode. Disabled scenario: Tingwu scenarios. Call constraint: You must call this interface before you call startWorkThread.
    startWorkThread	3.x	Starts the worker threads. The default value is 1, which starts one thread. If you set this parameter to -1, the number of started threads is equal to the number of CPU cores. In high-concurrency scenarios, we recommend that you set this parameter to -1. This can be considered the initialization of the NlsClient instance and must be called.
    releaseInstance	3.x	Destroys the NlsClient object instance.
    getVersion	2.x	Gets the SDK version number.
    createRecognizerRequest	2.x	Creates a short sentence recognition object. It is thread-safe and supports high-concurrency requests.
    releaseRecognizerRequest	2.x	Destroys the short sentence recognition object. You must call this after the closed event of the current request.

  • NlsToken: Creates a Token object to request a TokenId. When you request a new token, you must first obtain a valid timestamp. If the token expires, you must request a new one. Requesting tokens multiple times within the validity period may generate an incorrect TokenId that cannot be used.

    Interface name	Description
    setAccessKeyId	Sets the AccessKey ID of your Alibaba Cloud account.
    setKeySecret	Sets the AccessKey secret of your Alibaba Cloud account.
    setDomain	Sets the domain name. This parameter is optional.
    setServerVersion	Sets the API version. This parameter is optional.
    setServerResourcePath	Sets the service path. This parameter is optional.
    setRegionId	Sets the service region ID. This parameter is optional.
    setAction	Sets the feature. This parameter is optional.
    applyNlsToken	Requests a TokenId.
    getToken	Gets the TokenId.
    getExpireTime	Gets the UNIX timestamp (in seconds) when the token expires.
    getErrorMsg	Gets the error message.

  • NlsEvent: The event object. You can use it to obtain the request status code, the result returned from the cloud, failure information, and more.

    Interface name	Description
    getStatusCode	Gets the status code. A value of 0 or 20000000 indicates success. Otherwise, an error code is returned.
    getErrorMessage	In the TaskFailed callback, gets the error message when an NlsRequest operation fails.
    getTaskId	Gets the TaskId of the task.
    getAllResponse	Gets the recognition result returned from the cloud.
    getResult	Gets the intermediate and final recognition results.

• Recognition interfaces

  SpeechRecognizerRequest: The request object for short sentence recognition. It is used for short audio recognition. For more information about the interface, see the speechRecognizerRequest.h file.

  Interface name	Enabled version	Description
  setOnTaskFailed	2.x	Sets the error callback function.
  setOnRecognitionStarted	2.x	Sets the callback function for when short sentence recognition starts.
  setOnRecognitionResultChanged	2.x	Sets the callback function for intermediate results of short sentence recognition.
  setOnRecognitionCompleted	2.x	Sets the callback function for when the server-side service ends.
  setOnChannelClosed	2.x	Sets the callback function for when the channel is closed.
  setOnMessage	3.1.16	Sets the callback function for server response messages. All callbacks are output from this callback for you to parse. This parameter is optional. After setting it, you must call setEnableOnMessage to enable it.
  setAppKey	2.x	Sets the AppKey.
  setToken	2.x	Token authentication. All requests must be authenticated using the SetToken method before they can be used.
  setUrl	2.x	Sets the service URL.
  setIntermediateResult	2.x	Specifies whether to return intermediate recognition results.
  setPunctuationPrediction	2.x	Specifies whether to add punctuation in post-processing.
  setInverseTextNormalization	2.x	Specifies whether to perform number-to-text conversion in post-processing.
  setEnableVoiceDetection	2.x	Determines whether custom silence detection is enabled.
  setMaxStartSilence	2.x	If this duration is exceeded (no voice is detected after recognition starts), the server sends a TaskFailed event to end the current recognition task.
  setMaxEndSilence	2.x	If this duration is exceeded, the server sends a RecognitionCompleted event to end the current recognition task. Note that subsequent audio will not be recognized.
  setFormat	2.x	Sets the audio data encoding format. Valid values: PCM, OPUS, and OPU. Default value: PCM. We recommend that you use OPUS.
  setSampleRate	2.x	Sets the audio sample rate.
  setCustomizationId	2.x	Sets a custom model.
  setVocabularyId	2.x	Configure General Hotwords.
  setTimeout	2.x	Sets the socket receiving timeout period.
  setOutputFormat	2.x	Sets the encoding format of the output text. Valid values: UTF-8 and GBK.
  setPayloadParam	2.x	Sets parameters. The input parameter is a JSON string.
  setContextParam	2.x	Sets custom user parameters. The input parameter is a JSON string.
  AppendHttpHeaderParam	2.x	Sets custom HTTP header parameters for the WebSocket handshake phase.
  setAudioAddress	3.1.13	Experimental interface. A download link for an audio file that can be accessed over the public network. We recommend that you use Alibaba Cloud OSS.
  setSendTimeout	3.1.14	Sets the sending timeout period. Default value: 5000 ms.
  setRecvTimeout	3.1.14	Sets the receiving timeout period. Default value: 15000 ms. This takes effect only after you call setEnableRecvTimeout to enable it.
  setEnableRecvTimeout	3.1.16	Enables the receiving timeout period. Default value: false. This means the receiving timeout is disabled by default. If enabled, an error is reported if no data is received from the server for a long time.
  getOutputFormat	3.1.16	Gets the configured encoding format of the output text.
  setEnableOnMessage	3.1.16	Enables the callback for messages returned from the server.
  getTaskId	3.1.17	Gets the task_id of the current request.
  start	2.x	Starts the SpeechRecognizerRequest.
  stop	2.x	Confirms the shutdown with the server and stops the connection normally.
  cancel	2.x	Does not confirm the shutdown with the server and closes the connection directly.
  sendAudio	2.x	Sends audio data. We recommend that you send 640 to 16,384 bytes of audio data at a time.

C++ SDK error codes

Server response status codes

For more information about service status codes, see Service status codes.

Code example

#include <string.h>
#include <unistd.h>
#include <pthread.h>
#include <stdlib.h>
#include <ctime>
#include <string>
#include <iostream>
#include <vector>
#include <fstream>
#include <sys/time.h>
#include <errno.h>
#include "nlsClient.h"
#include "nlsEvent.h"
#include "nlsToken.h"
#include "speechRecognizerRequest.h"

#define FRAME_SIZE 3200
#define SAMPLE_RATE 16000

using namespace AlibabaNlsCommon; 
using AlibabaNls::NlsClient; 
using AlibabaNls::NlsEvent; 
using AlibabaNls::LogDebug; 
using AlibabaNls::LogInfo; 
using AlibabaNls::LogError;
using AlibabaNls::SpeechRecognizerRequest;

// Custom thread parameters.
struct ParamStruct {
  std::string fileName;
  std::string appkey; 
  std::string token; 
};
// Custom event callback parameters.
struct ParamCallBack {
 public:
  ParamCallBack() {
    pthread_mutex_init(&mtxWord, NULL);
    pthread_cond_init(&cvWord, NULL);
  };
  ~ParamCallBack() {
    pthread_mutex_destroy(&mtxWord);
    pthread_cond_destroy(&cvWord);
  };

  int userId;
  char userInfo[8];
  pthread_mutex_t mtxWord;
  pthread_cond_t cvWord;
};

/**
 * Maintain a global service authentication token and its validity timestamp.
 * Before each service call, check if the token has expired.
 * If it has expired, regenerate a token using the AccessKey ID and AccessKey Secret,
 * and update the global token and its validity timestamp.
 *
 * For more information about how to obtain a token, see https://help.aliyun.com/document_detail/450514.html
 *
 * Note: Do not regenerate a new token before every service call.
 * Regenerate it only when the token is about to expire. All concurrent service calls can share one token.
 */
std::string g_akId = "";
std::string g_akSecret = "";
std::string g_token = "";
long g_expireTime = -1;
int g_sync_timeout = 0;
struct timeval tv;
struct timeval tv1;

// Regenerate a token using the AccessKey ID and AccessKey Secret, and get its expiration timestamp.
int generateToken(std::string akId, std::string akSecret,
                  std::string* token, long* expireTime) {
  NlsToken nlsTokenRequest;
  nlsTokenRequest.setAccessKeyId(akId); 
  nlsTokenRequest.setKeySecret(akSecret); 

  int ret = nlsTokenRequest.applyNlsToken();
  if (ret < 0) {
    // Get the cause of the failure.
    printf("generateToken Failed, error code:%d msg:%s\n",
        ret, nlsTokenRequest.getErrorMsg());
    return ret;
  }
  *token = nlsTokenRequest.getToken();
  *expireTime = nlsTokenRequest.getExpireTime();
  return 0;
}

/**
 * @brief Gets the delay time for sending audio with sendAudio.
 * @param dataSize The size of the data to be sent.
 * @param sampleRate The sample rate, such as 16k or 8k.
 * @param compressRate The data compression ratio. For example, for 16k Opus encoded audio with a 10:1 compression ratio, this value is 10.
                       For uncompressed data, this value is 1.
 * @return The time to sleep after calling sendAudio.
 * @note For 8k PCM encoded data with 16-bit sampling, we recommend sleeping for 100 ms after sending every 1600 bytes.
         For 16k PCM encoded data with 16-bit sampling, we recommend sleeping for 100 ms after sending every 3200 bytes.
         For other encoding formats (such as OPUS), because the decoded data passed to the SDK is still PCM encoded data,
         you need to send 640 bytes and sleep for 20 ms each time, according to the SDK's OPUS/OPU data length limit.
 */
unsigned int getSendAudioSleepTime(int dataSize,
                                   int sampleRate,
                                   int compressRate) { 
  // Only 16-bit sampling is supported.
  const int sampleBytes = 16;     // Only single-channel is supported.
  const int soundChannel = 1;     // The size of the sampled data per second at the current sample rate and bit depth.
  int bytes = (sampleRate * sampleBytes * soundChannel) / 8;     // The size of the sampled data per millisecond at the current sample rate and bit depth.
  int bytesMs = bytes / 1000;     // Divide the size of the data to be sent by the size of the sampled data per millisecond to get the sleep time.
  int sleepMs = (dataSize * compressRate) / bytesMs;
  return sleepMs; 
}

/**
 * @brief Call start(). After a connection is successfully established with the cloud, the SDK's internal thread reports a started event.
 * @param cbEvent The callback event structure. For more information, see nlsEvent.h.
 * @param cbParam Custom callback parameters. Default value: NULL. You can customize parameters as needed.
 * @return
 */
void OnRecognitionStarted(NlsEvent* cbEvent, void* cbParam) {
  ParamCallBack* tmpParam = (ParamCallBack*)cbParam;
  // This demonstrates how to print or use custom user parameters.
  printf("OnRecognitionStarted: %d, %s\n", tmpParam->userId, tmpParam->userInfo);
  // Get the status code of the message. A value of 0 or 20000000 indicates success. Otherwise, an error code is returned.
  // The task_id of the current task. We recommend that you print this for troubleshooting, as it is the unique identifier for the interaction with the server.
  printf("OnRecognitionStarted: status code=%d, task id=%s\n", cbEvent->getStatusCode(), cbEvent->getTaskId()); 
  // Get all the information returned by the server.
  //printf("OnRecognitionStarted: all response=%s\n", cbEvent->getAllResponse());

  // Notify the sending thread that start() was successful and it can continue sending data.
  pthread_mutex_lock(&(tmpParam->mtxWord));
  pthread_cond_signal(&(tmpParam->cvWord));
  pthread_mutex_unlock(&(tmpParam->mtxWord));
} 

/**
 * @brief Set the parameter to allow returning intermediate results. When the SDK receives an intermediate result from the cloud,
 *        the SDK's internal thread reports a ResultChanged event.
 * @param cbEvent The callback event structure. For more information, see nlsEvent.h.
 * @param cbParam Custom callback parameters. Default value: NULL. You can customize parameters as needed.
 * @return
 */
void OnRecognitionResultChanged(NlsEvent* cbEvent, void* cbParam) {
  ParamCallBack* tmpParam = (ParamCallBack*)cbParam;
  // This demonstrates how to print or use custom user parameters.
  printf("OnRecognitionResultChanged: %d, %s\n", tmpParam->userId, tmpParam->userInfo);         // The task_id of the current task. We recommend that you print this for troubleshooting, as it is the unique identifier for the interaction with the server.
  printf("OnRecognitionResultChanged: status code=%d, task id=%s, result=%s\n", cbEvent->getStatusCode(), cbEvent->getTaskId(), cbEvent->getResult());
  // Get all the information returned by the server.
  //printf("OnRecognitionResultChanged: response=%s\n", cbEvent->getAllResponse());
}

/**
 * @brief When the SDK receives a recognition-ended message from the cloud, the SDK's internal thread reports a Completed event.
 * @note After the Completed event is reported, the SDK internally closes the recognition connection channel.
 *       At this point, calling sendAudio will return a negative value. Stop sending data.
 * @param cbEvent The callback event structure. For more information, see nlsEvent.h.
 * @param cbParam Custom callback parameters. Default value: NULL. You can customize parameters as needed.
 * @return
*/
void OnRecognitionCompleted(NlsEvent* cbEvent, void* cbParam) {
  ParamCallBack* tmpParam = (ParamCallBack*)cbParam;
  // This demonstrates how to print or use custom user parameters.
  printf("OnRecognitionCompleted: %d, %s\n", tmpParam->userId, tmpParam->userInfo);
  // The task_id of the current task. We recommend that you print this for troubleshooting, as it is the unique identifier for the interaction with the server.
  printf("OnRecognitionCompleted: status code=%d, task id=%s, result=%s\n", cbEvent->getStatusCode(), cbEvent->getTaskId(), cbEvent->getResult());
  // Get all the information returned by the server.
  //printf("OnRecognitionCompleted: response=%s\n", cbEvent->getAllResponse());
}

/**
 * @brief When an exception occurs during the recognition process, the SDK's internal thread reports a TaskFailed event.
 * @note After the TaskFailed event is reported, the SDK internally closes the recognition connection channel.
 *       At this point, calling sendAudio will return a negative value. Stop sending data.
 * @param cbEvent The callback event structure. For more information, see nlsEvent.h.
 * @param cbParam Custom callback parameters. Default value: NULL. You can customize parameters as needed.
 * @return
 */
void OnRecognitionTaskFailed(NlsEvent* cbEvent, void* cbParam) {
  ParamCallBack* tmpParam = (ParamCallBack*)cbParam;
  // This demonstrates how to print or use custom user parameters.
  printf("OnRecognitionTaskFailed: %d, %s\n", tmpParam->userId, tmpParam->userInfo);
  // The task_id of the current task. We recommend that you print this for troubleshooting, as it is the unique identifier for the interaction with the server.
  printf("OnRecognitionTaskFailed: status code=%d, task id=%s, error message=%s\n", cbEvent->getStatusCode(), cbEvent->getTaskId(), cbEvent->getErrorMessage());
  // Get all the information returned by the server.
  //printf("OnRecognitionTaskFailed: response=%s\n", cbEvent->getAllResponse());
}

/**
 * @brief When recognition ends or an exception occurs, the connection channel is closed.
 *        The SDK's internal thread reports a ChannelCloseed event.
 * @param cbEvent The callback event structure. For more information, see nlsEvent.h.
 * @param cbParam Custom callback parameters. Default value: NULL. You can customize parameters as needed.
 * @return
 */
void OnRecognitionChannelClosed(NlsEvent* cbEvent, void* cbParam) {
  ParamCallBack* tmpParam = (ParamCallBack*)cbParam;
  // This demonstrates how to print or use custom user parameters.
  printf("OnRecognitionChannelClosed: %d, %s\n", tmpParam->userId, tmpParam->userInfo);         // Get all the information returned by the server.
  printf("OnRecognitionChannelClosed: response=%s\n", cbEvent->getAllResponse());

  // Notify the sending thread that the final recognition result has been returned and stop() can be called.
  pthread_mutex_lock(&(tmpParam->mtxWord));
  pthread_cond_signal(&(tmpParam->cvWord));
  pthread_mutex_unlock(&(tmpParam->mtxWord));
}

/**
 * @brief Worker thread in short-lived connection mode.
 *        It loops as follows:
 *        createRecognizerRequest           <----|
 *                   |                              |
 *           request->start()                       |
 *                   |                              |
 *           request->sendAudio()                   |
 *                   |                              |
 *           request->stop()                        |
 *                   |                              |
 *           Receive OnRecognitionChannelClosed callback     |
 *                   |                              |
 *           releaseRecognizerRequest(request)  ----|
 */
void* pthreadFunction(void* arg) {
  int sleepMs = 0;
  int ret = 0;
  ParamCallBack *cbParam = NULL;

  // Initialize custom callback parameters. The following two variables are for demonstration purposes only and have no effect in this example.
  // After allocating callback parameters on the heap, release them before exiting the thread.
  cbParam = new ParamCallBack();
  cbParam->userId = rand() % 100;
  strcpy(cbParam->userInfo, "User.");
  // 0: Get the token, configuration file, and other parameters from the custom thread parameters.
  ParamStruct *tst = (ParamStruct *) arg; 
  if (tst == NULL) {
    printf("arg is not valid\n"); 
    delete cbParam;
    return NULL;
  } 
  // Open the audio file to get the data.
  std::ifstream fs;
  fs.open(tst->fileName.c_str(), std::ios::binary | std::ios::in);
  if (!fs) {
    printf("%s isn't exist..\n", tst->fileName.c_str());
    return NULL;
  }
  // 1: Create a SpeechRecognizerRequest object for short sentence recognition.
  SpeechRecognizerRequest *request =
      NlsClient::getInstance()->createRecognizerRequest();
  if (request == NULL) {
    printf("createRecognizerRequest failed\n");
    delete cbParam;
    return NULL;
  }

  // Set the callback function for successful start().
  request->setOnRecognitionStarted(OnRecognitionStarted, cbParam);
  // Set the callback function for recognition exceptions.
  request->setOnTaskFailed(OnRecognitionTaskFailed, cbParam);
  // Set the callback function for when the recognition channel is closed.
  request->setOnChannelClosed(OnRecognitionChannelClosed, cbParam);
  // Set the callback function for intermediate results.
  request->setOnRecognitionResultChanged(OnRecognitionResultChanged, cbParam);
  // Set the callback function for when recognition is completed.
  request->setOnRecognitionCompleted(OnRecognitionCompleted, cbParam);

  // Set the AppKey. This is a required parameter.
  request->setAppKey(tst->appkey.c_str());
  // Set the account verification token. This is a required parameter.
  request->setToken(tst->token.c_str());

  // Set the audio data encoding format. This is an optional parameter. PCM and OPUS are supported. Default value: PCM.
  request->setFormat("opus");
  // Set the audio data sample rate. This is an optional parameter. 16000 and 8000 are supported. Default value: 16000.
  request->setSampleRate(SAMPLE_RATE);
  // Set whether to return intermediate recognition results. This is an optional parameter. Default value: false.
  request->setIntermediateResult(true);
  // Set whether to add punctuation in post-processing. This is an optional parameter. Default value: false.
  request->setPunctuationPrediction(true);
  // Set whether to perform ITN in post-processing. This is an optional parameter. Default value: false.
  request->setInverseTextNormalization(true);
  // Enable voice activity detection. Optional. Default value: False.
  //request->setEnableVoiceDetection(true);
  // Maximum allowed initial silence in milliseconds. Optional.
  // If this is exceeded, the server sends a RecognitionCompleted event to end the current recognition.
  // Note: You must first set enable_voice_detection to true.
  //request->setMaxStartSilence(800);
  // Maximum allowed trailing silence in milliseconds. Optional.
  // If this is exceeded, the server sends a RecognitionCompleted event to end the current recognition.
  // Note: You must first set enable_voice_detection to true.
  //request->setMaxEndSilence(800);
  // Custom language model ID. Optional.
  //request->setCustomizationId("TestId_123");
  // Custom vocabulary ID. Optional.
  //request->setVocabularyId("TestId_456");
  // Used to pass some custom, advanced parameter settings in JSON format: {"key": "value"}.
  //request->setPayloadParam("{\"vad_model\": \"farfield\"}");

  struct timespec outtime;
  struct timeval now;
  /*
   * 2. start() can be a synchronous or asynchronous operation. The default is asynchronous. Because the asynchronous mode has a higher threshold for modification to determine if the request is running successfully through callbacks, and some older versions have synchronous interfaces,
   *    both synchronous and asynchronous calling methods are provided to allow for a smoother SDK upgrade.
   *    Asynchronous case: By default, when setSyncCallTimeout() is not called, the start() call returns immediately.
   *            The return value does not indicate that the request has started successfully. You need to wait for a started event to indicate a successful start, or a TaskFailed event to indicate failure.
   *    Synchronous case: Call setSyncCallTimeout() to set the timeout for the synchronous interface and enable synchronous mode. The start() call does not return immediately.
   *            It returns after an internal success (which also triggers a started event callback) or failure (which also triggers a TaskFailed event callback).
   *            This method is convenient for older SDK versions.
   */
  ret = request->start();
  if (ret < 0) {
    printf("start() failed. may be can not connect server. please check network or firewalld\n");
    NlsClient::getInstance()->releaseRecognizerRequest(request); // start() failed. Release the request object.
    delete cbParam;
    return NULL;
  } else {
    if (g_sync_timeout == 0) {
      /*
       * 2.1. g_sync_timeout is 0, meaning setSyncCallTimeout() was not called by default. start() is called asynchronously.
       *      You need to wait for a started event to indicate a successful start, or a TaskFailed event to indicate failure.
       * 
       * Wait for the started event to be returned to indicate that start() was successful, then send the audio data.
       * The voice server may not be able to process the current request in time, causing no callback to be returned within 10s.
       * After 10s, a TaskFailed callback is returned, so a timeout mechanism is needed.
       */
      printf("wait started callback.\n");

      // The voice server may not be able to process the current request in time, causing no callback to be returned within 10s.
      // After 10s, a TaskFailed callback is returned, so a timeout mechanism is needed.
      gettimeofday(&now, NULL);
      outtime.tv_sec = now.tv_sec + 5;
      outtime.tv_nsec = now.tv_usec * 1000;
      pthread_mutex_lock(&(cbParam->mtxWord));
      if (ETIMEDOUT == pthread_cond_timedwait(&(cbParam->cvWord), &(cbParam->mtxWord), &outtime)) {
        printf("start timeout.\n");
        pthread_mutex_unlock(&(cbParam->mtxWord));
        request->cancel();
        NlsClient::getInstance()->releaseRecognizerRequest(request);
        delete cbParam;
        return NULL;
      }
      pthread_mutex_unlock(&(cbParam->mtxWord));
    } else {
       /*
        * 2.2. g_sync_timeout is greater than 0, meaning setSyncCallTimeout() was called. start() is called synchronously.
        *      A return value of 0 indicates a successful start.
        */ 
    }
  }

  /*
   * 3. Get audio data from the file and send it in a loop.
   */
  while (!fs.eof()) {
    uint8_t data[FRAME_SIZE] = {0};
    fs.read((char *) data, sizeof(uint8_t) * FRAME_SIZE); 
    size_t nlen = fs.gcount();
    if (nlen <= 0) {
      continue;
    }

    /*
     * 3.1. Send audio data: sendAudio is an asynchronous operation. A negative return value indicates a sending failure, and you need to stop sending.
     *      A value greater than 0 indicates success.
     *      If you want to upload audio data in the traffic-saving Opus format, pass ENCODER_OPU/ENCODER_OPUS as the third parameter.
     *
     * In ENCODER_OPU/ENCODER_OPUS mode, some CPU resources will be used for audio compression.
     */
    ret = request->sendAudio(data, nlen, ENCODER_OPUS); 
    if (ret < 0) {
      // Sending failed. Exit the data sending loop.
      printf("send data fail.\n"); 
      break;
    } 

    /*
     * In actual use, audio data is real-time, so you don't need to use sleep to control the rate. You can send it directly.
     * Here, we use audio data from a file for simulation, so we need to control the rate to simulate a real recording scenario.
     */
    sleepMs = getSendAudioSleepTime(nlen, SAMPLE_RATE, 1); // Get the sleep time based on the data size, sample rate, and data compression ratio.
    /*
     * Audio data sending delay control. No sleep is needed in actual use.
     */
    usleep(sleepMs * 1000);
  } // while

  printf("sendAudio done.\n");
  // 5: Close the audio file.
  fs.close();
  
  /*
   * 4. Notify the cloud that data sending is complete.
   *    stop() can be a synchronous or asynchronous operation. The default is asynchronous. Because the asynchronous mode has a higher threshold for modification to determine if the request is running successfully through callbacks, and some older versions have synchronous interfaces,
   *    both synchronous and asynchronous calling methods are provided to allow for a smoother SDK upgrade.
   *    Asynchronous case: By default, when setSyncCallTimeout() is not called, the stop() call returns immediately.
   *            The return value does not indicate that the request has ended successfully. You need to wait for a closed event to indicate the end.
   *    Synchronous case: Call setSyncCallTimeout() to set the timeout for the synchronous interface and enable synchronous mode. The stop() call does not return immediately.
   *            It returns after the internal work is completed and a closed event callback is triggered.
   *            This method is convenient for older SDK versions.
   */
  ret = request->stop();
  if (ret == 0) {
    if (g_sync_timeout == 0) {
      /*
       * 4.1. g_sync_timeout is 0, meaning setSyncCallTimeout() was not called by default. stop() is called asynchronously.
       *      You need to wait for a closed event to indicate a successful stop, or a TaskFailed event to indicate failure.
       * 
       * Wait for the closed event to be returned to indicate that stop() was successful before releasing.
       * The voice server may not be able to process the current request in time, causing no callback to be returned within 10s.
       * After 10s, a TaskFailed callback is returned, so a timeout mechanism is needed.
       */
      // Wait for the closed event before releasing, otherwise a crash may occur.
      // If you have called setSyncCallTimeout() to enable synchronous calling mode, you do not need to wait for the closed event.
      std::cout << "wait closed callback." << std::endl;
        
      printf("wait closed callback.\n");
    
      // The voice server may not be able to process the current request in time, causing no callback to be returned within 10s.
      // After 10s, a TaskFailed callback is returned with the error message:
      // "Gateway:IDLE_TIMEOUT:Websocket session is idle for too long time, the last directive is 'StopRecognition'!"
      // So a timeout mechanism is needed.
      gettimeofday(&now, NULL);
      outtime.tv_sec = now.tv_sec + 5;
      outtime.tv_nsec = now.tv_usec * 1000;
      // Wait for the closed event before releasing.
      pthread_mutex_lock(&(cbParam->mtxWord));
      if (ETIMEDOUT == pthread_cond_timedwait(&(cbParam->cvWord), &(cbParam->mtxWord), &outtime)) {
        printf("stop timeout\n");
        pthread_mutex_unlock(&(cbParam->mtxWord));
        NlsClient::getInstance()->releaseRecognizerRequest(request);
        delete cbParam;
        return NULL;
      }
      pthread_mutex_unlock(&(cbParam->mtxWord));
    } else {
      /*
       * 4.2. g_sync_timeout is greater than 0, meaning setSyncCallTimeout() was called. stop() is called synchronously.
       *      A return value of 0 indicates a successful start.
       */
    }
  } else {
    printf("stop ret is %d\n", ret);
  }

  /*
   * 5. Release the current request after all work is done.
   *    Please release it after the closed event (to confirm all work is done), otherwise it may disrupt the internal state machine and forcibly unload the running request.
   */
  NlsClient::getInstance()->releaseRecognizerRequest(request);
  delete cbParam;
  return NULL;
}

// Recognize a single audio file.
int speechRecognizerFile(const char* appkey) {
  // Get the current system timestamp to check if the token has expired.
  std::time_t curTime = std::time(0);
  if (g_expireTime - curTime < 10) {
    printf("the token will be expired, please generate new token by AccessKey-ID and AccessKey-Secret.\n");
    if (generateToken(g_akId, g_akSecret, &g_token, &g_expireTime) < 0) {
      return -1;
    } 
  }

  ParamStruct pa;
  pa.token = g_token;
  pa.appkey = appkey;
  pa.fileName = "test0.wav";

  // Start a worker thread for a single recognition task.
  pthread_t pthreadId;
  pthread_create(&pthreadId, NULL, &pthreadFunction, (void *)&pa);
  pthread_join(pthreadId, NULL);
  return 0;
}

// Recognize multiple audio files.
// In the SDK, multi-threading means one thread corresponds to one audio data source, not multiple threads for one audio data source.
// The example code starts two threads to recognize two files simultaneously.
// For free-tier users, the number of concurrent connections cannot exceed 2.
#define AUDIO_FILE_NUMS 2
#define AUDIO_FILE_NAME_LENGTH 32
int speechRecognizerMultFile(const char* appkey) {
  // Get the current system timestamp to check if the token has expired.
  std::time_t curTime = std::time(0);
  if (g_expireTime - curTime < 10) {
    printf("the token will be expired, please generate new token by AccessKey-ID and AccessKey-Secret.\n");
    if (generateToken(g_akId, g_akSecret, &g_token, &g_expireTime) < 0) {
      return -1;
    }
  }

  char audioFileNames[AUDIO_FILE_NUMS][AUDIO_FILE_NAME_LENGTH] = {
      "test0.wav", 
      "test1.wav"
  };
  ParamStruct pa[AUDIO_FILE_NUMS];
  for (int i = 0; i < AUDIO_FILE_NUMS; i ++) {
    pa[i].token = g_token; 
    pa[i].appkey = appkey;
    pa[i].fileName = audioFileNames[i]; 
  } 

  // Start 2 worker threads to recognize 2 audio files simultaneously.
  std::vector<pthread_t> pthreadId(AUDIO_FILE_NUMS);
  for (int j = 0; j < AUDIO_FILE_NUMS; j++) {
    pthread_create(&pthreadId[j], NULL, &pthreadFunction, (void *)&(pa[j]));
  }
  for (int j = 0; j < AUDIO_FILE_NUMS; j++) {
    pthread_join(pthreadId[j], NULL);
  }
  return 0;
}

int main(int argc, char* argv[]) {
  printf("Usage: ./demo <your appkey> <your AccessKey ID> <your AccessKey Secret>\n");

  std::string appkey = getenv("NLS_APPKEY_ENV");
  g_akId = getenv("NLS_AK_ENV");
  g_akSecret = getenv("NLS_SK_ENV");

  // Set the SDK output log as needed. Optional.
  // This indicates that the SDK log is output to log-recognizer.txt.
  // LogDebug means all levels of logs are output. LogDebug, LogInfo, LogWarning, and LogError are supported.
  // 400 means a single file is 400 MB. 50 means 50 log files are recorded in a loop.
  int ret = NlsClient::getInstance()->setLogConfig(
      "log-recognizer", LogDebug, 400, 50);
  if (ret < 0) {
    printf("set log failed.\n");
    return -1;
  }

  // Set the socket address type required by the runtime environment. Default value: AF_INET.
  // Must be called before startWorkThread().
  //NlsClient::getInstance()->setAddrInFamily("AF_INET");

  // In a private cloud deployment, you can set a direct IP connection.
  // Must be called before startWorkThread().
  //NlsClient::getInstance()->setDirectHost("106.15.83.44");

  // Some devices may not be able to obtain a usable IP through the SDK's DNS even after setting the DNS.
  // You can call this interface to actively enable the system's getaddrinfo to solve this problem.
  //NlsClient::getInstance()->setUseSysGetAddrInfo(true);

  // g_sync_timeout is 0, meaning setSyncCallTimeout() was not called by default.
  // Asynchronous calling method:
  //   start(): You need to wait for a started event to indicate a successful start, or a TaskFailed event to indicate failure.
  //   stop(): You need to wait for a closed event to indicate that the interaction is complete.
  // Synchronous calling method:
  //   The return of the start()/stop() call indicates that the interaction has started/ended.
  if (g_sync_timeout > 0) {
    NlsClient::getInstance()->setSyncCallTimeout(g_sync_timeout);
  }
    
  // Start the worker thread. This function must be called before creating and starting a request. It can be understood as initializing NlsClient.
  // If the input parameter is negative, it starts the number of available cores in the current system.
  // For concurrency below 200, we recommend an input parameter of 1. For higher concurrency, see the readme for recommendations.
  NlsClient::getInstance()->startWorkThread(1);

  // Recognize a single audio file.
  speechRecognizerFile(appkey.c_str());
  // Concurrently recognize multiple audio files.
  //speechRecognizerMultFile(appkey.c_str());

  // After all tasks are completed, release nlsClient before the process exits.
  // Note that releaseInstance() is not thread-safe. Make sure all requests have stopped before releasing.
  NlsClient::releaseInstance();

  return 0;
}

FAQ