C++ SDK

快速接入

准备 Linux 编译环境，获取 SDK，配置访问凭证，然后运行完整示例。请求参数和返回事件的详细定义，请参见实时语音识别接口说明。

原 API 2.0 已下线，与本文使用的 API 3.1 定义不同。

准备环境

Linux 编译工具最低版本要求：CMake 3.0、glibc 2.5、GCC 4.8.5。

本示例使用 Linux x86_64、GCC 10.2.1 和 glibc 2.32。

获取和安装 SDK

git clone --depth 1 https://github.com/aliyun/alibabacloud-nls-cpp-sdk

./scripts/build_linux.sh

配置访问凭证

运行示例前，为当前进程设置以下环境变量。AppKey 和 AccessKey 必须属于实际使用的服务账号。不要在代码或日志中写入凭证明文。

export NLS_APPKEY_ENV='<appkey>'
export NLS_AK_ENV='<accesskey-id>'
export NLS_SK_ENV='<accesskey-secret>'

在 IDE 中运行时，也需在运行配置中设置这些环境变量。示例通过 NlsToken 获取访问令牌，缓存令牌及过期时间，并在临近过期时更新；多个并发请求可共用有效令牌。

代码示例

以下完整示例包含客户端初始化、Token 获取、异步事件处理、音频发送及资源释放。使用 SDK 包中的 test0.wav，并将文件放在程序工作目录；并发示例还使用 test1.wav。

示例中使用的音频文件为16000 Hz采样率，控制台设置的模型为通用模型。如果使用其他音频，请设置为支持该音频场景的模型。关于模型设置，请参见管理项目。

示例等待启动事件后发送音频，收到失败事件后停止发送，并将失败状态传递为非零进程退出码。正常完成时，依次收到 TranscriptionCompleted 和通道关闭事件。

#include <string.h>
#include <unistd.h>
#include <pthread.h>
#include <stdlib.h>
#include <ctime>
#include <string>
#include <iostream>
#include <vector>
#include <fstream>
#include <sys/time.h>
#include <errno.h>
#include <atomic>
#include "nlsClient.h"
#include "nlsEvent.h"
#include "nlsToken.h"
#include "speechTranscriberRequest.h"

#define FRAME_SIZE 3200
#define SAMPLE_RATE 16000

using namespace AlibabaNlsCommon; 
using AlibabaNls::NlsClient; 
using AlibabaNls::NlsEvent; 
using AlibabaNls::LogDebug; 
using AlibabaNls::LogInfo; 
using AlibabaNls::LogError;
using AlibabaNls::SpeechTranscriberRequest;

struct ParamStruct {
  std::string fileName;
  std::string appkey; 
  std::string token;
  int result = -1; 
};

struct ParamCallBack {
 public:
  ParamCallBack() {
    pthread_mutex_init(&mtxWord, NULL);
    pthread_cond_init(&cvWord, NULL);
  };
  ~ParamCallBack() {
    pthread_mutex_destroy(&mtxWord);
    pthread_cond_destroy(&cvWord);
  };

  std::atomic<bool> started{false}, closed{false}, failed{false}, completed{false};
  int userId;
  char userInfo[8];
  pthread_mutex_t mtxWord;
  pthread_cond_t cvWord;
};

std::string g_akId = "";
std::string g_akSecret = "";
std::string g_token = "";
long g_expireTime = -1;
int g_sync_timeout = 0;
struct timeval tv;
struct timeval tv1;

int generateToken(std::string akId, std::string akSecret,
                  std::string* token, long* expireTime) {
  NlsToken nlsTokenRequest;
  nlsTokenRequest.setAccessKeyId(akId); 
  nlsTokenRequest.setKeySecret(akSecret); 

  int ret = nlsTokenRequest.applyNlsToken();
  if (ret < 0) {

    printf("generateToken Failed, error code:%d msg:%s\n",
        ret, nlsTokenRequest.getErrorMsg());
    return ret;
  }
  *token = nlsTokenRequest.getToken();
  *expireTime = nlsTokenRequest.getExpireTime();
  return 0;
}

unsigned int getSendAudioSleepTime(int dataSize,
                                   int sampleRate,
                                   int compressRate) { 

  const int sampleBytes = 16;     
  const int soundChannel = 1;     
  int bytes = (sampleRate * sampleBytes * soundChannel) / 8;     
  int bytesMs = bytes / 1000;     
  int sleepMs = (dataSize * compressRate) / bytesMs;
  return sleepMs; 
}

void onTranscriptionStarted(NlsEvent* cbEvent, void* cbParam) {
  ParamCallBack* tmpParam = (ParamCallBack*)cbParam;

  printf("onTranscriptionStarted: %d\n", tmpParam->userId);

  printf("onTranscriptionStarted: status code=%d, task id=%s\n",
      cbEvent->getStatusCode(), cbEvent->getTaskId());

  pthread_mutex_lock(&(tmpParam->mtxWord));
  tmpParam->started = true;
  pthread_cond_signal(&(tmpParam->cvWord));
  pthread_mutex_unlock(&(tmpParam->mtxWord));
} 

void onSentenceBegin(NlsEvent* cbEvent, void* cbParam) {
  ParamCallBack* tmpParam = (ParamCallBack*)cbParam;

  printf("onSentenceBegin: %d\n", tmpParam->userId);
  printf("onSentenceBegin: status code=%d, task id=%s, index=%d, time=%d\n",
      cbEvent->getStatusCode(), cbEvent->getTaskId(),
      cbEvent->getSentenceIndex(), 
      cbEvent->getSentenceTime() 
  );

}

void onSentenceEnd(NlsEvent* cbEvent, void* cbParam) {
  ParamCallBack* tmpParam = (ParamCallBack*)cbParam;

  printf("onSentenceEnd: %d\n", tmpParam->userId);
  printf("onSentenceEnd: status code=%d, task id=%s, index=%d, time=%d, begin_time=%d, result=%s\n",
      cbEvent->getStatusCode(),
      cbEvent->getTaskId(),
      cbEvent->getSentenceIndex(), 
      cbEvent->getSentenceTime(), 
      cbEvent->getSentenceBeginTime(), 
      cbEvent->getResult()    
  );

}

void onTranscriptionResultChanged(NlsEvent* cbEvent, void* cbParam) {
  ParamCallBack* tmpParam = (ParamCallBack*)cbParam;

  printf("onTranscriptionResultChanged: %d\n", tmpParam->userId);
  printf("onTranscriptionResultChanged: status code=%d, task id=%s, index=%d, time=%d, result=%s\n",
      cbEvent->getStatusCode(),
      cbEvent->getTaskId(),
      cbEvent->getSentenceIndex(), 
      cbEvent->getSentenceTime(), 
      cbEvent->getResult()    
  );

}

void onTranscriptionCompleted(NlsEvent* cbEvent, void* cbParam) {
  ParamCallBack* tmpParam = (ParamCallBack*)cbParam;
  tmpParam->completed = true;

  printf("onTranscriptionCompleted: %d\n", tmpParam->userId);
  printf("onTranscriptionCompleted: status code=%d, task id=%s\n",
      cbEvent->getStatusCode(),
      cbEvent->getTaskId());
}

void onTaskFailed(NlsEvent* cbEvent, void* cbParam) { 
  ParamCallBack* tmpParam = (ParamCallBack*)cbParam;
  pthread_mutex_lock(&tmpParam->mtxWord);
  tmpParam->failed = true;
  pthread_cond_broadcast(&tmpParam->cvWord);
  pthread_mutex_unlock(&tmpParam->mtxWord); 

  printf("onTaskFailed: %d\n", tmpParam->userId); 
  printf("onTaskFailed: status code=%d, task id=%s, error message=%s\n", 
      cbEvent->getStatusCode(), 
      cbEvent->getTaskId(), 
      cbEvent->getErrorMessage()
  );     

}

void onChannelClosed(NlsEvent* cbEvent, void* cbParam) {     
  ParamCallBack* tmpParam = (ParamCallBack*)cbParam;     

  printf("onChannelClosed: %d, %s\n", tmpParam->userId, tmpParam->userInfo); 
  printf("onChannelClosed: response=%s\n", cbEvent->getAllResponse());

  pthread_mutex_lock(&(tmpParam->mtxWord));
  tmpParam->closed = true;
  pthread_cond_broadcast(&(tmpParam->cvWord));
  pthread_mutex_unlock(&(tmpParam->mtxWord));
}

void* pthreadFunction(void* arg) {
  int sleepMs = 0;
  int ret = 0;
  ParamCallBack *cbParam = NULL;

  cbParam = new ParamCallBack();
  cbParam->userId = rand() % 100;
  strcpy(cbParam->userInfo, "User.");

  ParamStruct *tst = (ParamStruct *) arg; 
  if (tst == NULL) {
    printf("arg is not valid\n"); 
    delete cbParam;
    return NULL;
  } 

  std::ifstream fs;
  fs.open(tst->fileName.c_str(), std::ios::binary | std::ios::in);
  if (!fs) {
    printf("Cannot open %s\n", tst->fileName.c_str());
    delete cbParam;
    return NULL;
  }
  
  SpeechTranscriberRequest* request =
      NlsClient::getInstance()->createTranscriberRequest();
  if (request == NULL) {
    printf("createTranscriberRequest failed\n");
    delete cbParam;
    return NULL;
  }

  request->setOnTranscriptionStarted(onTranscriptionStarted, cbParam);

  request->setOnTranscriptionResultChanged(onTranscriptionResultChanged, cbParam);

  request->setOnTranscriptionCompleted(onTranscriptionCompleted, cbParam);

  request->setOnSentenceBegin(onSentenceBegin, cbParam);

  request->setOnSentenceEnd(onSentenceEnd, cbParam);

  request->setOnTaskFailed(onTaskFailed, cbParam);

  request->setOnChannelClosed(onChannelClosed, cbParam);

  request->setAppKey(tst->appkey.c_str());

  request->setToken(tst->token.c_str());

  request->setUrl("wss://nls-gateway-cn-shanghai.aliyuncs.com/ws/v1");
  request->setFormat("opus");

  request->setSampleRate(SAMPLE_RATE);

  request->setIntermediateResult(true);

  request->setPunctuationPrediction(true);

  request->setInverseTextNormalization(true);

  struct timespec outtime;
  struct timeval now;
  
  ret = request->start();
  if (ret < 0) {
    printf("start() failed. may be can not connect server. please check network or firewalld\n");
    NlsClient::getInstance()->releaseTranscriberRequest(request); 
    delete cbParam;
    return NULL;
  } else {
    if (g_sync_timeout == 0) {

      printf("wait started callback.\n");

      gettimeofday(&now, NULL);
      outtime.tv_sec = now.tv_sec + 5;
      outtime.tv_nsec = now.tv_usec * 1000;
      pthread_mutex_lock(&(cbParam->mtxWord));
      while (!cbParam->started && !cbParam->failed && !cbParam->closed) {
      if (ETIMEDOUT == pthread_cond_timedwait(&(cbParam->cvWord), &(cbParam->mtxWord), &outtime)) {
        printf("start timeout.\n");
        pthread_mutex_unlock(&(cbParam->mtxWord));
        request->cancel();
        NlsClient::getInstance()->releaseTranscriberRequest(request);
        delete cbParam;
        return NULL;
      }
      }
      pthread_mutex_unlock(&(cbParam->mtxWord));
    } else {
      
    }
  }

  while (cbParam->started && !cbParam->failed && !cbParam->closed && !fs.eof()) {
    uint8_t data[FRAME_SIZE] = {0};
    fs.read((char *) data, sizeof(uint8_t) * FRAME_SIZE); 
    size_t nlen = fs.gcount();
    if (nlen <= 0) {
      continue;
    }

    ret = request->sendAudio(data, nlen, ENCODER_OPUS); 
    if (ret < 0) {

      printf("send data fail.\n");
      cbParam->failed = true; 
      break;
    } 

    sleepMs = getSendAudioSleepTime(nlen, SAMPLE_RATE, 1); 
    
    usleep(sleepMs * 1000);
  } 

  printf("sendAudio done.\n");

  fs.close();

  ret = cbParam->failed || cbParam->closed ? -1 : request->stop();
  if (ret == 0) {
      if (g_sync_timeout == 0) {

        printf("wait closed callback.\n");

        gettimeofday(&now, NULL);
        outtime.tv_sec = now.tv_sec + 5;
        outtime.tv_nsec = now.tv_usec * 1000;

        pthread_mutex_lock(&(cbParam->mtxWord));
        while (!cbParam->closed) {
        if (ETIMEDOUT == pthread_cond_timedwait(&(cbParam->cvWord), &(cbParam->mtxWord), &outtime)) {
          printf("stop timeout\n");
          pthread_mutex_unlock(&(cbParam->mtxWord));
          NlsClient::getInstance()->releaseTranscriberRequest(request);
          delete cbParam;
          return NULL;
        }
        }
        pthread_mutex_unlock(&(cbParam->mtxWord));
      } else {
        
      }
  } else {
    printf("stop ret is %d\n", ret);
    cbParam->failed = true;
    if (!cbParam->closed) request->cancel();
  }

  NlsClient::getInstance()->releaseTranscriberRequest(request);
  tst->result = (!cbParam->failed && cbParam->completed) ? 0 : -1;
  delete cbParam;
  return NULL;
}

int speechTranscriberFile(const char* appkey) {

  std::time_t curTime = std::time(0);
  if (g_expireTime - curTime < 10) {
    printf("the token will be expired, please generate new token by AccessKey-ID and AccessKey-Secret.\n");
    if (generateToken(g_akId, g_akSecret, &g_token, &g_expireTime) < 0) {
      return -1;
    } 
  }

  ParamStruct pa;
  pa.token = g_token;
  pa.appkey = appkey;
  pa.fileName = "test0.wav";

  pthread_t pthreadId;
  if (pthread_create(&pthreadId, NULL, &pthreadFunction, (void *)&pa) != 0) return -1;
  pthread_join(pthreadId, NULL);
  return pa.result;
}

#define AUDIO_FILE_NUMS 2
#define AUDIO_FILE_NAME_LENGTH 32
int speechTranscriberMultFile(const char* appkey) {

  std::time_t curTime = std::time(0);
  if (g_expireTime - curTime < 10) {
    printf("the token will be expired, please generate new token by AccessKey-ID and AccessKey-Secret.\n");
    if (generateToken(g_akId, g_akSecret, &g_token, &g_expireTime) < 0) {
      return -1;
    }
  }

  char audioFileNames[AUDIO_FILE_NUMS][AUDIO_FILE_NAME_LENGTH] = {
      "test0.wav", 
      "test1.wav"
  };
  ParamStruct pa[AUDIO_FILE_NUMS];
  for (int i = 0; i < AUDIO_FILE_NUMS; i ++) {
    pa[i].token = g_token; 
    pa[i].appkey = appkey;
    pa[i].fileName = audioFileNames[i]; 
  } 

  std::vector<pthread_t> pthreadId(AUDIO_FILE_NUMS);
  int created = 0;
  for (; created < AUDIO_FILE_NUMS; ++created) {
    if (pthread_create(&pthreadId[created], NULL, &pthreadFunction, &pa[created]) != 0) break;
  }
  int result = created == AUDIO_FILE_NUMS ? 0 : -1;
  for (int j = 0; j < created; ++j) {
    pthread_join(pthreadId[j], NULL);
    if (pa[j].result != 0) result = -1;
  }
  return result;
}

int main(int argc, char* argv[]) {
  const char* appkeyEnv = getenv("NLS_APPKEY_ENV");
  const char* akEnv = getenv("NLS_AK_ENV");
  const char* skEnv = getenv("NLS_SK_ENV");
  if (!appkeyEnv || !*appkeyEnv || !akEnv || !*akEnv || !skEnv || !*skEnv) {
    std::cerr << "Set NLS_APPKEY_ENV, NLS_AK_ENV, and NLS_SK_ENV." << std::endl;
    return EXIT_FAILURE;
  }
  std::string appkey = appkeyEnv;
  g_akId = akEnv;
  g_akSecret = skEnv;

  int ret = NlsClient::getInstance()->setLogConfig(
      "log-transcriber", LogError, 10, 3);
  if (ret < 0) {
    printf("set log failed.\n");
    NlsClient::releaseInstance();
    return EXIT_FAILURE;
  }

  if (g_sync_timeout > 0) {
    NlsClient::getInstance()->setSyncCallTimeout(g_sync_timeout);
  }

  NlsClient::getInstance()->startWorkThread(1);

  int result = speechTranscriberFile(appkey.c_str());
  // To transcribe two files concurrently, replace the preceding call with:
  // int result = speechTranscriberMultFile(appkey.c_str());

  NlsClient::releaseInstance();

  return result == 0 ? EXIT_SUCCESS : EXIT_FAILURE;
}

将 SDK 的 include 和 lib 目录加入项目编译、链接配置。以当前目录下的 NlsSdk3.X_LINUX 为例，示例文件 demo.cpp 可使用以下命令编译：

g++ -std=c++11 -D_GLIBCXX_USE_CXX11_ABI=1 demo.cpp   -I NlsSdk3.X_LINUX/include -L NlsSdk3.X_LINUX/lib   -lalibabacloud-idst-speech -lpthread -o demo
export LD_LIBRARY_PATH="$PWD/NlsSdk3.X_LINUX/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"

使用说明

• 示例按 16000 Hz、16 位、单声道音频计算发送间隔。8 kHz PCM 每发送 1600 字节可等待 100 ms，16 kHz PCM 每发送 3200 字节可等待 100 ms。OPUS/OPU 模式传给 SDK 的仍是 PCM 数据；建议为每次 640 字节、等待 20 ms。实时录音按采集速率发送，文件模拟录音时才需要控制发送速率。

• sendAudio 返回负值时停止发送；收到 TranscriptionCompleted 或 TaskFailed 后，SDK 关闭通道，不再继续发送音频。

• 异步模式下，start() 返回不代表启动成功，应等待启动事件或失败事件；stop() 返回不代表交互结束，应等待通道关闭事件。调用 setSyncCallTimeout 后启用同步模式。

• SDK 多线程指一个音频源对应一个线程，不是多个线程同时发送同一音频源。完整示例提供单文件及双文件并发入口；所有请求结束后再释放全局客户端，releaseInstance() 非线程安全。

免费用户并发连接不能超过 2 个。工作线程数在 200 并发以下建议设置为 1，更高并发的建议配置见 SDK readme.md。

语音断句检测阈值为一句话之后的静音长度，取值范围为 200～2000 ms，默认 800 ms。启用语义断句后，语音断句检测不生效；语义断句需要同时开启中间识别结果。

通过 setPayloadParam 传递定制或高级参数，参数为 JSON 格式字符串，例如 {"vad_model":"farfield"}。

request->setUrl("ws://nls-gateway-cn-shanghai-internal.aliyuncs.com/ws/v1");

关键接口

SDK包文件说明

• scripts/build_linux.sh：SDK源码中，以Linux平台为例的示例编译脚本。

• CMakeLists.txt：SDK源码中，以Linux平台为例的示例代码工程CMakeList文件。

• demo目录：SDK包中，集成示例代码，以Linux平台为例，如下表所示。

  文件名	描述
  speechRecognizerDemo.cpp	一句话识别示例。
  speechSynthesizerDemo.cpp	语音合成示例。
  speechTranscriberDemo.cpp	实时语音识别示例。
  fileTransferDemo.cpp	录音文件识别示例。

• resource目录：SDK源码中，语音服务范例音频，可用于功能测试，如下表所示。

  文件名	描述
  test0.wav test1.wav test2.wav test3.wav	测试音频（16k采样频率、16bit采样位数的音频文件）。

• include：SDK源码中的头文件，如下表所示。

  文件名	描述
  nlsClient.h	SDK实例。
  nlsEvent.h	回调事件说明。
  nlsGlobal.h	SDK全局头文件。
  nlsToken.h	SDK Access Token实例。
  iNlsRequest.h	NLS请求基础头文件。
  speechRecognizerRequest.h	一句话识别。
  speechSynthesizerRequest.h	语音合成、长文本语音合成。
  speechTranscriberRequest.h	实时音频流识别。
  FileTrans.h	录音文件识别。

• lib：SDK库文件。

• readme.md：SDK说明。

• release.log：版本说明。

• version：版本号。

C++ SDK 错误码

服务端响应状态码

关于服务状态码，请参见接口说明。

常见问题