C++ SDK

Prerequisites

Before you begin, ensure that you have:

• A Linux operating system

• CMake 3.1 or later

• Glibc 2.5 or later

• GCC 4.1.2 or later

• An Alibaba Cloud account and an appkey — to get an appkey, see Manage projects

Download and install the SDK

1. Download the SDK for C++. The package contains the following files and folders: The demo/ folder contains the following files: The include/ folder contains the following header files:

   File/folder	Description
   CMakeLists.txt	CMake file for the demo project
   readme.txt	SDK description
   release.log	Release notes
   version	Version number
   build.sh	Demo compilation script
   lib/	SDK libraries
   build/	Compilation directory
   demo/	Demo source files and test audio
   include/	SDK header files

   File	Description
   speechRecognizerDemo.cpp	Short sentence recognition
   speechSynthesizerDemo.cpp	Speech synthesis
   speechTranscriberDemo.cpp	Real-time speech recognition
   speechLongSynthesizerDemo.cpp	Long-text-to-speech synthesis
   test0.wav, test1.wav	16-bit test audio files at 16,000 Hz

   File	Description
   nlsClient.h	NlsClient object
   nlsEvent.h	Callback events
   speechRecognizerRequest.h	Short sentence recognition
   speechSynthesizerRequest.h	Speech synthesis and long-text-to-speech synthesis
   speechTranscriberRequest.h	Real-time speech recognition

2. Build the demo project. Run the following commands in your Linux terminal:

   Executable	Service
   srDemo	Short sentence recognition
   stDemo	Real-time speech recognition
   syDemo	Speech synthesis
   syLongDemo	Long-text-to-speech synthesis

   mkdir build
   cd build && cmake .. && make

   This generates four executables in the demo/ folder:

3. Run a demo to verify the build.

   ./stDemo <appkey> <AccessKey ID> <AccessKey secret>

Key objects

For a full API reference, see Overview.

Recognition modes

The SDK supports four recognition services. Choose the one that matches your use case before selecting a demo:

The sample code in this topic covers real-time speech recognition. For the other modes, see the corresponding demo files in the demo/ folder.

Sample code

The following sample code demonstrates real-time speech recognition using SpeechTranscriberRequest. The complete source is in speechTranscriberDemo.cpp.

The demo uses audio files at 16,000 Hz. To get accurate results, set the model to universal model for the project bound to your appkey in the Intelligent Speech Interaction console. Select a model that matches your audio sampling rate. For details, see Manage projects.

All examples use SpeechTranscriberRequest to submit asynchronous real-time recognition sessions. Each session follows this pattern: create request → register callbacks → configure parameters → start → send audio → stop → release.

#include <pthread.h>
#include <unistd.h>
#include <ctime>
#include <stdlib.h>
#include <string.h>
#include <string>
#include <vector>
#include <fstream>
#include "nlsClient.h"
#include "nlsEvent.h"
#include "speechTranscriberRequest.h"
#include "nlsCommonSdk/Token.h"

#define FRAME_SIZE 3200
#define SAMPLE_RATE 16000
using namespace AlibabaNlsCommon;
using AlibabaNls::NlsClient;
using AlibabaNls::NlsEvent;
using AlibabaNls::LogDebug;
using AlibabaNls::LogInfo;
using AlibabaNls::SpeechTranscriberRequest;

// Thread parameters: file to recognize, authentication token, and appkey.
struct ParamStruct {
    std::string fileName;
    std::string token;
    std::string appkey;
};

// Custom callback parameters (set these to whatever your application needs).
struct ParamCallBack {
    int userId;
    char userInfo[10];
};

// Store the token and its expiry globally so all threads can share them.
// Check expiry before each request and renew if needed — do not generate
// a new token on every call. One valid token works for all ISI services.
std::string g_akId = "";
std::string g_akSecret = "";
std::string g_token = "";
long g_expireTime = -1;

// Generate a new token using your AccessKey ID and AccessKey secret.
// Returns 0 on success, -1 on failure.
int generateToken(std::string akId, std::string akSecret, std::string* token, long* expireTime) {
    NlsToken nlsTokenRequest;
    nlsTokenRequest.setAccessKeyId(akId);
    nlsTokenRequest.setKeySecret(akSecret);

    if (-1 == nlsTokenRequest.applyNlsToken()) {
        printf("generateToken failed: %s\n", nlsTokenRequest.getErrorMsg());
        return -1;
    }

    *token = nlsTokenRequest.getToken();
    *expireTime = nlsTokenRequest.getExpireTime();
    return 0;
}

// Calculate how long to sleep after each sendAudio call to match real-time audio pace.
// dataSize: bytes sent; sampleRate: 8000 or 16000; compressRate: 1 for PCM, 10 for Opus 10:1.
// For 16-bit PCM at 8,000 Hz: sleep 100 ms per 1,600 bytes.
// For 16-bit PCM at 16,000 Hz: sleep 100 ms per 3,200 bytes.
// For Opus at 16,000 Hz (10:1 compression): sleep = 3200/10 = 320 ms per chunk.
unsigned int getSendAudioSleepTime(int dataSize, int sampleRate, int compressRate) {
    const int sampleBytes = 16;   // Only 16-bit audio is supported.
    const int soundChannel = 1;   // Only mono audio is supported.

    int bytes = (sampleRate * sampleBytes * soundChannel) / 8;
    int bytesMs = bytes / 1000;
    int sleepMs = (dataSize * compressRate) / bytesMs;
    return sleepMs;
}

// --- Callbacks ---
// Each callback receives an NlsEvent with the status code, task ID, and event data.
// Record the task ID — if an error occurs, include it when submitting a support ticket.

// Fired when the recognition session starts successfully.
void onTranscriptionStarted(NlsEvent* cbEvent, void* cbParam) {
    ParamCallBack* tmpParam = (ParamCallBack*)cbParam;
    printf("onTranscriptionStarted: userId=%d, status=%d, taskId=%s\n",
           tmpParam->userId, cbEvent->getStatusCode(), cbEvent->getTaskId());
}

// Fired when the server detects the beginning of a sentence.
void onSentenceBegin(NlsEvent* cbEvent, void* cbParam) {
    ParamCallBack* tmpParam = (ParamCallBack*)cbParam;
    printf("onSentenceBegin: userId=%d, status=%d, taskId=%s, index=%d, time=%d ms\n",
           tmpParam->userId, cbEvent->getStatusCode(), cbEvent->getTaskId(),
           cbEvent->getSentenceIndex(),   // Sentence number, starting from 1.
           cbEvent->getSentenceTime());   // Audio processed so far, in milliseconds.
}

// Fired when the server detects the end of a sentence.
void onSentenceEnd(NlsEvent* cbEvent, void* cbParam) {
    ParamCallBack* tmpParam = (ParamCallBack*)cbParam;
    printf("onSentenceEnd: userId=%d, status=%d, taskId=%s, index=%d, time=%d ms, "
           "beginTime=%d ms, result=%s\n",
           tmpParam->userId, cbEvent->getStatusCode(), cbEvent->getTaskId(),
           cbEvent->getSentenceIndex(),
           cbEvent->getSentenceTime(),
           cbEvent->getSentenceBeginTime(), // Time when SentenceBegin occurred.
           cbEvent->getResult());           // Final transcription for this sentence.
    // Additional fields available (uncomment as needed):
    // cbEvent->getSentenceConfidence()         — confidence score, 0.0–1.0
    // cbEvent->getStashResultBeginTime()       — start time of the next sentence
    // cbEvent->getStashResultCurrentTime()     — current processing time of next sentence
    // cbEvent->getStashResultSentenceId()      — ID of the next sentence
    // cbEvent->getStashResultText()            — leading words of the next sentence
}

// Fired when an intermediate (partial) recognition result is available.
void onTranscriptionResultChanged(NlsEvent* cbEvent, void* cbParam) {
    ParamCallBack* tmpParam = (ParamCallBack*)cbParam;
    printf("onTranscriptionResultChanged: userId=%d, status=%d, taskId=%s, "
           "index=%d, time=%d ms, result=%s\n",
           tmpParam->userId, cbEvent->getStatusCode(), cbEvent->getTaskId(),
           cbEvent->getSentenceIndex(),
           cbEvent->getSentenceTime(),
           cbEvent->getResult());
}

// Fired when the server stops recognition after stop() is called.
// After this event, sendAudio() returns -1 — stop sending audio.
void onTranscriptionCompleted(NlsEvent* cbEvent, void* cbParam) {
    ParamCallBack* tmpParam = (ParamCallBack*)cbParam;
    printf("onTranscriptionCompleted: userId=%d, status=%d, taskId=%s\n",
           tmpParam->userId, cbEvent->getStatusCode(), cbEvent->getTaskId());
}

// Fired when an error occurs during start(), sendAudio(), or stop().
// After this event, sendAudio() returns -1 — stop sending audio.
void onTaskFailed(NlsEvent* cbEvent, void* cbParam) {
    ParamCallBack* tmpParam = (ParamCallBack*)cbParam;
    printf("onTaskFailed: userId=%d, status=%d, taskId=%s, error=%s\n",
           tmpParam->userId, cbEvent->getStatusCode(), cbEvent->getTaskId(),
           cbEvent->getErrorMessage());
}

// Fired when the enable_nlp parameter is set — returns the final NLP result.
void onSentenceSemantics(NlsEvent* cbEvent, void* cbParam) {
    ParamCallBack* tmpParam = (ParamCallBack*)cbParam;
    printf("onSentenceSemantics: userId=%d, response=%s\n",
           tmpParam->userId, cbEvent->getAllResponse());
}

// Fired when the connection closes — either after completion or after an error.
void onChannelClosed(NlsEvent* cbEvent, void* cbParam) {
    ParamCallBack* tmpParam = (ParamCallBack*)cbParam;
    delete tmpParam; // Safe to release here — the SDK has finished using the request.
}

// --- Worker thread ---
// Each thread recognizes one audio file. Use multiple threads for concurrent files.
void* pthreadFunc(void* arg) {
    int sleepMs = 0;

    // Initialize custom callback parameters.
    // Store in heap — the SDK clears these when it releases the request object.
    ParamCallBack* cbParam = new ParamCallBack;
    cbParam->userId = 1234;
    strcpy(cbParam->userInfo, "User.");

    // Step 1: Read thread parameters (token, appkey, file name).
    ParamStruct* tst = (ParamStruct*)arg;
    if (tst == NULL) {
        printf("arg is not valid\n");
        return NULL;
    }

    // Open the audio file.
    std::ifstream fs;
    fs.open(tst->fileName.c_str(), std::ios::binary | std::ios::in);
    if (!fs) {
        printf("%s not found\n", tst->fileName.c_str());
        return NULL;
    }

    // Step 2: Create a SpeechTranscriberRequest.
    SpeechTranscriberRequest* request = NlsClient::getInstance()->createTranscriberRequest();
    if (request == NULL) {
        printf("createTranscriberRequest failed\n");
        return NULL;
    }

    // Register callbacks.
    request->setOnTranscriptionStarted(onTranscriptionStarted, cbParam);
    request->setOnTranscriptionResultChanged(onTranscriptionResultChanged, cbParam);
    request->setOnTranscriptionCompleted(onTranscriptionCompleted, cbParam);
    request->setOnSentenceBegin(onSentenceBegin, cbParam);
    request->setOnSentenceEnd(onSentenceEnd, cbParam);
    request->setOnTaskFailed(onTaskFailed, cbParam);
    request->setOnChannelClosed(onChannelClosed, cbParam);
    request->setOnSentenceSemantics(onSentenceSemantics, cbParam);

    // Configure recognition parameters.
    request->setAppKey(tst->appkey.c_str());
    request->setFormat("pcm");                    // Audio encoding format. Default: pcm.
    request->setSampleRate(SAMPLE_RATE);           // Sampling rate. Valid values: 16000, 8000. Default: 16000.
    request->setIntermediateResult(true);          // Return intermediate results. Default: false.
    request->setPunctuationPrediction(true);       // Add punctuation marks. Default: false.
    request->setInverseTextNormalization(true);    // Convert Chinese numerals to Arabic. Default: false.

    // Optional parameters (uncomment as needed):
    // request->setMaxSentenceSilence(800);        // End-of-sentence silence threshold, 200–2000 ms. Default: 800.
    // request->setCustomizationId("TestId_123");  // Custom model ID.
    // request->setVocabularyId("TestId_456");     // Custom hotword vocabulary ID.
    request->setPayloadParam("{\"enable_words\": true}");  // Return word-level recognition results.
    // request->setPayloadParam("{\"enable_semantic_sentence_detection\": false}"); // Disable VAD.
    // request->setPayloadParam("{\"disfluency\": true}");     // Enable disfluency detection.
    // request->setPayloadParam("{\"vad_model\": \"farfield\"}"); // Set VAD mode.
    // request->setPayloadParam("{\"enable_ignore_sentence_timeout\": false}"); // Ignore single-sentence timeout.
    // request->setPayloadParam("{\"enable_vad_unify_post\": true}"); // Enable post-processing for VAD.

    request->setToken(tst->token.c_str());

    // Step 3: Start the recognition session (asynchronous).
    // On success: onTranscriptionStarted fires. On failure: onTaskFailed fires.
    if (request->start() < 0) {
        printf("start() failed — check network connectivity and firewall settings\n");
        NlsClient::getInstance()->releaseTranscriberRequest(request);
        return NULL;
    }

    // Step 4: Send audio data in chunks.
    while (!fs.eof()) {
        uint8_t data[FRAME_SIZE] = {0};
        fs.read((char*)data, sizeof(uint8_t) * FRAME_SIZE);
        size_t nlen = fs.gcount();
        if (nlen <= 0) {
            continue;
        }

        int ret = request->sendAudio(data, nlen);
        if (ret < 0) {
            // sendAudio returns -1 on error (e.g., after onTaskFailed or onTranscriptionCompleted).
            printf("sendAudio failed — stopping audio send\n");
            break;
        }

        // Step 5: Pace the audio to match the real-time playback rate.
        // Skip this sleep if you are sending live microphone input.
        sleepMs = getSendAudioSleepTime(nlen, SAMPLE_RATE, 1);
        usleep(sleepMs * 1000);
    }

    fs.close();

    // Step 6: Notify the server that all audio has been sent.
    // On failure: onTaskFailed fires.
    request->stop();

    // Step 7: Release the request object after recognition completes.
    NlsClient::getInstance()->releaseTranscriberRequest(request);
    return NULL;
}

// Recognize a single audio file.
int speechTranscriberFile(const char* appkey) {
    // Renew the token if it expires within 10 seconds.
    std::time_t curTime = std::time(0);
    if (g_expireTime - curTime < 10) {
        printf("Token is about to expire — generating a new token\n");
        if (-1 == generateToken(g_akId, g_akSecret, &g_token, &g_expireTime)) {
            return -1;
        }
    }

    ParamStruct pa;
    pa.token = g_token;
    pa.appkey = appkey;
    pa.fileName = "test0.wav";

    pthread_t pthreadId;
    pthread_create(&pthreadId, NULL, &pthreadFunc, (void*)&pa);
    pthread_join(pthreadId, NULL);
    return 0;
}

// Recognize multiple audio files concurrently.
// Each file runs in its own thread. Free-trial accounts support up to 2 concurrent calls.
#define AUDIO_FILE_NUMS 2
#define AUDIO_FILE_NAME_LENGTH 32
int speechTranscriberMultFile(const char* appkey) {
    std::time_t curTime = std::time(0);
    if (g_expireTime - curTime < 10) {
        printf("Token is about to expire — generating a new token\n");
        if (-1 == generateToken(g_akId, g_akSecret, &g_token, &g_expireTime)) {
            return -1;
        }
    }

    char audioFileNames[AUDIO_FILE_NUMS][AUDIO_FILE_NAME_LENGTH] = {"test0.wav", "test1.wav"};
    ParamStruct pa[AUDIO_FILE_NUMS];
    for (int i = 0; i < AUDIO_FILE_NUMS; i++) {
        pa[i].token = g_token;
        pa[i].appkey = appkey;
        pa[i].fileName = audioFileNames[i];
    }

    std::vector<pthread_t> pthreadId(AUDIO_FILE_NUMS);
    for (int j = 0; j < AUDIO_FILE_NUMS; j++) {
        pthread_create(&pthreadId[j], NULL, &pthreadFunc, (void*)&(pa[j]));
    }
    for (int j = 0; j < AUDIO_FILE_NUMS; j++) {
        pthread_join(pthreadId[j], NULL);
    }
    return 0;
}

int main(int argc, char* argv[]) {
    if (argc < 4) {
        printf("Usage: ./stDemo <appkey> <AccessKey ID> <AccessKey secret>\n");
        return -1;
    }

    std::string appkey = argv[1];
    g_akId = argv[2];
    g_akSecret = argv[3];

    // Configure SDK logging (optional).
    // Writes logs to log-transcriber.txt at LogDebug level (all log levels included).
    int ret = NlsClient::getInstance()->setLogConfig("log-transcriber", LogDebug);
    if (-1 == ret) {
        printf("setLogConfig failed\n");
        return -1;
    }

    // Start 4 worker threads for handling SDK callbacks.
    NlsClient::getInstance()->startWorkThread(4);

    // Recognize a single audio file.
    speechTranscriberFile(appkey.c_str());

    // To recognize multiple files concurrently, use this instead:
    // speechTranscriberMultFile(appkey.c_str());

    // Release the NlsClient instance before the process exits.
    // Note: releaseInstance() is not thread-safe — call it only after all tasks complete.
    NlsClient::releaseInstance();
    return 0;
}

Error codes

For service-level status codes returned in callbacks, see the API reference.

What's next

• Overview — Understand how the ISI SDK works before integrating it.

• Manage projects — Configure the model for your appkey in the Intelligent Speech Interaction console.

• API reference — Full list of service status codes and API parameters.