This topic describes how to use the C++ software development kit (SDK) for Alibaba Cloud Intelligent Speech Interaction. It provides installation instructions and code examples.
Prerequisites
Read the API reference. For more information, see API reference.
Obtain the AppKey for your project. For more information, see Create a project.
Obtain an access token. For more information, see Overview of obtaining a token.
Download and install
Download the SDK
You can obtain the SDK in one of the following two ways.
Method 1: Obtain the latest source code from GitHub. For detailed compilation and running instructions, see the following sections or the
readme.mdfile in the source code.git clone --depth 1 https://github.com/aliyun/alibabacloud-nls-cpp-sdkMethod 2: Download the required SDK package directly from the table below. The SDK source code package contains the original SDK code, which you must compile to generate the library files required for integration. The other SDK packages are platform-specific, contain the relevant library and header files, and do not require compilation.
Latest SDK package
Platform
MD5
SDK source code
7257c0998654e611cf2e8ca9867670ef
Linux x86_64
9a93df607f26f1558bc1043a425af6d1
NoteThe Linux-x86_64 version is compiled with gcc 8.4.0 and _GLIBCXX_USE_CXX11_ABI=0. You can recompile it using the source code package and following the instructions in the readme.md file.
Where:
alibabacloud-nls-cpp-sdk<version>-master_<github commit id>.zipis the SDK source code package.NlsCppSdk_<Platform>_<VersionNumber>_<github commit id>.tar.gzis the SDK package required for development on the corresponding platform. For more information, see the readme.md file inside the package.
SDK package file description
scripts/build_linux.sh: A sample compilation script for the Linux platform in the SDK source code.CMakeLists.txt: TheCMakeListfile for a sample code-based project on Linux or Android platforms in the SDK source code.demo folder: Contains sample integration code. The following table shows examples for the Linux platform.
File name
Description
speechRecognizerDemo.cpp
Short sentence recognition demo.
speechSynthesizerDemo.cpp
Speech synthesis demo.
speechTranscriberDemo.cpp
Real-time speech recognition demo.
flowingSynthesizerDemo.cpp
Streaming text-to-speech synthesis demo.
fileTransferDemo.cpp
Audio file recognition demo.
resource folder: This folder in the SDK source code contains sample audio for the Voice Service, which you can use for functional testing. For more information, see the table below.
File name
Description
test0.wav
test1.wav
test2.wav
test3.wav
Test audio files (16 kHz audio sampling rate, 16-bit audio bit depth).
include: This folder in the SDK source code contains the SDK header files, as shown in the table below.File name
Description
nlsClient.h
SDK instance.
nlsEvent.h
Callback event description.
nlsGlobal.h
Global SDK header file.
nlsToken.h
SDK Access Token instance.
iNlsRequest.h
Base header file for NLS requests.
speechRecognizerRequest.h
Short sentence recognition.
speechSynthesizerRequest.h
Speech synthesis and long text-to-speech synthesis.
speechTranscriberRequest.h
Real-time audio stream recognition.
flowingSynthesizerRequest.h
Streaming text-to-speech synthesis.
FileTrans.h
Audio file recognition.
lib: SDK library files.readme.md: SDK description.release.log: Describes version updates.version: Version number.
Compile and run (Linux platform)
The minimum required tool versions are as follows:
CMake 3.0
Glibc 2.5
Gcc 4.8.5
Run the following commands in a Linux terminal.
Go to the root directory of the SDK source code.
Generate the SDK library files and executable programs: srDemo for short sentence recognition, stDemo for real-time speech recognition, syDemo for speech synthesis, daDemo for speech dialogue, and fsDemo for streaming text-to-speech synthesis.
./scripts/build_linux.shView the usage of the demo.
cd build/demo ./fsDemo
Key interfaces
Base interfaces
NlsClient: The client for voice processing. You can use this client to perform speech processing tasks, such as short sentence recognition, real-time speech recognition, and speech synthesis. This client is thread-safe. Create only one global instance.
Interface name
Enabled version
Feature description
getInstance
2.x
Gets (creates) an
NlsClientinstance.setLogConfig
2.x
Sets the log file and storage path.
setDirectHost
3.x
Skips
DNSdomain name resolution and directly sets the serverIPv4address. If called, it must be called beforestartWorkThread.setAddrInFamily
3.1.12
Sets the type of the socket address structure. The default is
AF_INET, which only returnsIPv4-related address information. This must be called beforestartWorkThread.setUseSysGetAddrInfo
3.1.13
If the
libeventDNSdoes not meet requirements and cannot completeDNSresolution, call this interface to switch to the system's interface. This must be called beforestartWorkThread.calculateUtf8Chars
3.1.14
Counts the number of characters in the text content. You must pass in text content encoded in
UTF-8. One Chinese character, one English letter, or one punctuation mark is counted as one character.setSyncCallTimeout
3.1.17
Sets the timeout period (in ms) for the synchronous call mode. The default value is 0, which means the synchronous mode is disabled.
When you use the synchronous call mode:
start() is blocked until a result is received from the server.
stop() is blocked until the close() callback is triggered.
setPreconnectedPool
3.3.0
Sets a pre-connection pool for each domain name URL.
Function:
Creates a persistent connection pool for a domain name URL. Connections are automatically reused after requests are complete.
Reduces the connection time before each request is initiated.
Significantly reduces the first-packet latency.
This conflicts with the long-lived connection mode and will disable any configured long-lived connection mode.
Disabled scenario: Tingwu scenarios.
Call constraint: You must call this interface before you call
startWorkThread.
startWorkThread
3.x
Starts the number of worker threads. The default is 1, which starts one thread. If set to -1, it starts a number of threads equal to the number of CPU cores. In high-concurrency situations, -1 is recommended. This can be understood as the initialization of the NlsClient instance and must be called.
getVersion
3.x
Gets the SDK version number.
releaseInstance
2.x
Destroys the
NlsClientobject instance.createFlowingSynthesizerRequest
3.2
Creates a streaming text-to-speech synthesis object. It is thread-safe and supports high-concurrency requests.
releaseFlowingSynthesizerRequest
3.2
Destroys the streaming text-to-speech synthesis object. This needs to be called after the closed event of the current request.
NlsToken: Creates a Token object that is used to request and obtain a Token ID. Before you request a new token, you must obtain a valid timestamp. If a token expires, you must request a new one. Requesting a token multiple times within its validity period causes a Token ID error and renders the token unusable.
Interface name
Feature description
setAccessKeyId
Sets the AccessKey ID of your Alibaba Cloud account.
setKeySecret
Sets the AccessKey secret of your Alibaba Cloud account.
setDomain
Sets the domain name. This is optional.
setServerVersion
Sets the API version. This is optional.
setServerResourcePath
Sets the service path. This is optional.
setRegionId
The service ID is an optional setting.
setAction
Sets the feature. This is optional.
applyNlsToken
Applies for and obtains a Token ID.
getToken
Gets the Token ID.
getExpireTime
Gets the token's expiration timestamp (in seconds).
getErrorMsg
Gets the error message.
NlsEvent: The event object. You can use this object to retrieve the request status code, the result returned from the cloud, failure information, and other data.
Interface name
Feature description
getStatusCode
Gets the status code. A normal status is 0 or 20000000. A failure corresponds to a specific error code.
getErrorMessage
In the
TaskFailedcallback, gets the error message for a failure that occurred during theNlsRequestoperation.getTaskId
Gets the
TaskIdof the task.getBinaryData
Gets the binary data returned from the cloud.
getAllResponse
Gets the full result returned from the cloud.
Streaming Text-to-Speech API
The interface descriptions are based on the contents of the flowingSynthesizerRequest.h file.
Interface name | Enabled version | Feature description |
setOnSynthesisStarted | 3.2 | Sets the callback function for when streaming text-to-speech synthesis starts. |
setOnSynthesisCompleted | 3.2 | Sets the callback function for when speech synthesis is completed. |
setOnChannelClosed | 2.x | Sets the callback function for when the channel is closed. |
setOnTaskFailed | 2.x | Sets the callback function for errors. |
setOnSentenceBegin | 3.2 | The callback function for when the server detects the beginning of a sentence. |
setOnSentenceEnd | 3.2 | The callback function for when the server detects the end of a sentence and returns the full timestamp for that sentence. |
setOnBinaryDataReceived | 2.x | Sets the callback function for receiving binary audio data from speech synthesis. |
setOnSentenceSynthesis | 3.2 | The callback function that incrementally returns the speech synthesis result, including the latest audio and timestamps, full intra-sentence data, and incremental inter-sentence data. |
setOnMessage | 3.1.16 | Sets the callback function for the server-side |
setAppKey | 2.x | Sets the AppKey. |
setToken | 2.x | Security token authentication. All requests must be authenticated using the |
setUrl | 2.x | Sets the service URL address. This is optional. |
sendText | 3.2 | In a single streaming TTS session, a single synthesis cannot exceed 5,000 characters, and the total cannot exceed 100,000 characters. One Chinese character, one English letter, one punctuation mark, or one space between sentences is counted as one character. |
setVoice | 2.x | Sets the voice. |
setVolume | 2.x | Sets the volume. |
setFormat | 2.x | Sets the audio data encoding format. The default is PCM. Supported formats are PCM, WAV, and MP3. |
setSampleRate | 2.x | Sets the audio sampling rate. |
setSpeechRate | 2.x | Sets the speech rate. |
setPitchRate | 2.x | Sets the pitch rate. |
setEnableSubtitle | 2.x | Enables the caption feature. |
setPayloadParam | 2.x | Sets parameters. The input parameter is a JSON formatted string. |
setTimeout | 2.x | Sets the connection timeout period. The default is 5000 ms. |
setContextParam | 2.x | Sets custom user parameters. The input parameter is a JSON formatted string. |
AppendHttpHeaderParam | 2.x | Sets custom |
setSendTimeout | 3.1.14 | Sets the sending timeout period. The default is 5000 ms. |
setEnableOnMessage | 3.1.16 | Enables the callback for server return messages. |
getTaskId | 3.1.17 | Gets the |
start | 2.x | Starts the |
stop | 3.2 | Ends the synthesis task. You must wait for the synthesis to complete. |
cancel | 2.x | Directly closes the speech synthesis process without confirming the closure with the server. |
Code examples
The example saves the synthesized audio to a file. For audio playback with high real-time requirements, use streaming playback. Streaming playback plays the audio data as it is received. This reduces latency because you do not need to wait for the synthesis to complete before you process the audio stream.
For a complete example, see the flowingSynthesizerRequest.cpp file in the demo directory of the SDK package.
Before you call the interface, you must configure the environment variables to read the access credentials. The environment variables for the
AccessKey ID,AccessKey secret, andAppKeyof Intelligent Speech Interaction areNLS_AK_ENV,NLS_SK_ENV, andNLS_APPKEY_ENV, respectively.
Status codes
C++ SDK status codes
Status code | Status message | Reason | Solution |
0 | Success | Success | None |
-10 | DefaultError | Default error. | Not in use. |
-11 | JsonParseFailed | Incorrect JSON format. | Check whether the input JSON string is in the correct JSON format. |
-12 | JsonObjectError | Incorrect JSON object. | Try again. |
-13 | MallocFailed | Malloc failed. | Check if there is sufficient memory. |
-14 | ReallocFailed | Realloc failed. | Check if there is sufficient memory. |
-15 | InvalidInputParam | An invalid parameter was passed in. | Not in use. |
-50 | InvalidLogLevel | Invalid log level. | Check the configured log level. |
-51 | InvalidLogFileSize | Invalid log file size. | Check the configured log file size parameter. |
-52 | InvalidLogFileNum | The number of invalid log files. | Check the configured number of log files parameter. |
-100 | EncoderExistent | The NLS encoder already exists. | Try again. |
-101 | EncoderInexistent | The NLS encoder does not exist. | We recommend that you re-initialize. |
-102 | OpusEncoderCreateFailed | Failed to create the Opus encoder. | We recommend reinitializing. |
-103 | OggOpusEncoderCreateFailed | Failed to create the OggOpus encoder. | We recommend re-initializing. |
-104 | InvalidEncoderType | Invalid encoder type. | OPUS may have been disabled at compile-time but is being used, or check the |
-150 | EventClientEmpty | The main worker thread has a null pointer and has been released. | Re-initialize by calling |
-151 | SelectThreadFailed | Failed to select a worker thread. It has not been initialized. | Re-initialize by calling |
-160 | StartCommandFailed | Failed to send the start command. | Try again. |
-161 | InvokeStartFailed | The request state machine is incorrect, which causes the start operation to fail. | Check whether the current request has not been created or has been completed. |
-162 | InvokeSendAudioFailed | The request state machine is incorrect, which causes the sendAudio operation to fail. | Check whether the current request has started (i.e., the started event callback has been received) or has been completed. |
-163 | InvalidOpusFrameSize | Invalid Opus frame size. The default is 640 bytes. | In OPU encoding mode, |
-164 | InvokeStopFailed | The request state machine is incorrect, which causes the stop operation to fail. | Check whether the current request has not yet started (meaning the `started` event callback has not been received) or has been completed. |
-165 | InvokeCancelFailed | The request state machine is incorrect, which causes the stop operation to fail. | Verify that the current request is either not yet started (the 'started' event callback has not been received) or is already complete. |
-166 | InvokeStControlFailed | The request state machine is incorrect, which causes the stControl operation to fail. | Check whether the 'started' event callback for the current request has not been received, or the request is already complete. |
-200 | NlsEventEmpty | The NLS event is empty. | For internal SDK use. The NlsEvent frame was lost. |
-201 | NewNlsEventFailed | Failed to create NlsEvent. | For internal SDK use. Failed to create the NlsEvent frame. |
-202 | NlsEventMsgEmpty | The message in the NLS event is empty. | The message string was found to be empty during parsing by |
-203 | InvalidNlsEventMsgType | Invalid message type in the NLS event. | For internal SDK use. The event type of the NlsEvent frame is invalid. |
-204 | InvalidNlsEventMsgStatusCode | Invalid message status code in the NLS event. | For internal SDK use. The event message status of the NlsEvent frame is invalid. |
-205 | InvalidNlsEventMsgHeader | Invalid message header in the NLS event. | For internal SDK use. The event message header of the NlsEvent frame is invalid. |
-250 | CancelledExitStatus | Cancel has been called. | Not in use. |
-251 | InvalidWorkStatus | Invalid working status. | For internal SDK use. The internal status of the current request is invalid. |
-252 | InvalidNodeQueue | The NodeQueue in WorkThread is invalid. | For internal SDK use. The current request to be run is invalid. Release the current request and try again. |
-300 | InvalidRequestParams | The request parameters are invalid. | The data passed to SendAudio is empty. |
-301 | RequestEmpty | The request is a null pointer. | For internal SDK use. The current request has been released. Release the current request and try again. |
-302 | InvalidRequest | Invalid request. | For internal SDK use. The current request has been released. Release the current request and try again. |
-303 | SetParamsEmpty | The parameters passed in for setting are empty. | Check if the passed-in parameters are empty. |
-350 | GetHttpHeaderFailed | Failed to get the HTTP header. | For internal SDK use. Locate the issue based on the feedback in the logs. |
-351 | HttpGotBadStatus | HTTP error status. | For internal SDK use. Locate the issue based on the feedback in the logs. |
-352 | WsResponsePackageFailed | Failed to parse the WebSocket response package. | For internal SDK use. Locate the issue based on the feedback in the logs. |
-353 | WsResponsePackageEmpty | The parsed WebSocket response package is empty. | For internal SDK use. Locate the issue based on the feedback in the logs. |
-354 | WsRequestPackageEmpty | The WebSocket request package is empty. | For internal SDK use. Locate the issue based on the feedback in the logs. |
-355 | UnknownWsFrameHeadType | Unknown WebSocket frame header type. | For internal SDK use. Locate the issue based on the feedback in the logs. |
-356 | InvalidWsFrameHeaderSize | Invalid WebSocket frame header size. | For internal SDK use. Locate the issue based on the feedback in the logs. |
-357 | InvalidWsFrameHeaderBody | Invalid WebSocket frame header body. | For internal SDK use. Locate the issue based on the feedback in the logs. |
-358 | InvalidWsFrameBody | Invalid WebSocket frame body. | For internal SDK use. Locate the issue based on the feedback in the logs. |
-359 | WsFrameBodyEmpty | The frame data is empty. This is often due to receiving dirty data. | For internal SDK use. Locate the issue based on the feedback in the logs. |
-400 | NodeEmpty | The node is a null pointer. | Release the current request and try again. |
-401 | InvaildNodeStatus | The node is in an invalid state. | For internal SDK use. Release the current request and try again. |
-402 | GetAddrinfoFailed | Detection is based on parsing the DNS address. | For internal SDK use. Check whether DNS is available in the current environment. |
-403 | ConnectFailed | Network connection failed. | Check whether the current network is available. |
-404 | InvalidDnsSource | The current device has no DNS. | For internal SDK use. Check whether DNS is available in the current environment. |
-405 | ParseUrlFailed | Invalid URL. | Check whether the configured URL is valid. |
-406 | SslHandshakeFailed | SSL handshake failed. | For internal SDK use. Check whether the current network is available and try again. |
-407 | SslCtxEmpty | SSL_CTX is empty. | For internal SDK use. Check whether the current network is available and try again. |
-408 | SslNewFailed | SSL_new failed. | For internal SDK use. Check whether the current network is available and try again. |
-409 | SslSetFailed | Failed to set SSL parameters. | For internal SDK use. Check whether the current network is available and try again. |
-410 | SslConnectFailed | SSL_connect failed. | For internal SDK use. Check whether the current network is available and try again. |
-411 | SslWriteFailed | Failed to send data over SSL. | For internal SDK use. Check whether the current network is available and try again. |
-412 | SslReadSysError | A SYSCALL error was received when receiving data over SSL. | For internal SDK use. Check whether the current network is available and try again. |
-413 | SslReadFailed | Failed to receive data over SSL. | For internal SDK use. Check whether the current network is available and try again. |
-414 | SocketFailed | Failed to create a socket. | For internal SDK use. Check whether the current network is available and try again. |
-415 | SetSocketoptFailed | Failed to set socket parameters. | For internal SDK use. Check whether the current network is available and try again. |
-416 | SocketConnectFailed | Failed to establish a socket connection. | For internal SDK use. Check whether the current network is available and try again. |
-417 | SocketWriteFailed | Failed to send data over the socket. | For internal SDK use. Check whether the current network is available and try again. |
-418 | SocketReadFailed | Failed to receive data from the socket. | For internal SDK use. Check whether the current network is available and try again. |
-430 | NlsReceiveFailed | Failed to receive NLS frame data. | For internal SDK use. Check whether the current network is available and try again. |
-431 | NlsReceiveEmpty | The received NLS frame data is empty. | For internal SDK use. Check whether the current network is available and try again. |
-432 | ReadFailed | Failed to receive data. | For internal SDK use. Check whether the current network is available and try again. |
-433 | NlsSendFailed | Failed to send NLS data. | For internal SDK use. Check whether the current network is available and try again. |
-434 | NewOutputBufferFailed | Failed to create a buffer. | For internal SDK use. Check if there is sufficient memory. |
-435 | NlsEncodingFailed | Audio encoding failed. | For internal SDK use. Release the current request and try again. |
-436 | EventEmpty | The event is empty. | For internal SDK use. Release the current request and try again. |
-437 | EvbufferTooMuch | Too much data in the evbuffer. | For internal SDK use. The send data buffer is full (max cache for 16K audio is 320000, for 8K audio is 160000). Check if audio data is being sent too frequently or if too much data is being sent at once. |
-438 | EvutilSocketFalied | Failed to set evutil parameters. | For internal SDK use. Release the current request and try again. |
-439 | InvalidExitStatus | Invalid exit status. | Check if the current request has been canceled. |
-450 | InvalidAkId | Invalid AccessKey ID for the Alibaba Cloud account. | Check if the AccessKey ID for the Alibaba Cloud account is empty. |
-451 | InvalidAkSecret | Invalid AccessKey secret for the Alibaba Cloud account. | Check if the AccessKey secret for the Alibaba Cloud account is empty. |
-452 | InvalidAppKey | Invalid project AppKey. | Check if the Alibaba Cloud project AppKey is empty. |
-453 | InvalidDomain | Invalid domain. | Check if the input domain is empty. |
-454 | InvalidAction | Invalid action. | Check if the input action is empty. |
-455 | InvalidServerVersion | Invalid ServerVersion. | Check if the input ServerVersion is empty. |
-456 | InvalidServerResource | Invalid ServerResource. | Check if the input ServerResource is empty. |
-457 | InvalidRegionId | Invalid RegionId. | Check if the input Region ID is empty. |
-500 | InvalidFileLink | Invalid audio file link. | The audio file transcription link is empty. |
-501 | ErrorStatusCode | Error status code. | Audio file transcription returned an error. For more information, see the error code. |
-502 | IconvOpenFailed | The request to transform the description failed. | UTF-8 to GBK conversion failed. |
-503 | IconvFailed | Encoding conversion failed. | UTF-8 to GBK conversion failed. |
-504 | ClientRequestFaild | The account client request failed. | The audio file transcription request failed. |
-999 | NlsMaxErrorCode | None | None |
Other status codes
Status code | Status message | Reason | Solution |
10000001 |
|
| We recommend re-initializing. |
10000002 |
|
| Try again. |
| |||
10000003 |
| System error. | Handle the error based on the feedback from the system. |
10000004 |
|
| The passed-in URL is empty. Enter a correct URL. |
10000005 |
|
| The passed-in URL format is incorrect. Enter a correct URL. |
10000007 |
|
| Abnormal JSON format. Check the logs for the specific error point. |
10000008 |
|
| Network connection failed. Check if the local DNS resolution and URL are valid. |
10000009 |
|
| Failed to connect to the cloud. Check the network and try again. |
10000010 |
| Out of memory. | Check if there is sufficient memory. |
10000015 |
|
| Network connection failed. Check if the local DNS resolution and URL are valid. |
10000100 |
|
| The connection was rejected. Check the account, especially whether the token has expired. |
10000101 |
|
| libevent timed out when sending an event. Check whether there are time-consuming tasks in the callback, or whether high concurrency prevents events from being processed promptly. |
10000102 |
|
| libevent timed out when receiving an event. Check whether there are time-consuming tasks in the callback, or whether high concurrency prevents events from being processed promptly. |
10000103 |
|
| Unknown libevent event. Try again. |
10000104 |
|
| The connection is in progress. Try again. |
10000105 |
|
| The pipe cannot handle the request. Try again. |
10000110 |
|
| Update the token. |
10000111 |
|
| Check the validity of the token. |
10000112 |
|
| You do not have permission to use this voice. |
10000113 |
|
| Check if the account has permission, or if the concurrency is within the limit. |
10000120 |
|
| UTF-8 transcoding failed. This is often a system issue. Try again. |
20000000 |
| Success |
Server-side response status codes
For more information about service status codes, see Service status codes.