Short sentence recognition mobile SDK APIs-Intelligent Speech Interaction(ISI)-阿里云帮助中心

This API recognizes short audio clips that are less than 60 seconds long. It is ideal for scenarios such as chat, voice commands, voice input, and voice search.

Feature overview

The NUI software development kit (SDK) provides a compact toolkit and improved state management. The NUI SDK offers end-to-end voice capabilities to meet a variety of needs. You can also use it as an atomic capability SDK with a unified API.

Usage notes

Strictly follow these requirements when you code. Otherwise, recognition may fail and return an empty result.

Input format: PCM encoding, 16-bit audio bit depth, and single-channel (mono).
Audio sample rate: 8000 Hz or 16000 Hz.
Duration limit: The audio data cannot exceed 60 s.
Configure the returned result: Set whether to return intermediate recognition results, add punctuation during post-processing, and convert Chinese numerals to Arabic numerals in the output.
Configure multi-language recognition: You cannot specify language and dialect models in the code. Instead, in the Intelligent Speech Interaction console, go to the All Projects page. Then, click Configure Project Features for the relevant project and select the corresponding model. For more information, see Manage projects.

Endpoints

Access type	Description	URL
Public access (default region is Shanghai)	All servers can use the public access URL. The public access URL is set by default in the SDK.	Shanghai: `wss://nls-gateway-cn-shanghai.aliyuncs.com/ws/v1` Beijing: `wss://nls-gateway-cn-beijing.aliyuncs.com/ws/v1` Shenzhen: `wss://nls-gateway-cn-shenzhen.aliyuncs.com/ws/v1`
ECS internal access	If you use Alibaba Cloud ECS instances in the China (Shanghai), China (Beijing), or China (Shenzhen) regions, you can use the internal access URL. ECS instances in the classic network cannot access AnyTunnel. This means they cannot access the Voice Service over the internal network. To use AnyTunnel, create a Virtual Private Cloud (VPC) and access the service from within the VPC. Important Using the internal access method does not incur data transfer costs for the ECS instance. For more information about ECS network types, see Network types.	Shanghai: `ws://nls-gateway-cn-shanghai-internal.aliyuncs.com:80/ws/v1` Beijing: `ws://nls-gateway-cn-beijing-internal.aliyuncs.com:80/ws/v1` Shenzhen: `ws://nls-gateway-cn-shenzhen-internal.aliyuncs.com:80/ws/v1`

Interaction flow

The following figure shows the interaction flow for the iOS SDK and Android SDK.

Note

All server-side responses, except for the audio stream, include the `task_id` parameter in the header. This ID uniquely identifies the recognition task.

1. Authentication and initialization

The client uses a token for authentication when it establishes a WebSocket connection with the server. For more information about how to obtain a token, see Overview of obtaining a token.

The initialization parameters are as follows.

Parameter	Type	Required	Description
workspace	String	Yes	The path of the working directory. The SDK reads configuration files from this path.
app_key	String	Yes	The AppKey of the project created in the console.
token	String	Yes	Make sure the token is valid and has not expired. Note You can set the token during initialization or update it through parameter settings.
device_id	String	Yes	The device ID. It uniquely identifies a device, such as a MAC address, SN, or UniquePsuedoID.
debug_path	String	No	The debug folder. If the `save_log` parameter is set to true during SDK initialization, this folder is used to save intermediate audio files.
save_wav	String	No	This parameter takes effect only when the `save_log` parameter is set to true during SDK initialization. It specifies whether to save the debug audio. The data is saved in the debug folder. Make sure that `debug_path` is valid and writable.

2. Start recognition

Before you initiate a short sentence recognition request, configure the required parameters. Use the `setParams` API in the SDK to set the parameters in JSON format. You only need to set these parameters once. The parameters are described as follows.

Parameter	Type	Required	Description
app_key	String	No	The AppKey of the project created in the console. This is usually set during initialization.
token	String	No	If an update is required, configure the settings.
service_type	Int	Yes	The type of voice service to request. For short sentence recognition, set this to `0`.
direct_ip	String	No	Lets the client perform DNS resolution and then pass in the IP address for access.
nls_config	JsonObject	No	Parameter settings related to accessing the Voice Service. For more information, see the following table.

The `nls_config` parameters are as follows.

Parameter	Type	Required	Description
sr_format	String	No	The audio coding format. OPUS encoding and raw PCM audio are supported. Default: OPUS. Note If you use a sample rate of 8000 Hz, only the PCM format is supported.
sample_rate	Integer	No	The audio sampling rate. Default: 16000 Hz. Configure a model that supports this sample rate and scenario in the corresponding project in the console.
enable_intermediate_result	Boolean	No	Specifies whether to return intermediate recognition results. Default: False.
enable_punctuation_prediction	Boolean	No	Specifies whether to add punctuation in post-processing. Default: False.
enable_inverse_text_normalization	Boolean	No	Specifies whether to enable inverse text normalization (ITN) to convert Chinese numerals to Arabic numerals. If set to True, Chinese numerals are converted to Arabic numerals in the output. Default: False.
customization_id	String	No	The self-learning model ID.
vocabulary_id	String	No	The ID of the custom hotword vocabulary.
enable_voice_detection	Boolean	No	Specifies whether to enable voice activity detection (VAD). When enabled, the service can detect the start and end of speech in an audio segment and remove noise. Default: False (disabled).
max_start_silence	Integer	No	This parameter takes effect only when `enable_voice_detection` is set to true. It specifies the maximum duration of silence allowed at the beginning of the audio. Unit: milliseconds. If this duration is exceeded (no speech is detected after recognition starts), the server sends a TaskFailed event and ends the recognition task.
max_end_silence	Integer	No	This parameter takes effect only when `enable_voice_detection` is set to true. It specifies the maximum duration of silence allowed at the end of the audio. Unit: milliseconds. Value range: 200 ms to 6000 ms. If this duration is exceeded, the server sends a RecognitionCompleted event and ends the recognition task. Note that subsequent audio will not be recognized.
extend_config	JsonObject	No	To set parameters that are supported by the interaction protocol but not described in this API reference, use this universal configuration interface. For specific configuration methods, see the code examples.

3. Send data

The client sends audio data in a loop and continuously receives recognition results.

If `enable_intermediate_result` is set to true, the SDK continuously reports the `EVENT_ASR_PARTIAL_RESULT` event through the `onNuiEventCallback` callback. This event contains the intermediate recognition result:

{
    "header": {
        "namespace": "SpeechRecognizer",
        "name": "RecognitionResultChanged",
        "status": 20000000,
        "message_id": "e06d2b5d50ca40d5a50d4215c7c8****",
        "task_id": "4c3502c7a5ce4ac3bdc488749ce4****",
        "status_text": "Gateway:SUCCESS:Success."
    },
    "payload": {
        "result": "The weather in Beijing"
    }
}

Description of header object parameters:

Parameter	Type	Description
namespace	String	The namespace to which the message belongs.
name	String	The message name. `RecognitionResultChanged` indicates that an intermediate recognition result was obtained.
status	Integer	The status code. It indicates whether the request was successful. See service status codes.
message_id	String	The ID of this message, automatically generated by the SDK.
task_id	String	The globally unique ID (GUID) of the task. Record this value for troubleshooting.
status_text	String	The status message.

Description of payload object parameters:

Parameter	Type	Description
result	String	The intermediate recognition result.

Note

The last intermediate recognition result may not be the same as the final recognition result. Use the result from the `EVENT_ASR_RESULT` event as the final result.

If `enable_intermediate_result` is set to false, the server does not return any message in this step.

4. End recognition

The client sends a request to stop short sentence recognition. This request notifies the server that the audio data stream has ended. The server then stops the speech recognition task and returns the final recognition result:

{
    "header": {
        "namespace": "SpeechRecognizer",
        "name": "RecognitionCompleted",
        "status": 20000000,
        "message_id": "10490c992aef44eaa4246614838f****",
        "task_id": "4c3502c7a5ce4ac3bdc488749ce4****",
        "status_text": "Gateway:SUCCESS:Success."
    },
    "payload": {
        "result": "The weather in Beijing."
    }
}

Description of header object parameters:

Parameter	Type	Description
namespace	String	The namespace to which the message belongs.
name	String	The message name. `RecognitionCompleted` indicates that recognition is complete.
status	Integer	The status code. It indicates whether the request was successful. See service status codes.
message_id	String	The ID of this message, automatically generated by the SDK.
task_id	String	The GUID of the task. Record this value for troubleshooting.
status_text	String	The status message.

Description of payload object parameters:

Parameter	Type	Description
result	String	The final result of short sentence recognition.

Error codes

General-purpose error codes

Status code	Status message	Cause	Solution
40000000	The default client error code. This code corresponds to multiple error messages.	Invalid parameters or call logic was used.	Compare your code with the sample code in the official documentation to test and verify it.
40000001	The token 'xxx' has expired. The token 'xxx' is invalid	Invalid parameters or call logic was used. This is a general-purpose client error code that usually indicates an incorrect token, such as an expired or invalid token.	Compare your code with the sample code in the official documentation to test and verify it.
40000002	Gateway:MESSAGE_INVALID:Can't process message in state'FAILED'!	The message is invalid or incorrect.	Compare your code with the sample code in the official documentation to test and verify it.
40000003	PARAMETER_INVALID Failed to decode url params	The parameters passed by the user are incorrect. This error is common for RESTful API calls.	Compare your code with the sample code in the official documentation to test and verify it.
40000005	Gateway:TOO_MANY_REQUESTS:Too many requests!	Too many concurrent requests.	If you are using the Free Edition, you can upgrade to a commercial version to increase the concurrency. If you are already using a commercial version, you can purchase a concurrency resource plan to increase your concurrency quota.
40000009	Invalid wav header!	The message header is invalid.	If you send a WAV audio file and set the `format` parameter to `wav`, check whether the WAV header of the audio file is correct. If the header is incorrect, the server may reject the request.
40000009	Too large wav header!	The WAV header of the transmitted audio is invalid.	You can send the audio stream in a format such as PCM or OPUS. If you use the WAV format, make sure that the WAV header of the audio file contains the correct data length.
40000010	Gateway:FREE_TRIAL_EXPIRED:The free trial has expired!	The trial period has ended, and the commercial version is not activated or your account has an overdue payment.	You can log on to the console to check the service activation status and your account balance.
40010001	Gateway:NAMESPACE_NOT_FOUND:RESTful url path illegal	The operation or parameter is not supported.	Check whether the parameters passed in the call are consistent with the requirements in the official documentation. You can compare them with the error message to identify and set the correct parameters. For example, if you are using a curl command to make a RESTful API request, check whether the URL you constructed is valid.
40010003	Gateway:DIRECTIVE_INVALID:[xxx]	A general-purpose client-side error code.	This error indicates that the client passed an incorrect parameter or instruction. Detailed error messages are available for different operations. You can refer to the corresponding documentation to set the parameters correctly.
40010004	Gateway:CLIENT_DISCONNECT:Client disconnected before task finished!	The client actively terminated the connection before the request was processed.	None. Alternatively, you can close the connection after the server responds.
40010005	Gateway:TASK_STATE_ERROR:Got stop directive while task is stopping!	The client sent a message instruction that is not currently supported.	Compare your code with the sample code in the official documentation to test and verify it.
40020105	Meta:APPKEY_NOT_EXIST:Appkey not exist!	A non-existent Appkey was used.	Confirm whether a non-existent Appkey was used. You can log on to the console and view the project configuration to find the Appkey.
40020106	Meta:APPKEY_UID_MISMATCH:Appkey and user mismatch!	The Appkey and token passed in the call were not created by the same Alibaba Cloud account UID. This causes a mismatch.	Check whether you are using resources from two different accounts. Do not use an Appkey from Account A with a token generated from Account B.
403	Forbidden	The token is invalid. For example, the token does not exist or has expired.	Set a valid token. Tokens have an expiration period. You must obtain a new token before the current one expires.
41000003	MetaInfo doesn't have end point info	Failed to retrieve the routing information for this Appkey.	Check whether you are using resources from two different accounts. Do not use an Appkey from Account A with a token generated from Account B.
41010101	UNSUPPORTED_SAMPLE_RATE	The sample rate is not supported.	Real-time speech recognition currently supports only audio with a sample rate of 8000 Hz or 16000 Hz.
41040201	Realtime:GET_CLIENT_DATA_TIMEOUT:Client data does not send continuously!	Failed to retrieve data from the client due to a timeout.	When you call real-time speech recognition, the client must send data at a real-time rate and close the connection promptly after the data is sent.
50000000	GRPC_ERROR:Grpc error!	An exception caused by factors such as machine load or network issues. This error usually occurs randomly.	You can retry the call to resolve the issue.
50000001	GRPC_ERROR:Grpc error!	An exception caused by factors such as machine load or network issues. This error usually occurs randomly.	You can retry the call to resolve the issue.
52010001	GRPC_ERROR:Grpc error!	An exception caused by factors such as machine load or network issues. This error usually occurs randomly.	You can retry the call to resolve the issue.

Short sentence recognition error codes

Status code	Status message	Cause	Solution
40000000	Gateway:CLIENT_ERROR:Empty audio data!	No audio data.	You can refer to the public cloud sample code and send audio data with the request.
40000004	Gateway:IDLE_TIMEOUT:Websocket session is idle for too long time	After a connection is established, the server returns this error message if no data is sent for more than 10 seconds.	After you establish a connection, you must maintain interaction with the server. For example, you can continuously send the audio stream while it is being captured. Close the connection promptly after the audio is sent.
40010002	Gateway:DIRECTIVE_NOT_SUPPORTED:Directive'SpeechRecognizer.EnhanceRecognition'isnotsupported!	A message instruction that is not supported by the server was sent.	Compare your code with the sample code in the official documentation to test and verify it.
40010003	Gateway:DIRECTIVE_INVALID:Too many items for ‘vocabulary'!(173)	Too many hotwords were set.	You can refer to the API documentation to set the parameters correctly.
40270002	NO_VALID_AUDIO_ERROR	The audio is invalid.	No valid text was recognized from the audio.
41010104	TOO_LONG_SPEECH	The duration of the sent audio exceeds the limit. This error occurs only with the short sentence recognition operation.	Short sentence recognition supports audio up to 60 seconds long. If the audio is longer than 60 seconds, you must call the real-time speech recognition operation.
41010105	SILENT_SPEECH	The data contains only silence or noise, so no valid speech can be detected.	None.

Short sentence recognition/Real-time speech recognition/Flash audio file recognition

Configuration or parameter errors

Status code	Status message	Cause	Solution
240999	DEFAULT_ERROR	Internal default error.	Internal unspecified error.
240001	NUI_CONFIG_INVALID	Configuration file error.	Configuration file error. Confirm that the resource file exists in the specified resource path. For Android, refer to the code sample and use the copyAssets interface.
240002	ILLEGAL_PARAM	Invalid parameter.	Confirm that the passed format is correct, including field types and value range limits. For example, enable_voice_detection must be set to the boolean value true, not 1 or the string "true".
240003	ILLEGAL_INIT_PARAM	Invalid initialization parameter.	Confirm whether the initialization parameter format is incorrect or if a required field is missing.
240004	NECESSARY_PARAM_LACK	A required parameter is missing.	Confirm the required parameters for the operation call.
240005	NULL_PARAM_ERROR	The parameter is null.	Confirm whether the parameter is null.
240006	NULL_LISTENER_ERROR	The event callback is not defined.	Confirm that the callback event is correctly assigned.
240007	NULL_DIALOG_ERROR	No valid dialog instance. This usually occurs during an internal state error.	Confirm that the state is correct before calling the operation. You can use the cancel operation to restore the idle state.
240008	NULL_ENGINE_ERROR	No valid engine instance. Check whether initialization was successful.	Confirm whether initialization was successful.
240009	ILLEGAL_DATA	The passed audio data address or length is invalid.	Confirm the passed data length value.

SDK status errors

Status code	Status message	Cause	Solution
240010	ILLEGAL_REENTRANT	An SDK operation was called after exiting.	This can be ignored if it does not affect functionality.
240011	SDK_NOT_INIT	The SDK is not initialized correctly.	Confirm that the initialization returns a correct value before you call other operations.
240012	SDK_ALREADY_INIT	The SDK initialization operation was called repeatedly.	Confirm the initialization call logic.
240013	DIALOG_INVALID_STATE	Internal dialog state error.	Read the SDK flowchart to confirm whether the operation was called in an incorrect state.
240014	STATE_INVALID	SDK internal state error.	Read the SDK flowchart to confirm whether the operation was called in an incorrect state.
240015	ILLEGAL_FUNC_CALL	The operation cannot be called in this mode.	Confirm whether the operation call is reasonable.

System call errors

Status code	Status message	Cause	Solution
240020	MEM_ALLOC_ERROR	Memory allocation error.	Check for insufficient memory.
240021	FILE_ACCESS_FAIL	File access error.	Check whether read and write permissions are granted for the file.
240022	CREATE_DIR_ERROR	Error creating directory.	Check for write permissions.

SDK internal call errors

Status code	Status message	Cause	Solution
240030	CREATE_NUI_ERROR	Engine creation failed.	Failed to create the instance, usually due to insufficient system resources.
240031	TEXT_DIALOG_START_FAIL	Failed to initiate text understanding.	Text-to-semantic understanding failed. Check the network connection or whether information such as the URL and token is valid.
240032	TEXT_CANCEL_START_FAIL	Failed to cancel text understanding.	This can be ignored.
240033	WUW_DUPLICATE	Duplicate dynamic wake-up word.	This can be ignored.

Local engine call errors
Status code
Status message
Cause
Solution
240040
CEI_INIT_FAIL
Local engine initialization failed.
Confirm whether the local engine's model is valid and whether the directory is readable and writable.

Audio errors

Status code	Status message	Cause	Solution
240051	UPDATE_AUDIO_ERROR	Error pushing audio, usually because the input audio length is greater than the required audio length.	Confirm whether the length of the pushed audio is invalid.
240052	MIC_ERROR	Failed to get audio for 2 consecutive seconds.	Confirm whether audio of the required length is correctly provided in the audio data callback.

Network errors

Status code	Status message	Cause	Solution
240060	CREATE_DA_REQUEST_ERROR	Failed to create a dialog assistant instance.	This can be ignored.
240061	START_DA_REQUEST_ERROR	Failed to initiate a dialog assistant request.	This can be ignored.
240062	DEFAULT_NLS_ERROR	A server-side error occurred. Note This error also contains the content of the server-side response.	For more information, see Server-side error codes to further locate the issue.
240063	SSL_ERROR	Error creating SSL instance.	Ignore if it occurs randomly.
240064	SSL_CONNECT_FAILED	SSL connection failed.	Connection exception. Check whether the service URL or local network connection is normal.
240065	HTTP_CONNECT_FAILED	HTTP connection failed.	Service connection error. Check the log file for the HTTP return value to confirm the cause.
240066	DNS_FAILED	DNS resolution failed.	Check whether the local network and DNS service are normal.
240067	CONNECT_FAILED	Socket connection failed.	Check the network connection.
240068	SERVER_NOT_ACCESS	The server is inaccessible.	Check whether the token has expired or the URL is correct.
240069	SOCKET_CLOSED	The socket is closed.	Ignore if it occurs randomly.
240070	AUTH_FAILED	Authentication failed.	Check whether you have provided the correct ak_secret, ak_id, app_key, sdk_code, and device_id. Also, confirm that you have a sufficient quota.
240071	HTTPDNS_FAILED	Failed to connect using the IP address passed by the client.	If you are accessing by directly passing an IP address, confirm whether the IP address is accessible.
240072	HTTP_SEND_FAILED	HTTP send failed for file transcription.	Confirm that the network connection is normal.
240073	HTTP_RECEIVE_FAILED	HTTP receive failed for file transcription.	Confirm that the network connection is normal.
240074	HTTP_RESPONSE_ERROR	Failed to parse received content for file transcription.	The content returned by the server is incorrect.
240075	HTTP_SERVER_ERROR	File transcription service error.	For more information, see Server-side error codes to further locate the issue.