The short sentence recognition feature recognizes audio clips that are shorter than one minute. It is ideal for short speech recognition scenarios, such as chat conversations, voice commands, voice input, and voice search.
Billing and concurrency limits
Short sentence recognition is available in a trial version and a commercial version. For more information, see Trial and commercial versions. To upgrade from the trial version to the commercial version, see Upgrade from the trial version to the commercial version.
For more information about billing methods, see Billing methods.
For more information about concurrency limits, see Concurrency and QPS.
Usage notes
To use the Android or iOS software development kit (SDK), see Mobile SDK API reference.
You must strictly follow these requirements during encoding. Otherwise, recognition may fail and return an empty result.
Supported input formats: PCM, WAV files with PCM encoding, OPUS in OGG containers, SPEEX in OGG containers, AMR, MP3, and AAC. The audio must be mono with a 16-bit audio bit depth.
Audio sample rates: 8000 Hz and 16000 Hz.
Duration limit: The audio must not exceed 60 s.
Audio file size: The file size cannot exceed 2 MB.
Sentiment analysis: Currently, sentiment analysis is available only for Chinese audio at 8 kHz.
You can configure the following settings for the returned results:
Whether to return intermediate recognition results.
Whether to add punctuation in post-processing.
Whether to convert Chinese numerals to Arabic numerals in the output.
Configure multi-language recognition: Language and dialect models cannot be specified during encoding. You must go to All Projects in the Intelligent Speech Interaction console, find the relevant project, and click Configure Project Features to select the model. For more information, see Manage projects.
The following tables list the supported languages and dialect models.
Intelligent access to the nearest region
Short sentence recognition supports intelligent access to the nearest region through the domain name nls-gateway.aliyuncs.com.
We recommend that you use the nearest region access domain name. When the API is called, the system automatically resolves the request to the server in the nearest region based on the client's geographical location. For example, if a request is initiated from the China (Beijing) region, the system automatically resolves it to a server in the China (Beijing) region. This provides the same result as using the region-specific domain name nls-gateway-cn-beijing.aliyuncs.com.
Endpoint
Access type | Description | URL |
Public network access (default: China (Shanghai) region) | All servers can use the public network access URL. The public network access URL is set by default in the SDK. |
|
ECS private network access | If you use Alibaba Cloud ECS instances in the China (Shanghai), China (Beijing), or China (Shenzhen) regions, you can use the private network access URL. The classic network of an ECS instance cannot access AnyTunnel, which means you cannot access the Voice Service over the private network. If you want to use AnyTunnel, create a VPC and access the service from within it. Important
|
|
Interaction flow
All server responses include the `task_id` parameter in the response header. This parameter identifies the current recognition task.
The interaction flowchart is for the Java, C++, iOS, and Android SDKs. It does not include the interaction flow for the RESTful API. For more information about the RESTful API interaction flowchart, see RESTful API.
1. Authentication
The client must use a token to authenticate when it establishes a WebSocket connection with the server. For more information about how to obtain a token, see Overview of obtaining a token.
2. Start recognition
The client sends a short sentence recognition request, and the server validates the request.
You must set parameters in the request message. In the SDK, you can set these parameters using the `set` methods of the `SpeechRecognizer` object. The following table describes the parameters.
Parameter | Type | Required | Description |
appkey | String | Yes | The AppKey of the project that you created in the console. |
format | String | No | The audio format. Supported formats include PCM, WAV, OPUS, SPEEX, AMR, MP3, and AAC. |
sample_rate | Integer | No | The audio sampling rate. Default value: 16000 Hz. Configure a model that supports this sample rate and scenario in the corresponding project in the console. |
enable_intermediate_result | Boolean | No | Specifies whether to return intermediate recognition results. Default value: False. |
enable_punctuation_prediction | Boolean | No | Specifies whether to add punctuation in post-processing. Default value: False. |
enable_inverse_text_normalization | Boolean | No | Inverse text normalization (ITN) for converting Chinese numerals to Arabic numerals. If you set this parameter to True, Chinese numerals are converted to Arabic numerals. Default value: False. |
disfluency | Boolean | No | Filters filler words. This is also known as disfluency detection. Default value: False (disabled). |
customization_id | String | No | The ID of the custom language model. For more information, see Customize a language model. |
vocabulary_id | String | No | The ID of the custom hotword vocabulary. For more information, see Create a hotword vocabulary in the console. |
enable_voice_detection | Boolean | No | Specifies whether to enable voice activity detection (VAD). When enabled, the service can detect the start and end of valid speech in an audio segment and remove noise. Default value: False (disabled). |
max_start_silence | Integer | No | This parameter takes effect only when `enable_voice_detection` is set to true. It specifies the maximum allowed duration of silence at the beginning. Recommended value range: (0, 60000]. Unit: milliseconds. If this duration is exceeded (meaning no sound is detected for a long time after recognition starts), the server sends a `TaskFailed` event and ends the current recognition task. |
max_end_silence | Integer | No | This parameter takes effect only when `enable_voice_detection` is set to true. It specifies the maximum allowed duration of silence at the end. Unit: milliseconds. Value range: 200 ms to 6000 ms. If this duration is exceeded, the server sends a `RecognitionCompleted` event and ends the current recognition task. Note that subsequent speech will not be recognized. |
audio_address | String | No | A download link for an audio file that can be accessed over the public network. We recommend using Alibaba Cloud Object Storage Service (OSS). For more information, see How to obtain an access URL in OSS. |
special_word_filter | String (JSON format) | No | The sensitive word filtering feature. You can customize up to 32 sensitive words. This parameter lets you: Do not process (default: false, displays the original text), Replace with *. For specific call instructions, see the custom word filtering call example below. |
enable_multi_thresh_mod | Boolean | No | This parameter takes effect only when `enable_voice_detection` is set to True. Valid values:
|
The following example shows how to call the custom word filtering feature:
// Take real-time transcription as an example.
JSONObject root = new JSONObject();
root.put("system_reserved_filter", true);
// Replace the following words with an empty string
JSONObject root1 = new JSONObject();
JSONArray array1 = new JSONArray();
array1.add("start");
array1.add("occur");
root1.put("word_list", array1);
// Replace the following words with *
JSONObject root2 = new JSONObject();
JSONArray array2 = new JSONArray();
array2.add("test");
root2.put("word_list", array2);
// You can set all or some of the parameters
root.put("filter_with_empty", root1);
root.put("filter_with_signed", root2);
transcriber.addCustomedParam("special_word_filter", root);3. Send data
Continuously send audio data and receive recognition results.
If `enable_intermediate_result` is set to true, the server continuously returns `RecognitionResultChanged` messages, which are intermediate recognition results. For example:
Beijing's sky Beijing's weatherThe server returns the following response message:
{ "header": { "namespace": "SpeechRecognizer", "name": "RecognitionResultChanged", "status": 20000000, "message_id": "e06d2b5d50ca40d5a50d4215c7c8****", "task_id": "4c3502c7a5ce4ac3bdc488749ce4****", "status_text": "Gateway:SUCCESS:Success." }, "payload": { "result": "Weather in Beijing" } }The following table describes the parameters in the header object.
Parameter
Type
Description
namespace
String
The namespace of the message.
name
String
The name of the message. `RecognitionResultChanged` indicates that an intermediate recognition result was obtained.
status
Integer
The status code. It indicates whether the request was successful. See the service status codes.
status_text
String
The status message.
task_id
String
The globally unique ID of the task. Record this value to help with troubleshooting.
message_id
String
The ID of this message.
The following table describes the parameters in the payload object.
Parameter
Type
Description
result
String
The intermediate recognition result.
ImportantThe last intermediate result may not be the same as the final recognition result. Always use the final result from the `RecognitionCompleted` message.
If `enable_intermediate_result` is set to false, the server does not return any messages in this step.
4. End recognition
The client sends a request to stop short sentence recognition. This notifies the server that all audio data has been sent and that it can stop the speech recognition task. The server then returns the final recognition result:
{
"header": {
"namespace": "SpeechRecognizer",
"name": "RecognitionCompleted",
"status": 20000000,
"message_id": "10490c992aef44eaa4246614838f****",
"task_id": "4c3502c7a5ce4ac3bdc488749ce4****",
"status_text": "Gateway:SUCCESS:Success."
},
"payload": {
"result": "Weather in Beijing.",
"emo_tag": "neutral",
"emo_confidence": 0.931
}
}The following table describes the parameters in the header object.
Parameter | Type | Description |
namespace | String | The namespace of the message. |
name | String | The name of the message. `RecognitionCompleted` indicates that the recognition is complete. |
status | Integer | The status code. It indicates whether the request was successful. See the service status codes. |
status_text | String | The status message. |
task_id | String | The globally unique ID of the task. Record this value to help with troubleshooting. |
message_id | String | The ID of this message, which is automatically generated by the SDK. |
The following table describes the parameters in the payload object.
Parameter | Type | Description |
result | String | The result of the short sentence recognition. |
emo_tag | String | The emotion of the current sentence. It can be `positive` (positive emotion, such as happy or satisfied), `negative` (negative emotion, such as angry, dull, or disappointed), or `neutral` (no obvious emotion). |
emo_confidence | Double | The confidence level of the emotion recognition for the current sentence. The value ranges from 0.0 to 1.0. A larger value indicates a higher confidence level. |
Service status codes
Each response from the service includes a `status` field, which indicates the service status code. The following tables list the common error codes and short sentence recognition error codes and their descriptions.
General-purpose error codes
Status code | Status message | Cause | Solution |
40000000 | The default client error code. This code corresponds to multiple error messages. | Invalid parameters or call logic was used. | Compare your code with the sample code in the official documentation to test and verify it. |
40000001 | The token 'xxx' has expired. The token 'xxx' is invalid | Invalid parameters or call logic was used. This is a general-purpose client error code that usually indicates an incorrect token, such as an expired or invalid token. | Compare your code with the sample code in the official documentation to test and verify it. |
40000002 | Gateway:MESSAGE_INVALID:Can't process message in state'FAILED'! | The message is invalid or incorrect. | Compare your code with the sample code in the official documentation to test and verify it. |
40000003 | PARAMETER_INVALID Failed to decode url params | The parameters passed by the user are incorrect. This error is common for RESTful API calls. | Compare your code with the sample code in the official documentation to test and verify it. |
40000005 | Gateway:TOO_MANY_REQUESTS:Too many requests! | Too many concurrent requests. | If you are using the Free Edition, you can upgrade to a commercial version to increase the concurrency. If you are already using a commercial version, you can purchase a concurrency resource plan to increase your concurrency quota. |
40000009 | Invalid wav header! | The message header is invalid. | If you send a WAV audio file and set the |
40000009 | Too large wav header! | The WAV header of the transmitted audio is invalid. | You can send the audio stream in a format such as PCM or OPUS. If you use the WAV format, make sure that the WAV header of the audio file contains the correct data length. |
40000010 | Gateway:FREE_TRIAL_EXPIRED:The free trial has expired! | The trial period has ended, and the commercial version is not activated or your account has an overdue payment. | You can log on to the console to check the service activation status and your account balance. |
40010001 | Gateway:NAMESPACE_NOT_FOUND:RESTful url path illegal | The operation or parameter is not supported. | Check whether the parameters passed in the call are consistent with the requirements in the official documentation. You can compare them with the error message to identify and set the correct parameters. For example, if you are using a curl command to make a RESTful API request, check whether the URL you constructed is valid. |
40010003 | Gateway:DIRECTIVE_INVALID:[xxx] | A general-purpose client-side error code. | This error indicates that the client passed an incorrect parameter or instruction. Detailed error messages are available for different operations. You can refer to the corresponding documentation to set the parameters correctly. |
40010004 | Gateway:CLIENT_DISCONNECT:Client disconnected before task finished! | The client actively terminated the connection before the request was processed. | None. Alternatively, you can close the connection after the server responds. |
40010005 | Gateway:TASK_STATE_ERROR:Got stop directive while task is stopping! | The client sent a message instruction that is not currently supported. | Compare your code with the sample code in the official documentation to test and verify it. |
40020105 | Meta:APPKEY_NOT_EXIST:Appkey not exist! | A non-existent Appkey was used. | Confirm whether a non-existent Appkey was used. You can log on to the console and view the project configuration to find the Appkey. |
40020106 | Meta:APPKEY_UID_MISMATCH:Appkey and user mismatch! | The Appkey and token passed in the call were not created by the same Alibaba Cloud account UID. This causes a mismatch. | Check whether you are using resources from two different accounts. Do not use an Appkey from Account A with a token generated from Account B. |
403 | Forbidden | The token is invalid. For example, the token does not exist or has expired. | Set a valid token. Tokens have an expiration period. You must obtain a new token before the current one expires. |
41000003 | MetaInfo doesn't have end point info | Failed to retrieve the routing information for this Appkey. | Check whether you are using resources from two different accounts. Do not use an Appkey from Account A with a token generated from Account B. |
41010101 | UNSUPPORTED_SAMPLE_RATE | The sample rate is not supported. | Real-time speech recognition currently supports only audio with a sample rate of 8000 Hz or 16000 Hz. |
41040201 | Realtime:GET_CLIENT_DATA_TIMEOUT:Client data does not send continuously! | Failed to retrieve data from the client due to a timeout. | When you call real-time speech recognition, the client must send data at a real-time rate and close the connection promptly after the data is sent. |
50000000 | GRPC_ERROR:Grpc error! | An exception caused by factors such as machine load or network issues. This error usually occurs randomly. | You can retry the call to resolve the issue. |
50000001 | GRPC_ERROR:Grpc error! | An exception caused by factors such as machine load or network issues. This error usually occurs randomly. | You can retry the call to resolve the issue. |
52010001 | GRPC_ERROR:Grpc error! | An exception caused by factors such as machine load or network issues. This error usually occurs randomly. | You can retry the call to resolve the issue. |
Short sentence recognition error codes
Status code | Status message | Cause | Solution |
40000000 | Gateway:CLIENT_ERROR:Empty audio data! | No audio data. | You can refer to the public cloud sample code and send audio data with the request. |
40000004 | Gateway:IDLE_TIMEOUT:Websocket session is idle for too long time | After a connection is established, the server returns this error message if no data is sent for more than 10 seconds. | After you establish a connection, you must maintain interaction with the server. For example, you can continuously send the audio stream while it is being captured. Close the connection promptly after the audio is sent. |
40010002 | Gateway:DIRECTIVE_NOT_SUPPORTED:Directive'SpeechRecognizer.EnhanceRecognition'isnotsupported! | A message instruction that is not supported by the server was sent. | Compare your code with the sample code in the official documentation to test and verify it. |
40010003 | Gateway:DIRECTIVE_INVALID:Too many items for ‘vocabulary'!(173) | Too many hotwords were set. | You can refer to the API documentation to set the parameters correctly. |
40270002 | NO_VALID_AUDIO_ERROR | The audio is invalid. | No valid text was recognized from the audio. |
41010104 | TOO_LONG_SPEECH | The duration of the sent audio exceeds the limit. This error occurs only with the short sentence recognition operation. | Short sentence recognition supports audio up to 60 seconds long. If the audio is longer than 60 seconds, you must call the real-time speech recognition operation. |
41010105 | SILENT_SPEECH | The data contains only silence or noise, so no valid speech can be detected. | None. |