API reference
Sound event detection identifies sounds such as background music, crying, laughter, and explosions in audio and returns their start and end times.
Sound event types
The following sound event types are supported:
Music: background music.Singing: singing.Laughter: laughter.Knock: knocking on a door or table.Keyboard: keyboard sounds.Cry: crying.Explosion: explosions.Water: water sounds.SirenAlarm: sirens.
Limits
The input must be an uncompressed PCM file or a PCM-encoded WAV file with a 16-bit audio bit depth and one channel.
The audio sample rate must be 16,000 Hz.
Audio must be sent as a real-time stream.
Billing and concurrency limits
Sound event detection is available in the Free Edition and Commercial Edition. For service editions and billable items, see Billable items. For billing methods and upgrade instructions, see Billing methods. For concurrency limits, see Concurrency and QPS.
Endpoints
|
Access method |
Description |
Endpoint |
|
Public network |
Available to all servers over the public network. |
|
|
Internal network from Shanghai ECS |
For ECS instances in the China (Shanghai) region through a VPC. The classic network does not support access to the speech service. |
|
The Java SDK's NlsClient(String token) constructor uses the Shanghai public endpoint by default. To use the internal network, explicitly pass the internal endpoint when creating NlsClient. Internal network access does not incur public data transfer fees for the ECS instance.
Interaction flow
1. Authentication
The client uses an NLS token to authenticate the WebSocket connection. For token acquisition, see Obtain an access token. To obtain a project Appkey, see Manage projects. The Appkey and token must belong to the same account.
2. Start detection
In the Java SDK, pass the namespace to the CommonRequest constructor, set the Appkey with setAppKey(), and set audio parameters with addCustomedParam(). The parameters are described below:
|
Parameter |
Type |
Description |
|
|
String |
The namespace. Set to |
|
|
String |
The audio format. Valid values: |
|
|
Integer |
The audio sample rate in Hz. Default: 16000. |
For a complete example of client initialization, listener implementation, and audio transmission, see Java SDK. The following snippet assumes that NlsClient client and CommonRequestListener listener have been created and that appkey contains the project Appkey:
CommonRequest request = new CommonRequest(client, listener, "AudioEventDetection");
request.setAppKey(appkey);
request.addCustomedParam("format", "pcm");
request.addCustomedParam("sample_rate", 16000);
request.start();
Call start() to start the task. When the service returns TaskStarted, the SDK invokes onStarted. Task failures are reported through onFailed.
3. Receive detection results
Send audio data and receive sound event detection results through CommonRequestListener.onEvent. Each TaskResult message can contain a list of sound events. Example response:
{
"header":{
"namespace":"AudioEventDetection",
"name":"TaskResult",
"status":20000000,
"message_id":"1727504541a143e2bf8cfc4037d4****",
"task_id":"f926a956efaf4d30a823abafe97d****",
"status_text":"Gateway:SUCCESS:Success."
},
"payload":{
"number":1,
"result":[
{
"confidence":0.81,
"end_time":32000,
"event_type":"Music",
"start_time":30000
}
]
}
}
header object
|
Parameter |
Type |
Description |
|
|
String |
The namespace of the message. |
|
|
String |
The message name. |
|
|
Integer |
The service status code. |
|
|
String |
The status message. |
|
|
String |
The globally unique task ID. Use it for troubleshooting. |
|
|
String |
The ID of the current message. |
payload object
|
Parameter |
Type |
Description |
|
|
Integer |
The number of sound events in the current result. |
|
|
Array |
The list of sound events. The fields of each element are described below. |
Elements of the result array
|
Parameter |
Type |
Description |
|
|
Integer |
The start time of the sound event relative to the beginning of the input audio, in milliseconds. |
|
|
Integer |
The end time of the sound event relative to the beginning of the input audio, in milliseconds. |
|
|
String |
The sound event type. See the list of sound event types above. |
|
|
Float |
The confidence score of the current result. Valid values: [0.0, 1.0]. A higher value indicates greater confidence. |
4. End detection
After all audio data has been sent, call stop() to send StopTask. When processing is complete, the service returns TaskCompleted and the SDK invokes onStopped, indicating that the detection task has ended.
Service status codes
The header.status field indicates the service status. Handle errors according to the status code and status_text. For more information about common errors, see Error codes.
General error codes
|
Error code |
Cause |
Solution |
|
|
Authentication failed. |
Check whether the token is correct and has expired. |
|
|
Invalid message. |
Check whether the message meets the API requirements. |
|
|
Idle timeout. |
Check whether no data has been sent to the service for 10 consecutive seconds. |
|
|
Too many requests. |
Check whether the concurrency or requests-per-second limit has been exceeded. If the concurrency limit has been exceeded, upgrade to the Commercial Edition or increase its concurrency capacity as needed. |
|
|
Default client error. |
Troubleshoot according to the status message. |
|
|
The free trial has expired. |
To continue using the service, activate the Commercial Edition. For instructions, see Billing methods. |
|
|
Client timeout. |
The client has sent no data for 10 seconds or longer. Check whether audio transmission has been interrupted. |
|
|
Invalid sample rate. |
Check the sample rate against the input requirements. It must be 16,000 Hz. |
|
|
Invalid audio format. |
Check the audio format parameter. |
|
|
Audio decoding failed. |
Check whether the audio format is valid. |
Gateway error codes
|
Error code |
Cause |
Solution |
|
|
Unsupported API. |
Check whether the namespace is correct and the endpoint provides the service. |
|
|
Unsupported directive. |
Check whether the directive name meets the API requirements. |
|
|
Invalid directive. |
Check the directive format and parameters. |
|
|
The client disconnected prematurely. |
Check whether the connection was closed before the task completed. |
|
|
Invalid task state. |
Check whether the directive can be processed in the current task state. |
Configuration error codes
|
Error code |
Cause |
Solution |
|
|
The application does not exist. |
Check whether the project Appkey exists. |
|
|
The Appkey and token do not match. |
Check whether the Appkey is correct and belongs to the same account as the token. |
|
|
RAM user authentication failed. |
Use the Alibaba Cloud account to grant the RAM user permissions to access the POP API. |