Sound event detection SDK API reference-Intelligent Speech Interaction(ISI)-阿里云帮助中心

Sound event detection detects and identifies sounds in audio, such as background music, crying, laughter, and explosions. It also annotates the start and end times of these sounds.

Billing and concurrency limits

Sound event detection offers two billing modes: Free Edition and Commercial Version. For more information, see Free Edition and Commercial Version. To upgrade from the Free Edition to the Commercial Version, see Upgrade from the Free Edition to the Commercial Version.
For more information about billing methods, see Billing methods.
For more information about concurrency limits, see Concurrency and QPS limits.

Sound event types

The service currently supports the following sound types:

Music: Background music
Singing: A person singing
Laughter: laughter
Knock: Knocking on a door or table
Keyboard: Typing on a keyboard
Cry: Sound of crying
Explosion: The sound of an explosion
Water: Sounds made by water
SirenAlarm: A siren sound

Usage notes

Supported input formats: PCM (uncompressed PCM or WAV files), 16-bit audio bit depth, and a mono sound channel.
The supported audio sample rate is 16,000 Hz.
Sound event detection currently supports only real-time streaming.

Endpoint

Access Type	Description	URL
Public network access	All servers can use the public network access URL. The software development kit (SDK) sets the public network access URL by default. You do not need to set it.	wss://nls-gateway-cn-shanghai.aliyuncs.com/ws/v1
Alibaba Cloud Shanghai ECS internal network access	If you use an Alibaba Cloud ECS instance in the China (Shanghai) region, you can use the internal network access URL. ECS instances in a classic network cannot access AnyTunnel. This means they cannot access the Voice Service over the internal network. To use AnyTunnel, create a VPC and access the service from within the VPC. Note Using the internal network access method does not incur data transfer costs for the ECS instance. For more information about ECS network types, see Network types.	ws://nls-gateway-cn-shanghai-internal.aliyuncs.com:80/ws/v1

Interaction flow

1. Authentication

The client uses a token for identity authentication when it establishes a WebSocket connection with the server-side. For more information about how to obtain a token, see Obtain a token.

2. Start detection

The client sends a request, and the server-side validates it. You can set the parameters in the request message using the `set` methods of the `CommonRequest` object in the SDK. The following table describes the parameters.

Parameter name	Type	Description
namespace	String	The namespace. When you create a `CommonRequest`, set this parameter to `AudioEventDetection`.
format	String	The audio coding format. Default value: PCM. Supported formats: PCM and WAV.
sample_rate	Integer	The audio sample rate. Default value: 16000. Unit: Hz.

3. Receive detection results

The client sends audio data in a loop and continuously receives the detection results.

The `onEvent` event indicates that the server-side has detected a sound event. The following code provides an example:

{
    "header":{
        "namespace":"AudioEventDetection",
        "name":"TaskResult",
        "status":20000000,
        "message_id":"1727504541a143e2bf8cfc4037d4****",
        "task_id":"f926a956efaf4d30a823abafe97d****",
        "status_text":"Gateway:SUCCESS:Success."
    },
    "payload":{
        "number":1,
        "result":[
            {
                "confidence":0.81,
                "end_time":32000,
                "event_type":"Music",
                "start_time":30000
            }
        ]
    }
}

The following table describes the parameters of the `header` object.

Parameter	Parameter type	Description
namespace	String	The namespace to which the message belongs.
name	String	The message name. `TaskResult` indicates an audio event.
status	Integer	The status code. It indicates whether the request was successful. For more information, see Service status codes.
status_text	String	The status message.
task_id	String	The globally unique ID (GUID) of the task. Record this value for troubleshooting.
message_id	String	The ID of this message.

The following table describes the parameters of the `payload` object.

Parameter name	Type	Description
number	Integer	The number of events.
start_time	Integer	The start timestamp of the audio. Unit: ms.
end_time	Integer	The end timestamp of the audio. Unit: ms.
event_type	String	The event type.
confidence	Float	The confidence level of the current result. Valid values: [0.0, 1.0]. A larger value indicates a higher confidence level.

4. End detection

To end the detection, notify the server-side after all audio data has been sent. The server-side then sends a notification to the client when the detection task is complete.

Service status codes

Each response from the service includes a `status` field, which contains the service status code. The following tables describe the general, gateway, and configuration error codes.

General error codes

Error code	Cause	Solution
40000001	Identity authentication failed	Check whether the token is correct and whether it has expired.
40000002	Invalid message	Check whether the sent message meets the requirements.
40000004	Idle timeout	Confirm whether data has not been sent to the server-side for a long time (10 seconds).
40000005	Too many requests	Check whether the number of concurrent connections or requests per second is exceeded. If the number of concurrent connections is exceeded, upgrade from the Free Edition to the Commercial Version, or scale out the resources for the Commercial Version.
40000000	Default client error code	Check the corresponding error message.
40000010	The 3-month free trial for new users has expired	To continue using the service, you must pay for the Commercial Version. Go to the console. On the Service Management and Activation page, find the target service and click Upgrade to Commercial Version.
41010120	Client timeout error	The client has not sent data for 10 or more consecutive seconds, which caused a client timeout error.
41160001	Incorrect sample rate parameter	Check the sample rate parameter settings.
41160002	Incorrect audio format parameter	Check the audio format parameter settings.
41160003	Audio decoding failed	Check whether the audio format is normal.

Gateway error codes

Error code	Cause	Solution
40010001	Unsupported API	Upgrade to the latest SDK.
40010002	Unsupported instruction	Upgrade to the latest SDK.
40010003	Invalid instruction	Upgrade to the latest SDK.
40010004	The client disconnected prematurely	Check whether the connection was closed before the request was completed.
40010005	Incorrect task status	An instruction was sent that cannot be processed in the current task state.

Configuration error codes

Error code	Cause	Solution
40020105	The application does not exist	The application cannot be found during routing parsing.
40020106	The AppKey and token do not match	Check whether the AppKey is correct and whether the token belongs to the same Alibaba Cloud account.
40020503	RAM user authentication failed	Use your Alibaba Cloud account to grant the RAM user the required access permissions to call the POP API.