Java SDK
Use the short sentence recognition Java SDK to send audio streams and receive recognition results. This topic provides SDK installation instructions, a complete example with client initialization, and guidance on sending audio, stopping recognition, and resolving dependency issues.
Before you begin
Activate Intelligent Speech Interaction and create a project. For instructions, see Quick start.
Prepare the project's Appkey and the AccessKey ID and AccessKey secret used to obtain an NLS token. For project and model configuration, see Manage projects. For token acquisition and validity, see Obtain a token.
Use JDK 8 or later. For audio encoding, sample rate, and model requirements, see the short sentence recognition API reference.
Install the SDK
Add the following dependencies to the Maven project's pom.xml file. The example uses Java SDK 2.2.1.
<dependency>
<groupId>com.alibaba.nls</groupId>
<artifactId>nls-sdk-recognizer</artifactId>
<version>2.2.1</version>
</dependency>
<!-- Add this dependency for JDK 9 and later. -->
<dependency>
<groupId>javax.xml.bind</groupId>
<artifactId>jaxb-api</artifactId>
<version>2.3.1</version>
</dependency>
Starting with version 2.1.0, nls-sdk-short-asr was renamed to nls-sdk-recognizer. When upgrading from an earlier version, remove the nls-sdk-short-asr dependency and implement the callback methods required by the compiler. If the project already depends on Netty, use version 4.1.17.Final or later.
Code example
Download nls-sample-en-16k.wav and place it in the program's working directory. This is a mono, 16 kHz WAV file. Select a project model that matches the audio language and sample rate. For other audio files, update the file name, format, and sample rate in the code accordingly.
Configure the following environment variables. The AccessKey pair and Appkey must belong to the account used to call the service.
|
Environment variable |
Description |
|
|
AccessKey ID used to obtain an NLS token. |
|
|
AccessKey secret used to obtain an NLS token. |
|
|
Appkey of the Intelligent Speech Interaction project. |
The AccessKey pair authenticates requests. Do not hard-code it or write it to logs. NLS tokens expire. For long-running applications, obtain a new token before expiration and update the client with NlsClient.setToken().
import com.alibaba.nls.client.AccessToken;
import com.alibaba.nls.client.protocol.InputFormatEnum;
import com.alibaba.nls.client.protocol.NlsClient;
import com.alibaba.nls.client.protocol.SampleRateEnum;
import com.alibaba.nls.client.protocol.asr.SpeechRecognizer;
import com.alibaba.nls.client.protocol.asr.SpeechRecognizerListener;
import com.alibaba.nls.client.protocol.asr.SpeechRecognizerResponse;
import java.io.FileInputStream;
import java.io.InputStream;
public class SpeechRecognizerDemo {
public static void main(String[] args) throws Exception {
String appKey = requireEnv("NLS_APP_KEY");
String accessKeyId = requireEnv("ALIYUN_AK_ID");
String accessKeySecret = requireEnv("ALIYUN_AK_SECRET");
String audioFile = "nls-sample-en-16k.wav";
// Obtain an NLS token. Refresh it and update the client before it expires.
AccessToken accessToken = new AccessToken(accessKeyId, accessKeySecret);
accessToken.apply();
NlsClient client = new NlsClient(
"wss://nls-gateway-cn-shanghai.aliyuncs.com/ws/v1", accessToken.getToken());
SpeechRecognizer recognizer = null;
try {
recognizer = new SpeechRecognizer(client, createListener());
recognizer.setAppKey(appKey);
// Match the format and sample rate to the audio file and project model.
recognizer.setFormat(InputFormatEnum.WAV);
recognizer.setSampleRate(SampleRateEnum.SAMPLE_RATE_16K);
recognizer.setEnableIntermediateResult(true);
recognizer.addCustomedParam("enable_voice_detection", true);
recognizer.start();
try (InputStream audio = new FileInputStream(audioFile)) {
// Simulate a live stream: send 3,200 bytes every 100 milliseconds.
recognizer.send(audio, 3200, 100);
}
recognizer.stop();
} finally {
if (recognizer != null) {
recognizer.close();
}
client.shutdown();
}
}
private static SpeechRecognizerListener createListener() {
return new SpeechRecognizerListener() {
@Override
public void onStarted(SpeechRecognizerResponse response) {
System.out.println("Task ID: " + response.getTaskId());
}
@Override
public void onRecognitionResultChanged(SpeechRecognizerResponse response) {
System.out.println("RecognitionResultChanged: " + response.getRecognizedText());
}
@Override
public void onRecognitionCompleted(SpeechRecognizerResponse response) {
System.out.println("RecognitionCompleted: " + response.getRecognizedText());
}
@Override
public void onFail(SpeechRecognizerResponse response) {
System.err.println("Task ID: " + response.getTaskId()
+ ", status: " + response.getStatus()
+ ", message: " + response.getStatusText());
}
};
}
private static String requireEnv(String name) {
String value = System.getenv(name);
if (value == null || value.isEmpty()) {
throw new IllegalArgumentException("Missing environment variable: " + name);
}
return value;
}
}
Recognition results are returned through onRecognitionResultChanged and onRecognitionCompleted. If a request fails, onFail provides the task ID, status code, and error information.
To access the service over the internal network from an Alibaba Cloud ECS instance in the Shanghai region, set the service URL in the NlsClient constructor to ws://nls-gateway-cn-shanghai-internal.aliyuncs.com/ws/v1.
Key interfaces
|
Interface |
Purpose and lifecycle |
|
|
A speech processing client for short sentence recognition, real-time speech recognition, and speech synthesis. It is thread-safe. We recommend reusing one instance globally and calling |
|
|
Configures short sentence recognition, sends audio, and stops recognition. It is not thread-safe. Create a separate instance for each task, do not reuse it for another task, and call |
|
|
Receives start, intermediate-result, completion, and failure events. It is not thread-safe. Use one listener per |
NlsClient uses Netty, and creating an instance consumes time and resources. Its lifecycle can match the application's lifecycle, while each recognition task creates and closes its own SpeechRecognizer. For more methods, see the Java API reference.
Run the sample project
You can also download the complete sample project and use its recognition and load-testing programs.
Download nls-sdk-java-demo.
Extract the package and run mvn package in the project root. The executable JAR is generated at nls-example-recognizer/target/nls-example-recognizer-2.0.0-jar-with-dependencies.jar. Copy the JAR and its sample audio to the target server, then run the following command. Supply the Appkey, NLS token, and optional service URL as prompted. The program writes logs/nls.log in its working directory.
java -cp nls-example-recognizer-2.0.0-jar-with-dependencies.jar com.alibaba.nls.client.SpeechRecognizerDemo
For load testing, run the following command and set the concurrency as prompted:
java -jar nls-example-recognizer-2.0.0-jar-with-dependencies.jar
The service URL is wss://nls-gateway-cn-shanghai.aliyuncs.com/ws/v1.
Use 16 kHz PCM audio for load testing, and keep concurrency within the quota of the activated service. Load-test calls are subject to the service's billing rules. Check the service edition, billing method, and concurrency quota before running a test.
FAQ
When should stop() be called?
After sending the audio, call recognizer.stop() to send StopRecognition and wait for completion. When voice activity detection (VAD) is enabled, the server can also return RecognitionCompleted after detecting the end of speech.
Send audio continuously in real time. An extended gap in audio transmission causes an idle timeout. To receive intermediate results, call setEnableIntermediateResult(true) before start().
How can a local audio file be recognized?
The example on this page reads a local file and sends its audio at a real-time rate. You can also upload an audio file through the RESTful API. For details, see the short sentence recognition RESTful API.
How is audio sent with send()?
Short sentence recognition provides four send() overloads:
public void send(byte[] data);
public void send(byte[] data, int length);
public void send(InputStream ins);
public void send(InputStream ins, int batchSize, int sleepInterval);
send(byte[]), send(byte[], int), and send(InputStream) send data as it arrives or is read. They do not automatically pace transmission according to audio duration. For live audio, send data continuously at the capture rate. To simulate a live stream from a file, use the overload with batchSize and sleepInterval to control the rate.
For 16-bit mono PCM audio at 16 kHz, 100 milliseconds corresponds to 3,200 bytes. At 8 kHz, 3,200 bytes corresponds to 200 milliseconds. For other encodings, adjust the batch size and interval according to the compressed data size. An interval that is too long can increase latency or disconnect the stream; one that is too short increases service and network resource consumption.
Based on the complete example above, use the following call for a 16 kHz audio file stream:
recognizer.send(audio, 3200, 100);
When reading a file in your own loop, use send(byte[], int), or call send(byte[]) for a complete 100-millisecond chunk:
recognizer.send(data);
Thread.sleep(100);
Here, data contains 3,200 bytes of 16 kHz, 16-bit mono PCM audio. For a final chunk smaller than 3,200 bytes, pass its actual length to send(data, length).
How can logs be used to analyze latency?
For short sentence recognition, calculate completion latency from the time StopRecognition is sent to the time RecognitionCompleted is received. For example, log timestamps of 14:24:44.844 and 14:24:44.984 correspond to 140 milliseconds.
The same approach applies to other speech processing tasks:
For real-time speech recognition, use the interval between
StopTranscriptionandTranscriptionCompleted.For speech synthesis, calculate first-packet latency from sending
StartSynthesisto receiving the first audio packet. For example, 14:32:13.813 to 14:32:14.035 corresponds to 222 milliseconds.
The complete sample project includes logging configuration. When integrating the SDK directly, configure log output using the application's logging framework. For RESTful API calls, record request and response timestamps in the client to calculate latency.
What if a com.alibaba JAR cannot be found?
Install the required dependencies when using Alibaba Cloud SDK for Java to obtain a token. For instructions, see V1.0 Java SDK.
How can the org.json.JSONArray.iterator() error be resolved?
If a legacy project that uses Alibaba Cloud SDK for Java to obtain a client or token reports that org.json.JSONArray.iterator()Ljava/util/Iterator does not exist, check that the JSON library loaded at runtime matches the version required by the SDK, and remove duplicate dependencies. Check that related dependencies, such as org.json:json and com.google.code.gson:gson, are complete and compatible with the Alibaba Cloud SDK version in use.