This topic describes how to use the Java software development kit (SDK) for Alibaba Cloud Voice Service. It includes installation instructions and code examples.
Usage notes
Before you use the SDK, read the API reference. For more information, see API reference.
To use the long text-to-speech service, you must update the SDK to version 2.1.1 or later.
Download and installation
Download the latest SDK version from the Maven server.
The dependency files are as follows:
<dependency> <groupId>com.alibaba.nls</groupId> <artifactId>nls-sdk-tts</artifactId> <version>2.2.14</version> </dependency> <dependency> <groupId>com.alibaba.nls</groupId> <artifactId>nls-sdk-common</artifactId> <version>2.2.14</version> </dependency>ImportantStarting from Java SDK version 2.1.7, the unit of the timeout period for the waitForComplete interface is changed from seconds to milliseconds.
Unzip the ZIP file and run
mvn packagein the pom directory. An executable JAR file named nls-example-long-tts-2.0.0-jar-with-dependencies.jar is generated in the target directory. You can copy this JAR file to the destination server for quick verification and stress testing.Verify the service.
Run the following code and provide the required parameters as prompted. After the command runs, the logs/nls.log file is generated in the directory where you run the command.
java -cp nls-example-long-tts-2.0.0-jar-with-dependencies.jar com.alibaba.nls.client.SpeechSynthesizerDemoPerform stress testing.
Run the following code and provide the required parameters as prompted. The Alibaba Cloud service URL is
wss://nls-gateway-cn-shanghai.aliyuncs.com/ws/v1. Select the number of concurrent connections based on your purchase.java -jar nls-example-long-tts-2.0.0-jar-with-dependencies.jarImportantIf you perform stress testing with more than two concurrent connections, you will incur fees.
Key interfaces
NlsClient: The client for voice processing. You can use this client to perform tasks such as short sentence recognition, real-time speech recognition, and speech synthesis. This client is thread-safe. Create only one global instance.
SpeechSynthesizer: The class for speech synthesis. It is used to set request parameters and send requests. This class is not thread-safe.
SpeechSynthesizerListener: The listener class for speech synthesis. It listens for returned results. This class is not thread-safe. You must implement the following two abstract methods:
/** * Receives binary data for speech synthesis. */ abstract public void onMessage(ByteBuffer message); /** * Notifies when the speech synthesis is complete. */ abstract public void onComplete(SpeechSynthesizerResponse response);
Notes on SDK calls:
NlsClient uses the Netty framework. Creating an NlsClient object is time-consuming and resource-intensive. After an object is created, it can be reused. Align the creation and shutdown of the NlsClient object with the lifecycle of your program.
SpeechSynthesizer objects cannot be reused. Each speech synthesis task corresponds to one SpeechSynthesizer object. For example, to perform N speech synthesis tasks for N texts, you must create N SpeechSynthesizer objects.
SpeechSynthesizerListener objects and SpeechSynthesizer objects have a one-to-one relationship. Do not assign one SpeechSynthesizerListener object to multiple SpeechSynthesizer objects. Otherwise, you cannot distinguish between the speech synthesis tasks.
The Java SDK depends on the Netty network library. If your application also depends on Netty, you must update Netty to version 4.1.17.Final or later.
Code example
The example uses the default public endpoint of the speech synthesis service that is built into the SDK. If you use an Alibaba Cloud ECS instance in the China (Shanghai) region and need to access the service over an internal network, set the internal endpoint when you create the NlsClient object:
client = new NlsClient("ws://nls-gateway-cn-shanghai-internal.aliyuncs.com/ws/v1", accessToken);The example saves the synthesized audio to a file. If you have high requirements for real-time performance, use streaming playback. Play the audio data as you receive it to reduce latency.
package com.alibaba.nls.client;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import com.alibaba.nls.client.protocol.NlsClient;
import com.alibaba.nls.client.protocol.OutputFormatEnum;
import com.alibaba.nls.client.protocol.SampleRateEnum;
import com.alibaba.nls.client.protocol.tts.SpeechSynthesizer;
import com.alibaba.nls.client.protocol.tts.SpeechSynthesizerListener;
import com.alibaba.nls.client.protocol.tts.SpeechSynthesizerResponse;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* This demo shows:
* How to call the long text-to-speech API (setLongText).
* Streaming text-to-speech (TTS) synthesis.
* How to calculate the first-packet latency.
*
* Note: This demo is different from SpeechSynthesizerLongTextDemo in nls-example-tts. Long text-to-speech is a separate feature where a long string of text is sent directly to the server for synthesis.
* In contrast, SpeechSynthesizerLongTextDemo shows how to split a long text on the client side and then call the speech synthesis API for each segment.
*/
public class SpeechLongSynthesizerDemo {
private static final Logger logger = LoggerFactory.getLogger(SpeechLongSynthesizerDemo.class);
private static long startTime;
private String appKey;
NlsClient client;
public SpeechLongSynthesizerDemo(String appKey, String token, String url) {
this.appKey = appKey;
// Create only one global NlsClient instance. Its lifecycle can be the same as that of the application. The default endpoint is the online service endpoint of Alibaba Cloud.
if(url.isEmpty()) {
client = new NlsClient(token);
} else {
client = new NlsClient(url, token);
}
}
private static SpeechSynthesizerListener getSynthesizerListener() {
SpeechSynthesizerListener listener = null;
try {
listener = new SpeechSynthesizerListener() {
File f=new File("ttsForLongText.wav");
FileOutputStream fout = new FileOutputStream(f);
private boolean firstRecvBinary = true;
// Speech synthesis is complete.
@Override
public void onComplete(SpeechSynthesizerResponse response) {
// When onComplete is called, it indicates that all TTS data has been received. This is the latency for the entire synthesis process. This latency may be long and may not be suitable for real-time scenarios.
System.out.println("name: " + response.getName() + ", status: " + response.getStatus()+", output file :"+f.getAbsolutePath());
}
// Binary voice data from speech synthesis.
@Override
public void onMessage(ByteBuffer message) {
try {
if(firstRecvBinary) {
// Calculate the first-packet latency of the audio stream here. You can start playing the audio as soon as the first packet is received. This improves the response speed, especially in real-time interactive scenarios.
firstRecvBinary = false;
long now = System.currentTimeMillis();
logger.info("tts first latency : " + (now - SpeechLongSynthesizerDemo.startTime) + " ms");
}
byte[] bytesArray = new byte[message.remaining()];
message.get(bytesArray, 0, bytesArray.length);
//System.out.println("write array:" + bytesArray.length);
fout.write(bytesArray);
} catch (IOException e) {
e.printStackTrace();
}
}
@Override
public void onFail(SpeechSynthesizerResponse response){
// The task_id is the unique identifier for the communication between the client and the server. If you encounter a problem, provide this task_id for troubleshooting.
System.out.println(
"task_id: " + response.getTaskId() +
// Status code
", status: " + response.getStatus() +
// Error message
", status_text: " + response.getStatusText());
}
};
} catch (Exception e) {
e.printStackTrace();
}
return listener;
}
public void process(String text) {
SpeechSynthesizer synthesizer = null;
try {
// Create an instance and establish a connection.
synthesizer = new SpeechSynthesizer(client, getSynthesizerListener());
synthesizer.setAppKey(appKey);
// Set the encoding format of the returned audio.
synthesizer.setFormat(OutputFormatEnum.WAV);
// Set the sample rate of the returned audio.
synthesizer.setSampleRate(SampleRateEnum.SAMPLE_RATE_16K);
// The voice. Note that the Java SDK does not support ultra-high definition voices such as "zhiqi". To use these voices, call the RESTful API.
synthesizer.setVoice("siyue");
// The pitch. The range is -500 to 500. This parameter is optional. The default value is 0.
synthesizer.setPitchRate(0);
// The speech rate. The range is -500 to 500. The default value is 0.
synthesizer.setSpeechRate(0);
// Set the text for speech synthesis.
// The setLongText interface is called here. The original speech synthesis interface is setText.
synthesizer.setLongText(text);
// This method serializes the preceding parameters into a JSON string, sends it to the server, and waits for the server to respond.
long start = System.currentTimeMillis();
synthesizer.start();
logger.info("tts start latency " + (System.currentTimeMillis() - start) + " ms");
SpeechLongSynthesizerDemo.startTime = System.currentTimeMillis();
// Wait for the speech synthesis to complete.
synthesizer.waitForComplete();
logger.info("tts stop latency " + (System.currentTimeMillis() - start) + " ms");
} catch (Exception e) {
e.printStackTrace();
} finally {
// Close the connection.
if (null != synthesizer) {
synthesizer.close();
}
}
}
public void shutdown() {
client.shutdown();
}
public static void main(String[] args) throws Exception {
String appKey = "";
String token = "your_token";
// Use the default URL.
String url = "wss://nls-gateway-cn-shanghai.aliyuncs.com/ws/v1";
if (args.length == 2) {
appKey = args[0];
token = args[1];
} else if (args.length == 3) {
appKey = args[0];
token = args[1];
url = args[2];
} else {
System.err.println("Run error. Required parameters (URL is optional): " + "<app-key> <token> [url]");
System.exit(-1);
}
String ttsTextLong = "From Baicao Garden to Sanwei Study by Lu Xun \n" +
"Behind my house was a large garden, known as the Baicao Garden. It has long since been sold, along with the house, to the descendants of Zhu Wengong. It has been seven or eight years since I last saw it. It seems there were only weeds there then, but at that time, it was my paradise.\n" +
"Needless to say, there were green vegetable plots, smooth stone well railings, tall soapberry trees, and purplish-red mulberries. Nor need I mention the cicadas singing in the leaves, the fat wasps on the vegetable flowers, or the nimble larks suddenly soaring from the grass into the sky.\n" +
"Just along the base of the short mud wall, there was endless fun. Tree crickets chirped softly, and field crickets played their tunes. Turning over a broken brick, you might sometimes find a centipede. There were also blister beetles. If you pressed its back with your finger, it would let out a puff of smoke from its rear with a pop.\n" +
"The vines of fleeceflower and manglietia intertwined. The manglietia had fruit like a lotus pod, and the fleeceflower had swollen roots. Some said that the fleeceflower root could be shaped like a human, and eating it would grant immortality. So I often pulled them up, one after another, sometimes damaging the mud wall in the process, but I never saw a root that looked like a person! If you were not afraid of thorns, you could also pick raspberries, which looked like little balls of tiny coral beads. They were sour and sweet, and their color and taste were far better than mulberries...";
SpeechLongSynthesizerDemo demo = new SpeechLongSynthesizerDemo(appKey, token, url);
demo.process(ttsTextLong);
demo.shutdown();
}
}