This topic describes how to use the Python software development kit (SDK) for Alibaba Cloud Intelligent Speech Interaction. It includes installation instructions and code examples.
Prerequisites
Before you use the SDK, familiarize yourself with the API reference. For more information, see the API reference.
Download and install
The SDK supports only Python 3.
Ensure that setuptools, the Python package management tool, is installed. If it is not installed, run the following command in the terminal to install it:
pip install setuptools
Download the Python SDK.
You can download the Python SDK from Github or download the streamInputTts-github-python package directly.
Install SDK dependencies.
Navigate to the SDK root directory and run the following command to install the dependencies:
python -m pip install -r requirements.txtInstall the SDK.
After the dependencies are installed, run the following command to install the SDK:
python -m pip install .After installation, import the SDK using the following code:
# -*- coding: utf-8 -*- import nls
Run all the preceding commands in the SDK root directory.
Multi-threading and concurrency
In CPython, the Global Interpreter Lock prevents multiple threads from executing Python code simultaneously. However, some performance-oriented libraries may remove this limitation. To make better use of the computing resources on a multi-core computer, you can use multiprocessing or concurrent.futures.ProcessPoolExecutor. If you want to run multiple I/O-intensive tasks concurrently, multi-threading is still a suitable model.
If a single interpreter has too many threads, switching between them increases overhead and may cause SDK errors. We recommend that you do not use more than 200 threads. You can use the multiprocessing technique or manually create multiple interpreters using a script.
Key interfaces
The nls.NlsSpeechTranscriber class is used for real-time speech recognition. Its core methods are as follows:
1. Initialization (__init__)
Parameters
Parameter | Type | Description |
url | String | The WebSocket URL of the gateway. Default value: |
appkey | String | The Appkey. For more information about how to obtain an Appkey, see Manage projects. |
token | String | The access token. For more information, see Overview of how to obtain a token. |
on_start | Function | The callback parameter for when real-time speech recognition is ready. The callback parameters include the following:
The user-defined parameter is the content returned in the callback_args field below. |
on_sentence_begin | Function | The callback parameter for when a sentence in real-time speech recognition begins. The callback parameters include the following:
The user-defined parameter is the content returned in the callback_args field below. |
on_sentence_end | Function | The callback parameter for when a sentence in real-time speech recognition ends. The callback parameters include the following:
The user-defined parameter is the content returned in the callback_args field below. |
on_result_changed | Function | The callback parameter for when an intermediate result is returned by real-time speech recognition. The callback parameters include the following:
The user-defined parameter is the content returned in the callback_args field below. |
on_completed | Function | The callback parameter for when the final recognition result is returned by real-time speech recognition. The callback parameters include the following:
The user-defined parameter is the content returned in the callback_args field below. |
on_error | Function | The callback parameter for when an error occurs in the SDK or on the cloud. The callback parameters include the following:
The user-defined parameter is the content returned in the callback_args field below. |
on_close | Function | The callback parameter for when the connection to the cloud is disconnected. The callback parameter is the user-defined parameter. This is the content returned in the callback_args field below. |
callback_args | List | A list of user-defined parameters. The content of the list is packed into a List data structure and passed as the last parameter to each callback. |
Return value: None
2. start
Starts real-time speech recognition synchronously. This method blocks the current thread until the real-time speech recognition task is ready and the on_start callback is received.
Parameters
Parameter | Type | Description |
aformat | String | The format of the audio to be recognized. Supported formats: PCM, OPUS, and OPU. Default value: PCM. The SDK does not automatically encode PCM into OPUS or OPU. To use OPUS or OPU, you must implement the encoding yourself. |
sample_rate | Integer | The audio sampling rate for recognition. Default value: 16000 Hz. |
ch | Integer | The number of audio channels. Default value: 1. Only single-channel audio is supported. |
enable_intermediate_result | Boolean | Specifies whether to return intermediate results. Default value: False. |
enable_punctuation_prediction | Boolean | Specifies whether to predict punctuation for the recognition result. Default value: False. |
enable_inverse_text_normalization | Boolean | Inverse text normalization (ITN) converts Chinese numerals to Arabic numerals. If set to True, Chinese numerals are converted to Arabic numerals in the output. Default value: False. |
timeout | Integer | The timeout period for blocking. Default value: 10 seconds. |
ping_interval | Integer | The interval for sending ping packets. Default value: 8 seconds. Set to 0 or None to disable. |
ping_timeout | Integer | Specifies whether to check for ping packet timeouts. Default value: None. A value of None means ping packet timeouts are not checked. |
ex | Dict | Extra parameters provided by the user. The content of this dictionary is merged into the payload section of the request as |
Return value: A Boolean value. True indicates success. False indicates failure.
3. stop
Stops real-time speech recognition and waits synchronously until the on_completed callback is received.
Parameters
Parameter | Type | Description |
timeout | Integer | The timeout period for blocking. Default value: 10 seconds. |
Return value: A Boolean value. True indicates success. False indicates failure.
4. shutdown
Forcibly shuts down the current request. Calling this method multiple times has no additional effect.
Parameters: None
Return value: None
5. send_audio
Sends binary audio data. The format of the data must match the aformat parameter specified in the start method.
Parameters
Parameter | Type | Description |
pcm_data | Bytes | The binary audio data to send. The format must correspond to the aformat from the last call to start. The SDK does not automatically encode PCM into OPUS or OPU. To use OPUS or OPU, you must implement the encoding yourself. |
Return value: A Boolean value. True indicates success. False indicates failure.
6. ctrl
Sends a control command.
Parameters
Parameter | Type | Description |
ex | Dict | A custom control command. The content of this dictionary is merged into the payload section of the request as |
Return value: A Boolean value. True indicates success. False indicates failure.
Code example
The audio file used in this example has a sample rate of 16000 Hz and is in the PCM format. You can use the test1.pcm file from the tests folder. In the Intelligent Speech Interaction console, set the model for the project that corresponds to your AppKey to the General model to obtain accurate recognition results. If you use other audio files, set the model to one that is suitable for the audio scenario. For more information about model settings, see Manage projects.
This example uses the default public URL that is built into the SDK to access the server. If you use an ECS instance in the China (Shanghai) region and need to access the server over the internal network, use the following URL:
URL="ws://nls-gateway-cn-shanghai-internal.aliyuncs.com/ws/v1".
import time
import threading
import sys
import nls
URL="wss://nls-gateway-cn-shanghai.aliyuncs.com/ws/v1"
TOKEN="yourToken" # For information about how to obtain a token, see https://help.aliyun.com/document_detail/450255.html
APPKEY="yourAppkey" # To obtain an Appkey, go to the console: https://nls-portal.console.aliyun.com/applist
# The following code repeatedly performs real-time speech recognition (file transcription) on the content of an audio file.
class TestSt:
def __init__(self, tid, test_file):
self.__th = threading.Thread(target=self.__test_run)
self.__id = tid
self.__test_file = test_file
def loadfile(self, filename):
with open(filename, "rb") as f:
self.__data = f.read()
def start(self):
self.loadfile(self.__test_file)
self.__th.start()
def test_on_sentence_begin(self, message, *args):
print("test_on_sentence_begin:{}".format(message))
def test_on_sentence_end(self, message, *args):
print("test_on_sentence_end:{}".format(message))
def test_on_start(self, message, *args):
print("test_on_start:{}".format(message))
def test_on_error(self, message, *args):
print("on_error args=>{}".format(args))
def test_on_close(self, *args):
print("on_close: args=>{}".format(args))
def test_on_result_chg(self, message, *args):
print("test_on_chg:{}".format(message))
def test_on_completed(self, message, *args):
print("on_completed:args=>{} message=>{}".format(args, message))
def __test_run(self):
print("thread:{} start..".format(self.__id))
sr = nls.NlsSpeechTranscriber(
url=URL,
token=TOKEN,
appkey=APPKEY,
on_sentence_begin=self.test_on_sentence_begin,
on_sentence_end=self.test_on_sentence_end,
on_start=self.test_on_start,
on_result_changed=self.test_on_result_chg,
on_completed=self.test_on_completed,
on_error=self.test_on_error,
on_close=self.test_on_close,
callback_args=[self.__id]
)
print("{}: session start".format(self.__id))
r = sr.start(aformat="pcm",
enable_intermediate_result=True,
enable_punctuation_prediction=True,
enable_inverse_text_normalization=True)
self.__slices = zip(*(iter(self.__data),) * 640)
for i in self.__slices:
sr.send_audio(bytes(i))
time.sleep(0.01)
sr.ctrl(ex={"test":"tttt"})
time.sleep(1)
r = sr.stop()
print("{}: sr stopped:{}".format(self.__id, r))
time.sleep(1)
def multiruntest(num=500):
for i in range(0, num):
name = "thread" + str(i)
t = TestSt(name, "tests/test1.pcm")
t.start()
nls.enableTrace(False)
multiruntest(1)