Python SDK

更新时间:
复制 MD 格式

This topic describes how to use the Python software development kit (SDK) for Alibaba Cloud Intelligent Speech Interaction. It includes installation instructions and code examples.

Prerequisites

Before you use the SDK, familiarize yourself with the API reference. For more information, see the API reference.

Download and install

Note
  • The SDK supports only Python 3.

  • Ensure that setuptools, the Python package management tool, is installed. If it is not installed, run the following command in the terminal to install it:

    pip install setuptools
  1. Download the Python SDK.

    You can download the Python SDK from Github or download the streamInputTts-github-python package directly.

  2. Install SDK dependencies.

    Navigate to the SDK root directory and run the following command to install the dependencies:

    python -m pip install -r requirements.txt
  3. Install the SDK.

    After the dependencies are installed, run the following command to install the SDK:

    python -m pip install .
  4. After installation, import the SDK using the following code:

    # -*- coding: utf-8 -*-
    import nls
Important

Run all the preceding commands in the SDK root directory.

Multi-threading and concurrency

In CPython, the Global Interpreter Lock prevents multiple threads from executing Python code simultaneously. However, some performance-oriented libraries may remove this limitation. To make better use of the computing resources on a multi-core computer, you can use multiprocessing or concurrent.futures.ProcessPoolExecutor. If you want to run multiple I/O-intensive tasks concurrently, multi-threading is still a suitable model.

If a single interpreter has too many threads, switching between them increases overhead and may cause SDK errors. We recommend that you do not use more than 200 threads. You can use the multiprocessing technique or manually create multiple interpreters using a script.

Key interfaces

The nls.NlsSpeechTranscriber class is used for real-time speech recognition. Its core methods are as follows:

1. Initialization (__init__)

  • Parameters

Parameter

Type

Description

url

String

The WebSocket URL of the gateway. Default value: wss://nls-gateway-cn-shanghai.aliyuncs.com/ws/v1.

appkey

String

The Appkey. For more information about how to obtain an Appkey, see Manage projects.

token

String

The access token. For more information, see Overview of how to obtain a token.

on_start

Function

The callback parameter for when real-time speech recognition is ready. The callback parameters include the following:

  • A string in JSON format

  • A user-defined parameter

The user-defined parameter is the content returned in the callback_args field below.

on_sentence_begin

Function

The callback parameter for when a sentence in real-time speech recognition begins. The callback parameters include the following:

  • A string in JSON format

  • A user-defined parameter

The user-defined parameter is the content returned in the callback_args field below.

on_sentence_end

Function

The callback parameter for when a sentence in real-time speech recognition ends. The callback parameters include the following:

  • A string in JSON format

  • A user-defined parameter

The user-defined parameter is the content returned in the callback_args field below.

on_result_changed

Function

The callback parameter for when an intermediate result is returned by real-time speech recognition. The callback parameters include the following:

  • A string in JSON format

  • A user-defined parameter

The user-defined parameter is the content returned in the callback_args field below.

on_completed

Function

The callback parameter for when the final recognition result is returned by real-time speech recognition. The callback parameters include the following:

  • A string in JSON format

  • A user-defined parameter

The user-defined parameter is the content returned in the callback_args field below.

on_error

Function

The callback parameter for when an error occurs in the SDK or on the cloud. The callback parameters include the following:

  • A string in JSON format

  • A user-defined parameter

The user-defined parameter is the content returned in the callback_args field below.

on_close

Function

The callback parameter for when the connection to the cloud is disconnected. The callback parameter is the user-defined parameter. This is the content returned in the callback_args field below.

callback_args

List

A list of user-defined parameters. The content of the list is packed into a List data structure and passed as the last parameter to each callback.

  • Return value: None

2. start

Starts real-time speech recognition synchronously. This method blocks the current thread until the real-time speech recognition task is ready and the on_start callback is received.

  • Parameters

Parameter

Type

Description

aformat

String

The format of the audio to be recognized. Supported formats: PCM, OPUS, and OPU. Default value: PCM.

The SDK does not automatically encode PCM into OPUS or OPU. To use OPUS or OPU, you must implement the encoding yourself.

sample_rate

Integer

The audio sampling rate for recognition. Default value: 16000 Hz.

ch

Integer

The number of audio channels. Default value: 1. Only single-channel audio is supported.

enable_intermediate_result

Boolean

Specifies whether to return intermediate results. Default value: False.

enable_punctuation_prediction

Boolean

Specifies whether to predict punctuation for the recognition result. Default value: False.

enable_inverse_text_normalization

Boolean

Inverse text normalization (ITN) converts Chinese numerals to Arabic numerals. If set to True, Chinese numerals are converted to Arabic numerals in the output. Default value: False.

timeout

Integer

The timeout period for blocking. Default value: 10 seconds.

ping_interval

Integer

The interval for sending ping packets. Default value: 8 seconds. Set to 0 or None to disable.

ping_timeout

Integer

Specifies whether to check for ping packet timeouts. Default value: None. A value of None means ping packet timeouts are not checked.

ex

Dict

Extra parameters provided by the user. The content of this dictionary is merged into the payload section of the request as key:value pairs. For more information, see the request data in the API reference topic.

  • Return value: A Boolean value. True indicates success. False indicates failure.

3. stop

Stops real-time speech recognition and waits synchronously until the on_completed callback is received.

  • Parameters

Parameter

Type

Description

timeout

Integer

The timeout period for blocking. Default value: 10 seconds.

  • Return value: A Boolean value. True indicates success. False indicates failure.

4. shutdown

Forcibly shuts down the current request. Calling this method multiple times has no additional effect.

  • Parameters: None

  • Return value: None

5. send_audio

Sends binary audio data. The format of the data must match the aformat parameter specified in the start method.

  • Parameters

Parameter

Type

Description

pcm_data

Bytes

The binary audio data to send. The format must correspond to the aformat from the last call to start.

The SDK does not automatically encode PCM into OPUS or OPU. To use OPUS or OPU, you must implement the encoding yourself.

  • Return value: A Boolean value. True indicates success. False indicates failure.

6. ctrl

Sends a control command.

  • Parameters

Parameter

Type

Description

ex

Dict

A custom control command. The content of this dictionary is merged into the payload section of the request as key:value pairs. For more information, see the request data in the API reference topic.

  • Return value: A Boolean value. True indicates success. False indicates failure.

Code example

Note
  • The audio file used in this example has a sample rate of 16000 Hz and is in the PCM format. You can use the test1.pcm file from the tests folder. In the Intelligent Speech Interaction console, set the model for the project that corresponds to your AppKey to the General model to obtain accurate recognition results. If you use other audio files, set the model to one that is suitable for the audio scenario. For more information about model settings, see Manage projects.

  • This example uses the default public URL that is built into the SDK to access the server. If you use an ECS instance in the China (Shanghai) region and need to access the server over the internal network, use the following URL: URL="ws://nls-gateway-cn-shanghai-internal.aliyuncs.com/ws/v1".

import time
import threading
import sys

import nls


URL="wss://nls-gateway-cn-shanghai.aliyuncs.com/ws/v1"
TOKEN="yourToken"  # For information about how to obtain a token, see https://help.aliyun.com/document_detail/450255.html
APPKEY="yourAppkey"    # To obtain an Appkey, go to the console: https://nls-portal.console.aliyun.com/applist

# The following code repeatedly performs real-time speech recognition (file transcription) on the content of an audio file.
class TestSt:
    def __init__(self, tid, test_file):
        self.__th = threading.Thread(target=self.__test_run)
        self.__id = tid
        self.__test_file = test_file
   
    def loadfile(self, filename):
        with open(filename, "rb") as f:
            self.__data = f.read()
    
    def start(self):
        self.loadfile(self.__test_file)
        self.__th.start()

    def test_on_sentence_begin(self, message, *args):
        print("test_on_sentence_begin:{}".format(message))

    def test_on_sentence_end(self, message, *args):
        print("test_on_sentence_end:{}".format(message))

    def test_on_start(self, message, *args):
        print("test_on_start:{}".format(message))

    def test_on_error(self, message, *args):
        print("on_error args=>{}".format(args))

    def test_on_close(self, *args):
        print("on_close: args=>{}".format(args))

    def test_on_result_chg(self, message, *args):
        print("test_on_chg:{}".format(message))

    def test_on_completed(self, message, *args):
        print("on_completed:args=>{} message=>{}".format(args, message))


    def __test_run(self):
        print("thread:{} start..".format(self.__id))
        sr = nls.NlsSpeechTranscriber(
                    url=URL,
                    token=TOKEN,
                    appkey=APPKEY,
                    on_sentence_begin=self.test_on_sentence_begin,
                    on_sentence_end=self.test_on_sentence_end,
                    on_start=self.test_on_start,
                    on_result_changed=self.test_on_result_chg,
                    on_completed=self.test_on_completed,
                    on_error=self.test_on_error,
                    on_close=self.test_on_close,
                    callback_args=[self.__id]
                )

        print("{}: session start".format(self.__id))
        r = sr.start(aformat="pcm",
        enable_intermediate_result=True,
        enable_punctuation_prediction=True,
        enable_inverse_text_normalization=True)

        self.__slices = zip(*(iter(self.__data),) * 640)
        for i in self.__slices:
            sr.send_audio(bytes(i))
            time.sleep(0.01)

        sr.ctrl(ex={"test":"tttt"})
        time.sleep(1)

        r = sr.stop()
        print("{}: sr stopped:{}".format(self.__id, r))
        time.sleep(1)

def multiruntest(num=500):
    for i in range(0, num):
        name = "thread" + str(i)
        t = TestSt(name, "tests/test1.pcm")
        t.start()

nls.enableTrace(False)
multiruntest(1)