Standard STT API

Interaction flow

Procedure

You must implement a speech-to-text (STT) model service that is accessible from the public network and supports the WebSocket protocol. Follow the input and output specifications in this document to add your self-developed STT model to a real-time workflow.

1. In the orchestration console, configure the following parameters in the speech-to-text node:

   In the Model section, select Self-developed.

   Parameter	Type	Required	Description	Example
   WebSocket URL	String	Yes	The WebSocket URL of your self-developed model.	wss://example.com/asr/ws
   ApiKey	String	Yes	The API authentication key.	AUJH-pf**************HBLKrI
   Custom parameters	String	No	Custom parameters.	key_a=value_a

2. After you complete the configuration, Alibaba Cloud establishes a connection with your service based on the following rules.

   Assume that your WSS URL is wss://example.com/asr/ws. A request is sent to wss://example.com/asr/ws?{request parameters}. The request parameters are as follows:

   Parameter	Type	Required	Description
   session_id	string	Yes	A unique identifier for the speech recognition connection.
   token	string	Yes	A signature string calculated from the session_id. For details, see Token calculation.
   language	string	No	The audio language. Defaults to Chinese and English.

   Request examples

   Without custom parameters

   wss://example.com/asr/ws?session_id=992204bfdca241e78dca2872625cf99f&token=muebPMT%2BnLe*********UJY4%3D&language=cn

   With custom parameters

   If you set custom parameters in the console, such as key_a=value_a and key_b=value_b, the final URL is:

   wss://example.com/asr/ws?session_id=992204bfdca241e78dca2872625cf99f&token=muebPMT%2BnLeTr**********4%3D&language=cn&key_a=value_a&key_b=value_b

   You can also set custom parameters when you call the API to launch an instance. This will override the configurations in your workflow. You need to specify the custom parameters for the STT node in AIAgentConfig.AsrConfig.CustomParams. These parameters are passed as URL parameters.

   {
       "AsrConfig": {
           "CustomParams": "key1=value1&key2=value2&key3=value3"     
       }
   }

3. After the connection is established, Alibaba Cloud streams PCM audio data to your model with the following specifications:

   Format	Data format	Channels	Sample rate	Protocol
   PCM	S16LE	mono	16 kHz	wss

4. Your self-developed model must return the speech recognition results to Alibaba Cloud as a JSON message that conforms to the following specifications:

   Parameter	Type	Required	Description
   session_id	String	Yes	The unique identifier for the ASR connection. This must be the same as the session_id from the connection request.
   name	String	Yes	The message name. Valid values: start: A notification message sent after the WebSocket connection is successfully established. result: A message that contains speech recognition results. error: A notification message for a server error.
   code	Int	Yes	The return code: 0: Success. Other values: Failure.
   message	String	Yes	A message describing the operation's result. On failure, this field contains the reason.
   result_type	Int	No	Required if name is result. Valid values: 0: temporary result 1: final result
   payload	Object	No	Required if name is result. You must specify all fields in the payload object.
   result	String	Yes	All currently recognized results.
   begin_time	Int	Yes	The start time of the recognized speech, measured in milliseconds (ms) as an offset from the time the first audio packet was received.
   end_time	Int	Yes	The end time of the recognized speech, measured in milliseconds (ms) as an offset from the time the first audio packet was received.

5. To end the transmission, send a binary message that indicates the end of the session.

   ws.send(bytes("{\"stop_session\": true}", encoding='utf-8'))

Code example

import hashlib
import hmac
import base64
from urllib.parse import quote
def calc_token():
    api_key = '12345678'
    session_id = '992204bfdca241e78dca2872625cf99f'
    sessionId = session_id.encode('utf-8')
    md5 = hashlib.md5()
    md5.update(sessionId)
    baseString = md5.hexdigest()
    baseString = bytes(baseString, encoding='utf-8')
    # Step 1: Calculate the MD5 hash of the session_id. Example result: f481faf07ec18481bc275a3ef3d61ea0
    apiKey = api_key.encode('utf-8')
    token = hmac.new(apiKey, baseString, hashlib.sha1).digest()
    # Step 2: Use the api_key to encrypt the MD5 hash with HMAC-SHA1. Example result: b'\x9a\xe7\x9b<\xc4\xfe\x9c\xb7\x93\xae\xbaY\xc3\x91|!\x8b\x14%\x8e'
    token = base64.b64encode(token)
    # Step 3: Base64-encode the encrypted data. Example result: muebPMT+nLeTrrpZw5F8IYsUJY4=
    token = str(token, 'utf-8') 
    token = quote(token)
    # Step 4: URL-encode the result. Example result: muebPMT%2BnLeTrrpZw5F8IYsUJY4%3D
    return token