The Mobi platform offers the Tongyi Tingwu real-time stream feature for easy real-time stream ingest. This document describes how to integrate the Tongyi Tingwu real-time stream.
What is the Tongyi Tingwu real-time stream
The Tongyi Tingwu real-time stream is a real-time speech recognition service that converts real-time speech input into text output. This feature is ideal for scenarios that require instant speech-to-text conversion, such as real-time meeting transcription and live captions.
The Tongyi Tingwu real-time stream must be used with Tongyi Tingwu integration operations. Specifically, you must create a real-time task and obtain an ingest URL through a Tongyi Tingwu integration operation before you can use the real-time stream service.
The process is as follows:
Create a real-time task using the Start real-time task operation in the Tongyi Tingwu integration.
Obtain the ingest URL returned by the integration.
Use the ingest URL to start the real-time transmission of audio data.
Trigger the stop real-time stream operation when the audio transmission is complete.
Stop the real-time task using the Stop real-time task operation in the Tongyi Tingwu integration.
Features
The Tongyi Tingwu real-time stream feature on the Mobi platform supports the following capabilities:
Real-time transcription: Speech input is converted to text in real time. Real-time translation is also supported if the translation feature is configured in the integration operation.
Integrated operation: Seamlessly integrates with other features of the Mobi platform, enabling you to use transcription results for further processing.
Callbacks and data transforms: Supports executing specific integration operation events when a speech-to-text sentence begins or is fully recognized. You can also customize the processing of returned data.
For more information about events, see Integration operation events.
Use the Tongyi Tingwu integration stream
Step 1: Trigger the "Start real-time task" operation for the Tongyi Tingwu integration
For more information, see Start real-time task.
Step 2: Trigger the "Start real-time task" operation for the Tongyi Tingwu integration
1. Create a Tingwu real-time stream operation.
2. Obtain the WebSocket ingest URL returned in Step 1.
3. Enter the ingest URL from the previous step in the Real-time stream operation.
4. Click Start Recording to begin real-time stream ingest.
Step 3: Use the microphone for interactive voice response
1. Use the microphone for interactive voice response. Monitor the results panel or status panel for the returned real-time stream data.
Step 4: Trigger the stop real-time stream operation
1. Click Stop Recording to stop the stream ingest.
Step 5: Trigger the "Stop real-time task" operation for the Tongyi Tingwu integration
For more information, see Stop real-time task.
Configuration panel details
Basic configurations
The following table describes the configuration panel parameters.
Ingest URL
Specifies the ingest URL for the Tongyi Tingwu real-time stream. You must provide the WebSocket ingest URL that is returned when you create the Tongyi Tingwu task integration operation.
General configurations
Parameters: Specifies the parameters required to complete the operation, including type, default value, and description. In the input box, you can reference parameters using the
{{ varName }}format.Description: Provides a detailed description of the integration operation.
Callback configurations
Data transform: Processes the data returned by the API. The default value is
return data, which means no processing is performed.Callback events: You can configure events to trigger when a speech transcription sentence begins or ends.
Manually trigger the integration
Event trigger
In the component where you need to trigger the integration operation, configure an event handler.
Set the action type to "Trigger Tongyi Tingwu real-time stream" or "Stop Tongyi Tingwu real-time stream" and set the trigger parameters as needed.
API trigger
If the name of this operation is tongyi, you can use the tongyi.start() or tongyi.stop() method to trigger the integration operation wherever a script can be executed. If the real-time stream operation has parameters configured, you must enter them in object format.