This topic describes the AI capabilities of speech transcription and how to implement this feature.
Speech transcription is a core feature of Tingwu that transcribes speech from audio and video files or real-time audio streams into text. This feature is the first and required node in the Tingwu API service chain and cannot be disabled. It supports languages such as Chinese, English, Cantonese, and Japanese. You can configure speaker diarization using the transcription parameters.
Request parameters
Parameter Name | Type | Required | Description |
Transcription | object | No | The object for speech recognition control parameters. |
Transcription.DiarizationEnabled | boolean | No | Specifies whether to enable speaker diarization during transcription. |
Transcription.Diarization | object | No | The speaker diarization object. |
Transcription.Diarization.SpeakerCount | int | No | 0: The number of speakers is not specified. 2: The number of speakers is two. |
Transcription.PhraseId | string | No | The ID of the hotword vocabulary. |
Transcription.Model | string | No | The speech transcription model. Use this parameter to call a domain-specific model to improve recognition accuracy in a specific domain. If this parameter is empty, the default model is used. The available parameters are:
|
For descriptions of other request parameters, see the document for your scenario:
The Transcription.Model parameter: Use this parameter to call a domain-specific model to improve recognition accuracy in a specific domain. The available domain-specific models are listed in the following table:
Model name | Parameter value | Supported language | Sample rate | Real-time/Offline | Scenarios |
Speech recognition model for sales conversations in the automotive domain | domain-automotive | Chinese | 16 kHz | Offline | Suitable for speech recognition in the automotive industry, including scenarios such as in-store reception, test drives, and car model promotions. |
Speech recognition model for online courses in the education domain | domain-education | Chinese | 16 kHz | Offline | Suitable for speech recognition in the education industry, including scenarios such as online courses. |
Example settings
// No settings
{
"Input":{
...
},
"Parameters":{
}
}
// Enable speaker diarization
{
"Input":{
...
},
"Parameters":{
"Transcription":{
"DiarizationEnabled":true,
"Diarization":{
"SpeakerCount":2
}
}
}
}
Code examples
#!/usr/bin/env python
#coding=utf-8
import os
import json
import datetime
from aliyunsdkcore.client import AcsClient
from aliyunsdkcore.request import CommonRequest
from aliyunsdkcore.auth.credentials import AccessKeyCredential
def create_common_request(domain, version, protocolType, method, uri):
request = CommonRequest()
request.set_accept_format('json')
request.set_domain(domain)
request.set_version(version)
request.set_protocol_type(protocolType)
request.set_method(method)
request.set_uri_pattern(uri)
request.add_header('Content-Type', 'application/json')
return request
def init_parameters():
root = dict()
root['AppKey'] = 'Enter the AppKey that you created in the Tingwu console'
# Basic request parameters
input = dict()
input['SourceLanguage'] = 'cn'
input['TaskKey'] = 'task' + datetime.datetime.now().strftime('%Y%m%d%H%M%S')
input['FileUrl'] = 'Enter the URL of the audio file to test'
root['Input'] = input
# AI-related parameters. Set them as needed.
parameters = dict()
# Speech recognition control
transcription = dict()
# Speaker diarization
transcription['DiarizationEnabled'] = True
diarization = dict()
diarization['SpeakerCount'] = 2
transcription['Diarization'] = diarization
parameters['Transcription'] = transcription
root['Parameters'] = parameters
return root
body = init_parameters()
print(body)
# TODO: Set your AccessKeyId and AccessKeySecret as environment variables.
credentials = AccessKeyCredential(os.environ['ALIBABA_CLOUD_ACCESS_KEY_ID'], os.environ['ALIBABA_CLOUD_ACCESS_KEY_SECRET'])
client = AcsClient(region_id='cn-beijing', credential=credentials)
request = create_common_request('tingwu.cn-beijing.aliyuncs.com', '2023-09-30', 'https', 'PUT', '/openapi/tingwu/v2/tasks')
request.add_query_param('type', 'offline')
request.set_content(json.dumps(body).encode('utf-8'))
response = client.do_action_with_exception(request)
print("response: \n" + json.dumps(json.loads(response), indent=4, ensure_ascii=False))package com.alibaba.tingwu.client.demo.aitest;
import com.alibaba.fastjson.JSONObject;
import com.aliyuncs.CommonRequest;
import com.aliyuncs.CommonResponse;
import com.aliyuncs.DefaultAcsClient;
import com.aliyuncs.IAcsClient;
import com.aliyuncs.exceptions.ClientException;
import com.aliyuncs.http.FormatType;
import com.aliyuncs.http.MethodType;
import com.aliyuncs.http.ProtocolType;
import com.aliyuncs.profile.DefaultProfile;
import org.junit.Test;
/**
* @author tingwu2023
*/
public class TranscriptionTest {
@Test
public void testFiletrans() throws ClientException {
CommonRequest request = createCommonRequest("tingwu.cn-beijing.aliyuncs.com", "2023-09-30", ProtocolType.HTTPS, MethodType.PUT, "/openapi/tingwu/v2/tasks");
request.putQueryParameter("type", "offline");
JSONObject root = new JSONObject();
root.put("AppKey", "Enter the AppKey that you created in the Tingwu console");
JSONObject input = new JSONObject();
input.fluentPut("FileUrl", "Enter the URL of the audio file to test")
.fluentPut("SourceLanguage", "cn")
.fluentPut("TaskKey", "task" + System.currentTimeMillis());
root.put("Input", input);
JSONObject parameters = new JSONObject();
JSONObject transcription = new JSONObject();
transcription.put("DiarizationEnabled", true);
JSONObject speakerCount = new JSONObject();
speakerCount.put("SpeakerCount", 2);
transcription.put("Diarization", speakerCount);
parameters.put("Transcription", transcription);
root.put("Parameters", parameters);
System.out.println(root.toJSONString());
request.setHttpContent(root.toJSONString().getBytes(), "utf-8", FormatType.JSON);
// TODO: Set your AccessKeyId and AccessKeySecret as environment variables.
DefaultProfile profile = DefaultProfile.getProfile("cn-beijing", System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
IAcsClient client = new DefaultAcsClient(profile);
CommonResponse response = client.getCommonResponse(request);
System.out.println(response.getData());
}
public static CommonRequest createCommonRequest(String domain, String version, ProtocolType protocolType, MethodType method, String uri) {
// Create an API request and set its parameters.
CommonRequest request = new CommonRequest();
request.setSysDomain(domain);
request.setSysVersion(version);
request.setSysProtocol(protocolType);
request.setSysMethod(method);
request.setSysUriPattern(uri);
request.setHttpContentType(FormatType.JSON);
return request;
}
}Example output
{
"Code":"0",
"Data":{
"TaskId":"10683ca4ad3f4f06bdf6e9dd********",
"TaskStatus":"COMPLETED",
"Result":{
"Transcription":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu_data/output/1738248129743478/10683ca4ad3f4f06bdf6e9dc1f3c1584/10683ca4ad3f4f06bdf6e9dc1f3c1584_Transcription_20231031165005.json?Expires=1698828606&LTAI****************&Signature=GMm%2BdN2tUPx*********ehu74%3D"
}
},
"Message":"success",
"RequestId":"7a63ee43-8a35-4ef2-8d5c-c76*********"
}The Transcription field provides the HTTP URL that you can use to download the speech transcription result.
Protocol parsing
The speech transcription result, which is available at the specified URL, is a JSON-formatted message. The following example shows the message structure. The Size, SampleRate, and Language fields in AudioInfo are not included in real-time transcription results.
{
"TaskId":"10683ca4ad3f4f06bdf6e9dc*********",
"Transcription":{
"AudioInfo": {
"Size": 670663,
"Duration": 10394,
"SampleRate": 48000,
"Language": "cn"
},
"Paragraphs":[
{
"ParagraphId":"16987422100275*******",
"SpeakerId":"1",
"Words":[
{
"Id":10,
"SentenceId":1,
"Start":4970,
"End":5560,
"Text":"Hello,"
},
{
"Id":20,
"SentenceId":1,
"Start":5730,
"End":6176,
"Text":"I am"
}
]
}
]
"AudioSegments": [
[12130, 16994],
[17000, 19720],
[19940, 28649]
]
}
}The fields are defined as follows.
Parameter name | Type | Description |
TaskId | string | The ID of the task. This ID is generated when the task is created. |
Transcription | object | The speech transcription result object. |
Transcription.Paragraphs | list[] | A collection of speech transcription results organized into paragraphs. |
Transcription.Paragraphs[i].ParagraphId | string | The paragraph-level ID. |
Transcription.Paragraphs[i].SpeakerId | string | The speaker ID. |
Transcription.Paragraphs[i].Words | list[] | The word information in the paragraph. |
Transcription.Paragraphs[i].Words[i].Id | int | The ordinal number of the word. You can usually ignore this parameter. |
Transcription.Paragraphs[i].Words[i].SentenceId | int | The sentence ID. Words that have the same SentenceId can be combined into a sentence. |
Transcription.Paragraphs[i].Words[i].Start | long | The start time of the word relative to the beginning of the audio. This is a relative timestamp in milliseconds. |
Transcription.Paragraphs[i].Words[i].End | long | The end time of the word relative to the beginning of the audio. This is a relative timestamp in milliseconds. |
Transcription.Paragraphs[i].Words[i].Text | string | The text of the word. |
Transcription.AudioInfo | object | The audio information object. |
Transcription.AudioInfo.Size | long | The size of the audio file in bytes. |
Transcription.AudioInfo.Duration | long | The duration of the audio in milliseconds. For real-time speech transcription, this field does not indicate the actual audio duration. |
Transcription.AudioInfo.SampleRate | int | The audio sample rate. |
Transcription.AudioInfo.Language | string | The language of the audio. |
Transcription.AudioSegments | list[][] | The range of valid audio segments. |
Transcription.AudioSegments[i][0] | int | The start time of the valid audio segment in milliseconds. |
Transcription.AudioSegments[i][1] | int | The end time of the valid audio segment in milliseconds. |
FAQ
Why is the recognition result empty after I upload a file?
Check whether the original audio or video file contains valid human speech or has excessive background noise.