This topic describes the flow for ingesting audio and video recordings.
Audio and video file transcription is a service that performs offline processing on pre-recorded audio or video files. This processing includes features such as speech recognition, translation, key point extraction, summary generation, and PowerPoint (PPT) extraction and summarization.
Offline transcription is designed for non-real-time scenarios. You can submit a file by providing a URL that is accessible over HTTP or HTTPS. The service does not support submitting local files or files that use other protocols, such as FTP.
Usage notes
The service supports single-track or dual-track audio files in MP3, WAV, M4A, WMA, AAC, OGG, AMR, FLAC, and AIFF formats, and video files in MP4, WMV, M4V, FLV, RMVB, DAT, MOV, MKV, WEBM, AVI, MPEG, 3GP, and OGG formats.
The file size cannot exceed 6 GB.
The audio duration cannot exceed 6 hours.
The audio sampling rate can be 8 kHz, 16 kHz, 24 kHz, or 48 kHz.
The audio or video file for transcription must be accessible and downloadable from a public URL.
The access permissions for the submitted audio or video file URL must be set to public. The URL must use a domain name instead of an IP address. The URL cannot contain spaces, and we do not recommend using Chinese characters in the URL.
We recommend using Alibaba Cloud OSS. If file access permissions in OSS are set to public, you can obtain a file access URL. For more information, see Using Signed URLs to Download or Preview Files. If file access permissions in OSS are set to private, you can generate a time-limited access URL using the SDK. For more information, see Using Signed URLs to Download or Preview Files.
Alternatively, you can store the recording files on your own file server and provide a download link. Ensure that the `Content-Length` value in the HTTP response header matches the actual size of the data in the body. Otherwise, the download fails.
Offline tasks may be queued. To ensure that tasks are completed within 3 hours, if you use a time-limited access URL, you must set its validity period to at least 3 hours. This prevents the URL from expiring before the audio or video file can be downloaded and processed.
The service supports two invocation methods: polling and callback. You can choose either method.
The service supports speech recognition for multiple languages, including Chinese, English, Cantonese, Mixed Chinese and English (for long, mixed-language segments), Japanese, and Korean. For more information, see Audio and video file service parameter table.
The Queries Per Second (QPS) limits for the APIs are as follows:
The user-level QPS limit for the Create Tingwu Task API is 20.
The user-level QPS limit for the Query task status and task result API is 100.
After you submit an offline transcription task, the task is typically completed within 3 hours. The recognized text and Large Language Model (LLM) results, if any, are then returned. This timeframe does not apply to large-scale data uploads, such as uploading more than 500 hours of recordings within 30 minutes.
To process large-scale offline files, contact our pre-sales experts for further discussion.
If your audio or video files are stored in the Alibaba Cloud OSS Beijing region, we strongly recommend that you generate an internally signed URL before you submit jobs. This practice reduces public network costs and speeds up job processing. For more information, see ECS Instances Access OSS Resources Through OSS Internal Endpoints.
If you exceed the QPS limit, the following error may occur: Throttling.User : Request was denied due to user flow control.. We recommend that you set a longer polling interval for the query API to avoid being rate-limited.
Interaction flow
The interaction flow between the client and the server is shown in the following figure:

Integration steps
The client submits a request to transcribe a recording file.
If the request is successful, the server returns a TaskId. You can use this TaskId to query the transcription result later.
If an error occurs, the response message includes error information and the RequestId. You can use this information to troubleshoot the request parameters.
You can query the processing result using the TaskId that is returned after you submit the task.
If you use this method to poll for results, do not set the polling frequency too high, which helps avoid rate limiting. For example, you can query the result at a frequency of every 1 minute or every 5 minutes.
You can use callback notifications to retrieve the results. Unlike active polling, this method lets you wait for the server to proactively send notifications about the task status after you submit the task.
We currently support HTTP callbacks to send notifications about the task processing status.
After a task is completed, you can add new parameters to trigger new algorithm processing and refresh the task result.
Prerequisites
Set AccessKey environment variables
Replace `YOUR_ACCESS_KEY_ID` and `YOUR_ACCESS_KEY_SECRET` in the following command with your AccessKey ID and AccessKey secret.
export ALIBABA_CLOUD_ACCESS_KEY_ID=YOUR_ACCESS_KEY_ID &&
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=YOUR_ACCESS_KEY_SECRETSubmit an offline audio or video transcription task
To create an audio or video offline transcription job, see API details. The common parameters are described in the following table.
Request parameters
Name | Parameter | Type | Default | Description |
AppKey | AppKey | string | - | Required. Set this to the AppKey you created in the console. |
Basic request information (Input) | Input.FileUrl | string | - | Required. The download URL of the audio or video file. HTTP and HTTPS URLs are supported. |
Input.SourceLanguage | string | - | Required, Configure the corresponding parameters based on the language categories in audio and video.
| |
Input.LanguageHints | list[string] | null | List of languages for speech recognition when using a multilingual model. When the audio contains multiple languages that all need to be recognized, this parameter is used to limit the language categories. It takes effect only when `Input.SourceLanguage` is set to `multilingual`. | |
Input.TaskKey | string | null | A custom ID set by the user. | |
Input.ProgressiveCallbacksEnabled | boolean | false | Specifies whether to enable the callback feature. To enable the callback feature, configure the callback type and address in the console and set this parameter to `true` when creating the task. | |
Transcoding | Parameters.Transcoding.TargetAudioFormat | string | null | Currently, only conversion to MP3 is supported. |
Parameters.Transcoding.TargetVideoFormat | string | null | Currently, only conversion to MP4 is supported. This parameter is meaningful only if the source file is a video. | |
Speech recognition (Transcription) | Parameters.Transcription.DiarizationEnabled | boolean | false | Specifies whether to enable speaker diarization during speech recognition. |
Parameters.Transcription.Diarization.SpeakerCount | int | - | The number of speakers to set when speaker diarization is enabled. A maximum of 100 speakers is supported.
If this parameter is set, it helps specify the number of speakers. The final number of separated tracks is based on the actual number of classified speakers. | |
Parameters.Transcription.PhraseId | String | - | The ID of the hotword vocabulary. | |
Translation (Translation) | Parameters.TranslationEnabled | boolean | false | Specifies whether to enable the translation feature. |
Parameters.Translation.TargetLanguages | list[string] | - | If translation is enabled, you must set the target translation languages. Supported languages include Chinese (cn), English (en), Japanese (ja), Korean (ko), German (de), French (fr), and Russian (ru). | |
Chapter overview | Parameters.AutoChaptersEnabled | boolean | false | The chapter overview feature, which includes agenda titles and agenda summaries. |
Key point extraction (MeetingAssistance) | Parameters.MeetingAssistanceEnabled | boolean | false | The key point extraction feature, which includes keywords, key content, to-do items, and scenario recognition. |
Parameters.MeetingAssistance.Types | list[string] | - | If you enable the key point extraction feature, you can set one or more of the following types:
Note: If you only enable the feature but do not add any algorithm types, the model processes all algorithm models. | |
Summarization (Summarization) | Parameters.SummarizationEnabled | boolean | false | Specifies whether to enable the summarization feature. |
Parameters.Summarization.Types | list[string] | - | If you enable the summarization feature, you must set the summary types. You can set one or more types.
| |
PPT extraction and summarization | Parameters.PptExtractionEnabled | boolean | false | Specifies whether to enable the PPT feature, which includes PPT extraction and PPT summarization. Note: This feature only supports PPTs in the main view (screen sharing or with a person's video nearby). It does not support scenarios where a person is walking or speaking in front of the PPT. You can test the effect on the Tongyi Tingwu website. Click here to test |
Spoken-to-written text conversion | Parameters.TextPolishEnabled | boolean | false | Specifies whether to enable the spoken-to-written text conversion feature. |
Service quality inspection | Parameters.ServiceInspectionEnabled | boolean | false | Specifies whether to enable the service quality inspection feature. |
Parameters.ServiceInspection | object | - | Refer to the Service Quality Inspection Parameters Object to conduct quality inspection of dialogues during service processes. It supports customizing multiple quality inspection dimensions to help customers improve their service levels. | |
Custom Prompt | Parameters.CustomPromptEnabled | boolean | false | Specifies whether to enable the custom prompt feature. |
Parameters.CustomPrompt | object | - | Allows you to define custom prompts to guide the Large Language Model (LLM) in completing various tasks. For more information, see Custom prompt parameter object. | |
Global parameters for LLM post-processing tasks | Parameters.Model | String | - | The models supported for LLM post-processing tasks. Valid values: |
Parameters.LlmOutputLanguage | String | - | Specifies the output language for LLM tasks. Valid values:
Note: This parameter currently only takes effect for mind maps and chapter overviews. |
Sample code
#!/usr/bin/env python
#coding=utf-8
import os
import json
import datetime
from aliyunsdkcore.client import AcsClient
from aliyunsdkcore.request import CommonRequest
from aliyunsdkcore.auth.credentials import AccessKeyCredential
def create_common_request(domain, version, protocolType, method, uri):
request = CommonRequest()
request.set_accept_format('json')
request.set_domain(domain)
request.set_version(version)
request.set_protocol_type(protocolType)
request.set_method(method)
request.set_uri_pattern(uri)
request.add_header('Content-Type', 'application/json')
return request
def init_parameters():
body = dict()
body['AppKey'] = 'Enter the AppKey you created in the Tingwu console'
# Basic request parameters
input = dict()
input['SourceLanguage'] = 'cn'
input['TaskKey'] = 'task' + datetime.datetime.now().strftime('%Y%m%d%H%M%S')
input['FileUrl'] = 'Enter the URL of the audio file to be tested'
body['Input'] = input
# AI-related parameters. Set them as needed.
parameters = dict()
# Audio and video conversion
transcoding = dict()
# Convert the source audio or video file to MP3 for playback in a browser
# transcoding['TargetAudioFormat'] = 'mp3'
# transcoding['SpectrumEnabled'] = False
# parameters['Transcoding'] = transcoding
# Speech recognition control
transcription = dict()
# Speaker diarization: Optional
transcription['DiarizationEnabled'] = True
diarization = dict()
diarization['SpeakerCount'] = 2
transcription['Diarization'] = diarization
parameters['Transcription'] = transcription
# Text translation control: Optional
parameters['TranslationEnabled'] = True
translation = dict()
translation['TargetLanguages'] = ['en'] # Translate to English
parameters['Translation'] = translation
# Chapter overview: Optional. Includes titles and agenda summaries.
parameters['AutoChaptersEnabled'] = True
# Intelligent minutes: Optional. Includes to-do items and key information (keywords, key content, scenario recognition).
parameters['MeetingAssistanceEnabled'] = True
meetingAssistance = dict()
meetingAssistance['Types'] = ['Actions', 'KeyInformation']
parameters['MeetingAssistance'] = meetingAssistance
# Summarization control: Optional. Includes full-text summary, speaker summary, and Q&A summary (Q&A review).
parameters['SummarizationEnabled'] = True
summarization = dict()
summarization['Types'] = ['Paragraph', 'Conversational', 'QuestionsAnswering', 'MindMap']
parameters['Summarization'] = summarization
# PPT extraction and summarization: Optional
parameters['PptExtractionEnabled'] = True
# Spoken-to-written text conversion: Optional
parameters['TextPolishEnabled'] = True
# Global parameters for LLM post-processing tasks: Optional
parameters['Model'] = 'qwq'
parameters['LlmOutputLanguage'] = 'en'
body['Parameters'] = parameters
return body
body = init_parameters()
print(body)
# TODO Set your AccessKeyId and AccessKeySecret using environment variables
credentials = AccessKeyCredential(os.environ['ALIBABA_CLOUD_ACCESS_KEY_ID'], os.environ['ALIBABA_CLOUD_ACCESS_KEY_SECRET'])
client = AcsClient(region_id='cn-beijing', credential=credentials)
request = create_common_request('tingwu.cn-beijing.aliyuncs.com', '2023-09-30', 'https', 'PUT', '/openapi/tingwu/v2/tasks')
request.add_query_param('type', 'offline')
request.set_content(json.dumps(body).encode('utf-8'))
response = client.do_action_with_exception(request)
print("response: \n" + json.dumps(json.loads(response), indent=4, ensure_ascii=False))package com.alibaba.tingwu.client.demo.offlinetask;
import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import com.aliyuncs.CommonRequest;
import com.aliyuncs.CommonResponse;
import com.aliyuncs.DefaultAcsClient;
import com.aliyuncs.IAcsClient;
import com.aliyuncs.exceptions.ClientException;
import com.aliyuncs.http.FormatType;
import com.aliyuncs.http.MethodType;
import com.aliyuncs.http.ProtocolType;
import com.aliyuncs.profile.DefaultProfile;
import org.junit.Test;
/**
* @author tingwu2023
* @desc This demo shows how to call the OpenAPI to create an offline transcription task for an audio or video file.
*/
public class SubmitFiletransTaskTest {
@Test
public void summitTask() throws ClientException {
CommonRequest request = createCommonRequest("tingwu.cn-beijing.aliyuncs.com", "2023-09-30", ProtocolType.HTTPS, MethodType.PUT, "/openapi/tingwu/v2/tasks");
request.putQueryParameter("type", "offline");
JSONObject root = new JSONObject();
root.put("AppKey", "Enter the AppKey you created in the Tingwu console");
String path = "Enter the URL of the audio file to be tested";
JSONObject input = new JSONObject();
input.fluentPut("FileUrl", path)
.fluentPut("SourceLanguage", "cn")
.fluentPut("TaskKey", "task" + System.currentTimeMillis());
root.put("Input", input);
JSONObject parameters = initRequestParameters();
root.put("Parameters", parameters);
System.out.println(root.toJSONString());
request.setHttpContent(root.toJSONString().getBytes(), "utf-8", FormatType.JSON);
// TODO Set your AccessKeyId and AccessKeySecret using environment variables
DefaultProfile profile = DefaultProfile.getProfile("cn-beijing", System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
IAcsClient client = new DefaultAcsClient(profile);
CommonResponse response = client.getCommonResponse(request);
System.out.println(response.getData());
JSONObject body = JSONObject.parseObject(response.getData());
JSONObject data = (JSONObject) body.get("Data");
System.out.println("TaskId = " + data.getString("TaskId"));
}
public static JSONObject initRequestParameters() {
JSONObject parameters = new JSONObject();
// Audio and video conversion: Optional
JSONObject transcoding = new JSONObject();
transcoding.put("TargetAudioFormat", "mp3");
transcoding.put("SpectrumEnabled", false);
parameters.put("Transcoding", transcoding);
// Speech recognition
JSONObject transcription = new JSONObject();
transcription.put("DiarizationEnabled", true);
JSONObject speaker_count = new JSONObject();
speaker_count.put("SpeakerCount", 2);
transcription.put("Diarization", speaker_count);
parameters.put("Transcription", transcription);
// Translation: Optional
JSONObject translation = new JSONObject();
JSONArray langArry = new JSONArray();
langArry.add("en");
translation.put("TargetLanguages", langArry);
parameters.put("Translation", translation);
parameters.put("TranslationEnabled", true);
// Chapter overview: Optional
parameters.put("AutoChaptersEnabled", true);
// Intelligent minutes: Optional
parameters.put("MeetingAssistanceEnabled", true);
JSONObject meetingAssistance = new JSONObject();
JSONArray mTypes = new JSONArray().fluentAdd("Actions").fluentAdd("KeyInformation");
meetingAssistance.put("Types", mTypes);
parameters.put("MeetingAssistance", meetingAssistance);
// Summarization: Optional
parameters.put("SummarizationEnabled", true);
JSONObject summarization = new JSONObject();
JSONArray sTypes = new JSONArray().fluentAdd("Paragraph").fluentAdd("Conversational").fluentAdd("QuestionsAnswering").fluentAdd("MindMap");
summarization.put("Types", sTypes);
parameters.put("Summarization", summarization);
// PPT extraction: Optional
parameters.put("PptExtractionEnabled", true);
// Spoken-to-written text conversion: Optional
parameters.put("TextPolishEnabled", true);
// Global parameters for LLM post-processing tasks: Optional
parameters.put("Model", "qwq");
parameters.put("LlmOutputLanguage", "en");
return parameters;
}
public static CommonRequest createCommonRequest(String domain, String version, ProtocolType protocolType, MethodType method, String uri) {
// Create an API request and set parameters
CommonRequest request = new CommonRequest();
request.setSysDomain(domain);
request.setSysVersion(version);
request.setSysProtocol(protocolType);
request.setSysMethod(method);
request.setSysUriPattern(uri);
request.setHttpContentType(FormatType.JSON);
return request;
}
}package main
import (
"encoding/json"
"fmt"
"log"
"os"
"time"
"github.com/aliyun/alibaba-cloud-sdk-go/sdk"
"github.com/aliyun/alibaba-cloud-sdk-go/sdk/requests"
)
type TranscodeingParam struct {
TargetAudioFormat string `json:"TargetAudioFormat,omitempty"`
TargetVideoFormat string `json:"TargetVideoFormat,omitempty"`
VideoThumbnailEnabled bool `json:"VideoThumbnailEnabled,omitempty"`
SpectrumEnabled bool `json:"SpectrumEnabled,omitempty"`
}
type DiarizationParam struct {
SpeakerCount *int `json:"SpeakerCount,omitempty"`
}
type TranscriptionParam struct {
AudioEventDetectionEnabled bool `json:"AudioEventDetectionEnabled,omitempty"`
DiarizationEnabled bool `json:"DiarizationEnabled,omitempty"`
Diarization *DiarizationParam `json:"Diarization,omitempty"`
}
type TranslationParam struct {
TargetLanguages []string `json:"TargetLanguages,omitempty"`
}
type SummarizationParam struct {
Types []string `json:"Types,omitempty"`
}
type ExtraParamerters struct {
Transcoding *TranscodeingParam `json:"Transcoding,omitempty"`
Transcription *TranscriptionParam `json:"Transcription,omitempty"`
TranslationEnabled bool `json:"TranslationEnabled,omitempty"`
Translation *TranslationParam `json:"Translation,omitempty"`
AutoChaptersEnabled bool `json:"AutoChaptersEnabled,omitempty"`
MeetingAssistanceEnabled bool `json:"MeetingAssistanceEnabled,omitempty"`
SummarizationEnabled bool `json:"SummarizationEnabled,omitempty"`
Summarization *SummarizationParam `json:"Summarization,omitempty"`
TextPolishEnabled bool `json:"TextPolishEnabled,omitempty"`
Model string `json:"Model,omitempty"`
LlmOutputLanguage string `json:"LlmOutputLanguage,omitempty"`
}
type InputParam struct {
SourceLanguage string `json:"SourceLanguage"`
FileUrl string `json:"FileUrl,omitempty"`
TaskKey string `json:"TaskKey,omitempty"`
Format string `json:"Format,omitempty"`
SampleRate int `json:"SampleRate,omitempty"`
}
type TaskBodyParam struct {
Appkey string `json:"AppKey"`
Input InputParam `json:"Input"`
Paramerters *ExtraParamerters `json:"Parameters,omitempty"`
}
type GetTaskInfoResponse struct {
RequestId string `json:"RequestId"`
Code string `json:"Code"`
Message string `json:"Message"`
Data struct {
TaskId string `json:"TaskId"`
TaskKey string `json:"TaskKey"`
TaskStatus string `json:"TaskStatus"`
} `json:"Data"`
}
func init_request_param_offline() *ExtraParamerters {
param := new(ExtraParamerters)
param.Transcoding = new(TranscodeingParam)
transcription := new(TranscriptionParam)
transcription.Diarization = new(DiarizationParam)
transcription.Diarization.SpeakerCount = new(int)
*transcription.Diarization.SpeakerCount = 0
transcription.DiarizationEnabled = true
param.Transcription = transcription
param.TranslationEnabled = false
param.AutoChaptersEnabled = false
param.MeetingAssistanceEnabled = false
param.SummarizationEnabled = true
summarization := new(SummarizationParam)
summarization.Types = []string{
"Paragraph",
"Conversational",
"QuestionsAnswering",
"MindMap",
}
param.Summarization = summarization
param.TextPolishEnabled = false
param.Model = "qwq"
param.LlmOutputLanguage = "en"
return param
}
func test_submit_offline_task() {
akkey := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_ID")
aksecret := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET")
client, err := sdk.NewClientWithAccessKey("cn-beijing", akkey, aksecret)
if err != nil {
log.Default().Fatalln(err)
return
}
request := requests.NewCommonRequest()
request.Method = "PUT"
request.Domain = "tingwu.cn-beijing.aliyuncs.com"
request.Version = "2023-09-30"
request.SetContentType("application/json")
request.PathPattern = "/openapi/tingwu/v2/tasks"
request.QueryParams["type"] = "offline"
param := new(TaskBodyParam)
param.Appkey = "Enter the AppKey you created in the Tingwu console"
param.Input.SourceLanguage = "cn"
param.Input.FileUrl = "https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example.wav"
param.Input.TaskKey = "task_" + fmt.Sprint(time.Now().Unix())
param.Paramerters = init_request_param_offline()
b, _ := json.Marshal(param)
log.Default().Print("request body:\n", string(b))
request.SetContent(b)
request.SetScheme("https")
response, err := client.ProcessCommonRequest(request)
if err != nil {
log.Default().Fatalln(err)
return
}
log.Default().Print("response body:\n", string(response.GetHttpContentBytes()))
}
func main() {
test_submit_offline_task()
}
#include <cstdlib>
#include <string>
#include <alibabacloud/core/AlibabaCloud.h>
#include <alibabacloud/core/CommonRequest.h>
#include <alibabacloud/core/CommonClient.h>
#include <alibabacloud/core/CommonResponse.h>
#include "jsoncpp/json.h"
/**
* @author tingwu2023
* @desc This demo shows how to call the OpenAPI to create an offline transcription task for an audio or video file.
*/
int main( int argc, char** argv ) {
AlibabaCloud::InitializeSdk();
AlibabaCloud::ClientConfiguration configuration( "cn-beijing" );
// specify timeout when create client.
configuration.setConnectTimeout(1500);
configuration.setReadTimeout(4000);
// Please ensure that the environment variables ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET are set.
AlibabaCloud::Credentials credential( getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET") );
AlibabaCloud::CommonClient client( credential, configuration );
AlibabaCloud::CommonRequest request(AlibabaCloud::CommonRequest::RequestPattern::RoaPattern);
request.setHttpMethod(AlibabaCloud::HttpRequest::Method::Put);
request.setDomain("tingwu.cn-beijing.aliyuncs.com");
request.setVersion("2023-09-30");
request.setResourcePath("/openapi/tingwu/v2/tasks");
request.setHeaderParameter("Content-Type", "application/json");
request.setQueryParameter("type", "offline");
Json::Value root;
root["Appkey"] = "Enter the AppKey you created in the Tingwu console";
Json::Value input;
input["FileUrl"] = "Enter the URL of the audio file to be tested";
input["SourceLanguage"] = "cn";
input["TaskKey"] = "Enter a custom ID for this request";
root["Input"] = input;
Json::Value parameters;
// Audio and video file conversion. Generally, you do not need to set this parameter.
// Json::Value transcoding;
// transcoding["TargetAudioFormat"] = "mp3";
// parameters["Transcoding"] = transcoding;
// Speech transcription: The following enables speaker diarization. You do not need to set this if it is not required.
Json::Value transcription;
transcription["DiarizationEnabled"] = true;
Json::Value speakerCount;
speakerCount["SpeakerCount"] = 2;
transcription["Diarization"] = speakerCount;
parameters["Transcription"] = transcription;
// Translation: Optional
Json::Value translation;
Json::Value langauges;
langauges.append("en");
translation["TargetLanguages"] = langauges;
parameters["Translation"] = translation;
parameters["TranslationEnabled"] = true;
// Chapter overview: Optional
parameters["AutoChaptersEnabled"] = true;
// Intelligent minutes: Optional. Includes to-do items and key information (keywords, key content, scenario recognition).
parameters["MeetingAssistanceEnabled"] = true;
Json::Value meetingAssistance;
Json::Value meetingAssistanceTypeList;
meetingAssistanceTypeList.append("Actions");
meetingAssistanceTypeList.append("KeyInformation");
meetingAssistance["Types"] = meetingAssistanceTypeList;
parameters["MeetingAssistanceTypeList"] = meetingAssistanceTypeList;
// Summarization: Optional. The following settings enable all three summary types. You can add or remove them as needed.
parameters["SummarizationEnabled"] = true;
Json::Value summarization;
Json::Value summarizationTypeList;
summarizationTypeList.append("Paragraph");
summarizationTypeList.append("Conversational");
summarizationTypeList.append("QuestionsAnswering");
summarizationTypeList.append("MindMap");
summarization["Types"] = summarizationTypeList;
parameters["Summarization"] = summarization;
// PPT extraction: Optional. We recommend enabling this only for video files that contain PPT-related video frames.
//parameters["PptExtractionEnabled"] = true;
// Spoken-to-written text conversion: Optional
parameters["TextPolishEnabled"] = true;
# Global parameters for LLM post-processing tasks: Optional
parameters["Model"] = "qwq";
parameters["LlmOutputLanguage"] = "en";
root["Parameters"] = parameters;
Json::FastWriter writer;
std::string body = writer.write(root);
printf("input json: [%s]\n", body.c_str());
request.setContent(body.c_str(), body.size());
auto response = client.commonResponse(request);
if (response.isSuccess()) {
printf("request success.\n");
printf("result: %s\n", response.result().payload().c_str());
} else {
printf("error: %s\n", response.error().errorMessage().c_str());
printf("request id: %s\n", response.error().requestId().c_str());
}
AlibabaCloud::ShutdownSdk();
return 0;
} Sample output
When the task is running:
{
"Code":"0",
"Data":{
"TaskId":"e8adc0b3vc4b45d898fcadb*********",
"TaskKey":"task16988********",
"TaskStatus":"ONGOING"
},
"Message":"success",
"RequestId":"2001dc2a-9b46-4a2f-9822-b140********"
}Protocol parsing
The fields are defined as follows.
Parameter Name | Type | Description |
TaskId | string | The ID generated when the task was created. Use it to query the task status and result, and for troubleshooting. |
TaskStatus | string | The task status. |
TaskKey | string | The TaskKey you set when creating the task. |
RequestId | string | The ID of the request. Use it for troubleshooting. |
After you submit an offline audio or video transcription task, the task enters the running state.
Query task status and result
When you query a task, you must use the TaskId that was returned when you submitted the task. You can then initiate a query request and check the returned status to determine whether the task is complete.
Request parameters
Parameter | Type | Required | Description |
TaskId | string | Yes | The TaskId returned when you submitted the task. |
Sample code
#!/usr/bin/env python
#coding=utf-8
import os
import json
from aliyunsdkcore.client import AcsClient
from aliyunsdkcore.request import CommonRequest
from aliyunsdkcore.auth.credentials import AccessKeyCredential
def create_common_request(domain, version, protocolType, method, uri):
request = CommonRequest()
request.set_accept_format('json')
request.set_domain(domain)
request.set_version(version)
request.set_protocol_type(protocolType)
request.set_method(method)
request.set_uri_pattern(uri)
request.add_header('Content-Type', 'application/json')
return request
# TODO Set your AccessKeyId and AccessKeySecret using environment variables
credentials = AccessKeyCredential(os.environ['ALIBABA_CLOUD_ACCESS_KEY_ID'], os.environ['ALIBABA_CLOUD_ACCESS_KEY_SECRET'])
client = AcsClient(region_id='cn-beijing', credential=credentials)
uri = '/openapi/tingwu/v2/tasks' + '/' + 'Enter the TaskId returned when you submitted the task'
request = create_common_request('tingwu.cn-beijing.aliyuncs.com', '2023-09-30', 'https', 'GET', uri)
response = client.do_action_with_exception(request)
print("response: \n" + json.dumps(json.loads(response), indent=4, ensure_ascii=False))package com.alibaba.tingwu.client.demo.common;
import com.aliyuncs.CommonRequest;
import com.aliyuncs.CommonResponse;
import com.aliyuncs.DefaultAcsClient;
import com.aliyuncs.IAcsClient;
import com.aliyuncs.exceptions.ClientException;
import com.aliyuncs.http.MethodType;
import com.aliyuncs.http.ProtocolType;
import com.aliyuncs.profile.DefaultProfile;
import org.junit.Test;
/**
* @author tingwu2023
* @desc This demo shows how to call the OpenAPI to query the task status and result by TaskId.
*/
public class GetTaskInfoTest {
@Test
public void getTaskInfo() throws ClientException {
String taskId = "Enter the TaskId of the created task (offline transcription or real-time meeting)";
String queryUrl = String.format("/openapi/tingwu/v2/tasks" + "/%s", taskId);
CommonRequest request = createCommonRequest("tingwu.cn-beijing.aliyuncs.com", "2023-09-30", ProtocolType.HTTPS, MethodType.GET, queryUrl);
// TODO Set your AccessKeyId and AccessKeySecret using environment variables
DefaultProfile profile = DefaultProfile.getProfile("cn-beijing", System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
IAcsClient client = new DefaultAcsClient(profile);
CommonResponse response = client.getCommonResponse(request);
System.out.println(response.getData());
}
public static CommonRequest createCommonRequest(String domain, String version, ProtocolType protocolType, MethodType method, String uri) {
// Create an API request and set parameters
CommonRequest request = new CommonRequest();
request.setSysDomain(domain);
request.setSysVersion(version);
request.setSysProtocol(protocolType);
request.setSysMethod(method);
request.setSysUriPattern(uri);
request.setHttpContentType(FormatType.JSON);
return request;
}
}package main
import (
"encoding/json"
"log"
"os"
"github.com/aliyun/alibaba-cloud-sdk-go/sdk"
"github.com/aliyun/alibaba-cloud-sdk-go/sdk/requests"
)
type GetTaskInfoResponse struct {
RequestId string `json:"RequestId"`
Code string `json:"Code"`
Message string `json:"Message"`
Data struct {
TaskId string `json:"TaskId"`
TaskKey string `json:"TaskKey"`
TaskStatus string `json:"TaskStatus"`
} `json:"Data"`
}
func get_task_info(taskid string, akkey string, aksecret string) (*GetTaskInfoResponse, string, error) {
client, err := sdk.NewClientWithAccessKey("cn-beijing", akkey, aksecret)
if err != nil {
return nil, "", err
}
request := requests.NewCommonRequest()
request.Method = "GET"
request.Domain = "tingwu.cn-beijing.aliyuncs.com"
request.Version = "2023-09-30"
request.PathPattern = "/openapi/tingwu/v2/tasks/" + taskid
request.SetScheme("https")
log.Default().Print("request task:", taskid)
response, err := client.ProcessCommonRequest(request)
if err != nil {
return nil, "", err
}
log.Default().Print("response body:\n", string(response.GetHttpContentBytes()))
resp := new(GetTaskInfoResponse)
err = json.Unmarshal(response.GetHttpContentBytes(), resp)
if err != nil {
return nil, response.GetHttpContentString(), err
}
return resp, response.GetHttpContentString(), nil
}
func main() {
akkey := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_ID")
aksecret := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET")
_, raw, _ := get_task_info("Enter the TaskId of the created task (offline transcription or real-time meeting)", akkey, aksecret)
log.Default().Println("response :", raw)
}#include <cstdlib>
#include <iostream>
#include <string>
#include <alibabacloud/core/AlibabaCloud.h>
#include <alibabacloud/core/CommonRequest.h>
#include <alibabacloud/core/CommonClient.h>
#include <alibabacloud/core/CommonResponse.h>
/**
* @author tingwu2023
* @desc This demo shows how to call the OpenAPI to query the task status and result by TaskId.
*/
int main( int argc, char** argv ) {
std::string taskId = "Enter the TaskId of the created task (offline transcription or real-time meeting)";
AlibabaCloud::InitializeSdk();
AlibabaCloud::ClientConfiguration configuration( "cn-beijing" );
// specify timeout when create client.
configuration.setConnectTimeout(1500);
configuration.setReadTimeout(4000);
// Please ensure that the environment variables ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET are set.
AlibabaCloud::Credentials credential( getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET") );
AlibabaCloud::CommonClient client( credential, configuration );
AlibabaCloud::CommonRequest request(AlibabaCloud::CommonRequest::RequestPattern::RoaPattern);
request.setHttpMethod(AlibabaCloud::HttpRequest::Method::Get);
request.setDomain("tingwu.cn-beijing.aliyuncs.com");
request.setVersion("2023-09-30");
request.setHeaderParameter("Content-Type", "application/json");
request.setResourcePath("/openapi/tingwu/v2/tasks/" + taskId);
auto response = client.commonResponse(request);
if (response.isSuccess()) {
printf("request success.\n");
printf("result: %s\n", response.result().payload().c_str());
} else {
printf("error: %s\n", response.error().errorMessage().c_str());
printf("request id: %s\n", response.error().requestId().c_str());
}
AlibabaCloud::ShutdownSdk();
return 0;
} Sample output
When the task is running:
{
"Code":"0",
"Data":{
"TaskId":"e8adc0b3bc4b42d898fcadb0********",
"TaskStatus":"ONGOING"
},
"Message":"success",
"RequestId":"5fa32fc6-441f-4dd1-bb86-c030********"
}When the task is running and partial results are available:
{
"Code":"0",
"Data":{
"TaskId":"e8adc0b3bc4b42d898fcadb0********",
"TaskStatus":"ONGOING",
"OutputMp3Path":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_20231101141801.mp3?ExLTAI****************AccessKeyId=LTAI****************&Signature=********JBMijH7wLq0xX6aivHc%3D",
"Result":{
"Transcription":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu_data/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_Transcription_20231101141926.json?Expires=1698906034&OSSAcceLTAI****************&Signature=********NJlqSEWJxfkMwjwsHCA%3D"
}
},
"Message":"success",
"RequestId":"1b20e0d9-c55c-4cc3-85af-80b4********"
}When the task is completed:
{
"Code":"0",
"Data":{
"TaskId":"e8adc0b3bc4b42d898fcadb0********",
"TaskStatus":"COMPLETED",
"OutputMp3Path":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_20231101141801.mp3?ExLTAI****************AccessKeyId=LTAI****************&Signature=********JBMijH7wLq0xX6aivHc%3D",
"Result":{
"AutoChapters":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu_data/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_AutoChapters_20231101141955.json?Expires=1698906034&OSSAcceLTAI****************&Signature=********Ax9FvifYAO8dj4qzWg%3D",
"Transcription":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu_data/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_Transcription_20231101141926.json?Expires=1698906034&OSSAccessKeyId=LTAI****************&Signature=********NJlqSEWJxfkMwjwsHCA%3D"
}
},
"Message":"success",
"RequestId":"1b20e0d9-c55c-4cc3-85af-80b4********"
}When the task has failed:
{
"Code":"0",
"Data":{
"TaskId":"b76389677b1441fa82165cb1********",
"TaskStatus":"FAILED",
"ErrorCode":"TSC.AudioFileLink",
"ErrorMessage":"Audio file link invalid."
},
"Message":"success",
"RequestId":"d181d898-b627-4040-b7c9-9563********"
}Callback notifications for task status and results
If you want to receive task status and results through callbacks, you must configure the callback type and address in the console. When you create an offline transcription task, you must set the `AppKey` parameter to the corresponding project AppKey and set the `Input.ProgressiveCallbacksEnabled` parameter to `true`.
When you use the HTTP callback method, a maximum request timeout period is set. This means that if the service does not receive a response from your server within 5 seconds after sending the request, the service stops waiting for a response. The system then determines that the callback request failed and retries the request later.
We recommend that you review and adjust your system to ensure that it can respond to callback requests within 5 seconds. This may include, but is not limited to, the following actions:
Optimize your processing logic for efficient execution.
Handle unnecessarily long operations asynchronously instead of waiting for them in the main callback response path.
Implement error handling and retry mechanisms to cope with timeouts or failures.
Callback retry mechanism:
If a callback message fails to be sent, the Tingwu server resends the callback message after 5 seconds. If the second attempt also fails, the message is resent every 5 minutes thereafter, up to a maximum of 3 retries.
Sample output
When a single AI model sub-task is completed:
{
"Code":"0",
"Data":{
"TaskId":"e8adc0b3bc4b42d898fcadb0********",
"TaskStatus":"ONGOING",
"TaskKey":"TingwuDemo",
"Result":{
"Transcription":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu_data/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_Transcription_20231101141926.json?Expires=1698906034&OSSAcceLTAI****************&Signature=********NJlqSEWJxfkMwjwsHCA%3D"
}
},
"Message":"success",
"RequestId":"5cf2b587ac6e41e6a1f06ca1********"
}When the task is completed:
{
"Code":"0",
"Data":{
"TaskId":"e8adc0b3bc4b42d898fcadb0********",
"TaskStatus":"COMPLETED",
"TaskKey":"TingwuDemo",
"Result":{
"AutoChapters":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu_data/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_AutoChapters_20231101141955.json?Expires=1698906034&OSSAcceLTAI****************&Signature=********Ax9FvifYAO8dj4qzWg%3D",
"Transcription":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu_data/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_Transcription_20231101141926.json?Expires=1698906034&OSSAccessKeyId=LTAI****************&Signature=********NJlqSEWJxfkMwjwsHCA%3D"
}
},
"Message":"success",
"RequestId":"9c60b07f152445349eaa6161********"
}When the task has failed:
{
"Code":"0",
"Data":{
"TaskId":"e8adc0b3bc4b42d898fcadb0********",
"TaskStatus":"FAILED",
"TaskKey":"TingwuDemo",
"ErrorCode":"TSC.AudioFileLink",
"ErrorMessage":"Audio file link invalid."
},
"Message":"success",
"RequestId":"5e1c0babe36844e49f1b7bee********"
}Rerun a task to refresh the result
To refresh a completed task, for example, to regenerate a specific algorithm result or add a new one, you can initiate the task processing request again. Use the TaskId that was returned when you submitted the task and the new control parameters as input, and then wait for the result.
For example, if you did not enable the chapter overview feature when you first created the task, you can add the new parameter to continue the task. This way, you do not need to resubmit the audio or video file and start from scratch.
You can only rerun a task that is already completed (COMPLETED). You cannot rerun a task that is running or has failed.
Request parameters
Feature name | Parameter | Type | Default | Description |
Basic request information (Input) | TaskId | string | - | Required. The TaskId of the task you want to rerun. |
Translation (Translation) | TranslationEnabled | boolean | false | Specifies whether to enable the translation feature. |
Translation.TargetLanguages | list[string] | [en] |
| |
Chapter overview | AutoChaptersEnabled | boolean | false | The chapter overview feature, which includes agenda titles and agenda summaries. |
Key point extraction | MeetingAssistanceEnabled | boolean | false | Keywords, action items, to-do items, and scenario recognition. |
Summarization | SummarizationEnabled | boolean | false | Specifies whether to enable the summarization feature. |
Summarization.Types | list[string] | [] | You can set one or more types.
| |
PPT extraction and summarization | PptExtractionEnabled | boolean | false | Specifies whether to enable the PPT feature, which includes PPT extraction and PPT summarization. |
Spoken-to-written text conversion | TextPolishEnabled | boolean | false | Specifies whether to enable the spoken-to-written text conversion feature. |
Service quality inspection | ServiceInspectionEnabled | boolean | false | Specifies whether to enable the service quality inspection feature. |
ServiceInspection | object | - | Refer to the Service Quality Inspection Parameter Object to perform quality inspection on conversations during service processes, support customizing multiple quality inspection dimensions, and help customers improve service levels. | |
Custom Prompt | CustomPromptEnabled | boolean | false | Specifies whether to enable the custom prompt feature. |
CustomPrompt | object | - | Allows you to define custom prompts to guide the Large Language Model (LLM) to complete various tasks that you specify. For more information, see CustomPrompt parameter object. |
Sample code
#!/usr/bin/env python
#coding=utf-8
import os
import json
import datetime
from aliyunsdkcore.client import AcsClient
from aliyunsdkcore.request import CommonRequest
from aliyunsdkcore.auth.credentials import AccessKeyCredential
def create_common_request(domain, version, protocolType, method, uri):
request = CommonRequest()
request.set_accept_format('json')
request.set_domain(domain)
request.set_version(version)
request.set_protocol_type(protocolType)
request.set_method(method)
request.set_uri_pattern(uri)
request.add_header('Content-Type', 'application/json')
return request
def init_parameters():
root = dict()
root['AppKey'] = 'Enter the AppKey you created in the Tingwu console'
# Basic request parameters
input = dict()
input['TaskId'] = 'Enter the TaskId of the completed task'
root['Input'] = input
# AI-related parameters. Set them as needed.
parameters = dict()
# Audio and video conversion: Rerunning is not supported.
# Speech recognition: Rerunning the speech recognition algorithm is not supported.
# Translation: Optional
parameters['TranslationEnabled'] = True
translation = dict()
translation['TargetLanguages'] = ['en'] # Translate to English
parameters['Translation'] = translation
# Chapter overview: Optional
parameters['AutoChaptersEnabled'] = True
# Intelligent minutes: Optional
parameters['MeetingAssistanceEnabled'] = True
# Summarization: Optional
parameters['SummarizationEnabled'] = True
summarization = dict()
summarization['Types'] = ['Paragraph', 'Conversational', 'QuestionsAnswering', 'MindMap']
parameters['Summarization'] = summarization
root['Parameters'] = parameters
# PPT extraction: Optional (Rerunning is not supported for real-time meetings or offline transcription tasks where the source file is an audio format).
parameters['PptExtractionEnabled'] = True
# Spoken-to-written text conversion: Optional
parameters['TextPolishEnabled'] = True
root['Parameters'] = parameters
return root
body = init_parameters()
print(body)
# TODO Set your AccessKeyId and AccessKeySecret using environment variables
credentials = AccessKeyCredential(os.environ['ALIBABA_CLOUD_ACCESS_KEY_ID'], os.environ['ALIBABA_CLOUD_ACCESS_KEY_SECRET'])
client = AcsClient(region_id='cn-beijing', credential=credentials)
request = create_common_request('tingwu.cn-beijing.aliyuncs.com', '2023-09-30', 'https', 'PUT', '/openapi/tingwu/v2/tasks')
request.add_query_param('type', 'offline')
request.set_content(json.dumps(body).encode('utf-8'))
response = client.do_action_with_exception(request)
print("response: \n" + json.dumps(json.loads(response), indent=4, ensure_ascii=False))package com.alibaba.tingwu.client.demo.common;
import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import com.aliyuncs.CommonRequest;
import com.aliyuncs.CommonResponse;
import com.aliyuncs.DefaultAcsClient;
import com.aliyuncs.IAcsClient;
import com.aliyuncs.exceptions.ClientException;
import com.aliyuncs.http.FormatType;
import com.aliyuncs.http.MethodType;
import com.aliyuncs.http.ProtocolType;
import com.aliyuncs.profile.DefaultProfile;
import org.junit.Test;
/**
* @author tingwu2023
* @desc This demo shows how to call the OpenAPI to rerun a completed task by TaskId.
*/
public class RerunTaskTest {
@Test
public void testRerunTask() throws ClientException {
CommonRequest request = createCommonRequest("tingwu.cn-beijing.aliyuncs.com", "2023-09-30", ProtocolType.HTTPS, MethodType.PUT, "/openapi/tingwu/v2/tasks");
request.putQueryParameter("type", "offline");
JSONObject root = new JSONObject();
JSONObject input = new JSONObject();
input.put("TaskId", "Enter the TaskId of the completed task");
root.put("Input", input);
root.put("AppKey", "Enter the AppKey you created in the Tingwu console");
JSONObject parameters = initRequestParametersForReRun();
root.put("Parameters", parameters);
System.out.println(root.toJSONString());
request.setHttpContent(root.toJSONString().getBytes(), "utf-8", FormatType.JSON);
// TODO Set your AccessKeyId and AccessKeySecret using environment variables
DefaultProfile profile = DefaultProfile.getProfile("cn-beijing", System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
IAcsClient client = new DefaultAcsClient(profile);
CommonResponse response = client.getCommonResponse(request);
System.out.println(response.getData());
}
public static JSONObject initRequestParametersForReRun() {
JSONObject parameters = new JSONObject();
// Audio and video conversion: Rerunning is not supported.
// Speech recognition: Rerunning the speech recognition algorithm is not supported.
// Translation: Optional
JSONObject translation = new JSONObject();
JSONArray langArry = new JSONArray();
langArry.add("en");
translation.put("TargetLanguages", langArry);
parameters.put("Translation", translation);
parameters.put("TranslationEnabled", true);
// Chapter overview: Optional
parameters.put("AutoChaptersEnabled", true);
// Intelligent minutes: Optional
parameters.put("MeetingAssistanceEnabled", true);
// Summarization: Optional
parameters.put("SummarizationEnabled", true);
JSONObject summarization = new JSONObject();
JSONArray types = new JSONArray().fluentAdd("Paragraph").fluentAdd("Conversational").fluentAdd("QuestionsAnswering").fluentAdd("MindMap");
summarization.put("Types", types);
parameters.put("Summarization", summarization);
// PPT extraction: Optional (Rerunning is not supported for real-time meetings or offline transcription tasks where the source file is an audio format).
//parameters.put("PptExtractionEnabled", false);
// Spoken-to-written text conversion: Optional
parameters.put("TextPolishEnabled", true);
return parameters;
}
public static CommonRequest createCommonRequest(String domain, String version, ProtocolType protocolType, MethodType method, String uri) {
CommonRequest request = new CommonRequest();
request.setSysDomain(domain);
request.setSysVersion(version);
request.setSysProtocol(protocolType);
request.setSysMethod(method);
request.setSysUriPattern(uri);
request.setHttpContentType(FormatType.JSON);
return request;
}
}package main
import (
"encoding/json"
"fmt"
"log"
"os"
"time"
"github.com/aliyun/alibaba-cloud-sdk-go/sdk"
"github.com/aliyun/alibaba-cloud-sdk-go/sdk/requests"
)
type TranscodeingParam struct {
TargetAudioFormat string `json:"TargetAudioFormat,omitempty"`
TargetVideoFormat string `json:"TargetVideoFormat,omitempty"`
VideoThumbnailEnabled bool `json:"VideoThumbnailEnabled,omitempty"`
SpectrumEnabled bool `json:"SpectrumEnabled,omitempty"`
}
type DiarizationParam struct {
SpeakerCount int `json:"SpeakerCount,omitempty"`
}
type TranscriptionParam struct {
AudioEventDetectionEnabled bool `json:"AudioEventDetectionEnabled,omitempty"`
DiarizationEnabled bool `json:"DiarizationEnabled,omitempty"`
Diarization *DiarizationParam `json:"Diarization,omitempty"`
}
type TranslationParam struct {
TargetLanguages []string `json:"TargetLanguages,omitempty"`
}
type SummarizationParam struct {
Types []string `json:"Types,omitempty"`
}
type ExtraParamerters struct {
Transcoding *TranscodeingParam `json:"Transcoding,omitempty"`
Transcription *TranscriptionParam `json:"Transcription,omitempty"`
TranslationEnabled bool `json:"TranslationEnabled,omitempty"`
Translation *TranslationParam `json:"Translation,omitempty"`
AutoChaptersEnabled bool `json:"AutoChaptersEnabled,omitempty"`
MeetingAssistanceEnabled bool `json:"MeetingAssistanceEnabled,omitempty"`
SummarizationEnabled bool `json:"SummarizationEnabled,omitempty"`
Summarization *SummarizationParam `json:"Summarization,omitempty"`
TextPolishEnabled bool `json:"TextPolishEnabled,omitempty"`
}
type InputParam struct {
SourceLanguage string `json:"SourceLanguage"`
FileUrl string `json:"FileUrl,omitempty"`
TaskKey string `json:"TaskKey,omitempty"`
TaskId string `json:"TaskId,omitempty"`
Format string `json:"Format,omitempty"`
SampleRate int `json:"SampleRate,omitempty"`
}
type TaskBodyParam struct {
Appkey string `json:"AppKey"`
Input InputParam `json:"Input"`
Paramerters *ExtraParamerters `json:"Parameters,omitempty"`
}
type GetTaskInfoResponse struct {
RequestId string `json:"RequestId"`
Code string `json:"Code"`
Message string `json:"Message"`
Data struct {
TaskId string `json:"TaskId"`
TaskKey string `json:"TaskKey"`
TaskStatus string `json:"TaskStatus"`
} `json:"Data"`
}
func init_request_param_offline() *ExtraParamerters {
param := new(ExtraParamerters)
param.Transcoding = new(TranscodeingParam)
transcription := new(TranscriptionParam)
transcription.Diarization = new(DiarizationParam)
transcription.Diarization.SpeakerCount = 2
transcription.DiarizationEnabled = true
param.Transcription = transcription
// Set translation language
translation := new(TranslationParam)
translation.TargetLanguages = []string{"en"}
param.Translation = translation
param.TranslationEnabled = true
param.AutoChaptersEnabled = false
param.MeetingAssistanceEnabled = false
param.SummarizationEnabled = true
summarization := new(SummarizationParam)
summarization.Types = []string{"Paragraph", "Conversational", "QuestionsAnswering", "MindMap"}
param.Summarization = summarization
param.TextPolishEnabled = true
return param
}
func test_rerun_task() {
akkey := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_ID")
aksecret := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET")
client, err := sdk.NewClientWithAccessKey("cn-beijing", akkey, aksecret)
if err != nil {
log.Default().Fatalln(err)
return
}
request := requests.NewCommonRequest()
request.Method = "PUT"
request.Domain = "tingwu.cn-beijing.aliyuncs.com"
request.Version = "2023-09-30"
request.SetContentType("application/json")
request.PathPattern = "/openapi/tingwu/v2/tasks"
request.QueryParams["type"] = "offline"
param := new(TaskBodyParam)
param.Appkey = "Enter the AppKey you created in the Tingwu console"
param.Input.SourceLanguage = "cn"
param.Input.TaskId = "Enter the TaskId of the completed task"
param.Input.FileUrl = "https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example.wav"
param.Input.TaskKey = "task_" + fmt.Sprint(time.Now().Unix())
b, _ := json.Marshal(param)
log.Default().Print("request body:\n", string(b))
request.SetContent(b)
request.SetScheme("https")
response, err := client.ProcessCommonRequest(request)
if err != nil {
log.Default().Fatalln(err)
return
}
log.Default().Print("response body:\n", string(response.GetHttpContentBytes()))
}
func main() {
test_rerun_task()
}Sample output
After you rerun the task, you can continue to query the task status and result using the same TaskId. The input request is almost identical to the request that you used to submit the task. Pass a JSON-formatted string to the body of the request object. The following code shows a sample JSON message:
{
"Input":{
"TaskId":"be602978409b47ecb1788e36********"
},
"Parameters":{
"Translation":{
"TargetLanguages":[
"en"
]
},
"TranslationEnabled":true
}
}Protocol parsing
The fields are defined as follows.
Parameter name | Type | Description |
TaskId | string | The TaskId generated when the task was created. |
TaskStatus | string | A task can have one of the following statuses:
|
OutputMp3Path | string | If you set the request parameter to convert the source audio or video to MP3 format, this field returns the HTTP download link for the generated MP3 file. |
Result | list[] | A collection of multiple algorithm results:
Note: The returned values are download links. You can refer to the corresponding model's documentation to process the results. |
ErrorCode | string | The cause of the error. |
ErrorMessage | string | The detailed error message. |
FAQ
The generated HTTP link is valid. Why do I receive an 'AudioFileLink' error?
Results are often returned in minutes, but this is not guaranteed. Offline transcription is an asynchronous process, and tasks may be queued. Therefore, the time limit for offline tasks is 3 hours. If the validity period of the audio HTTP link that you set is less than 3 hours, the link may expire before the task is processed. This makes the file impossible to download.
Do the HTTP links in the result have an expiration time?
Yes, they do. The HTTP links within the `Result` parameter are valid for 30 days. After 30 days, you cannot access the task result through these links. Other HTTP links within the `Result` content also have a 30-day validity period. For example, the `Result` of PPT extraction includes an HTTP link to the PPT thumbnail image, which also expires after 30 days.
If an HTTP link in the result expires, you can call the task query API again to obtain a new task result with a 30-day validity period.
In which scenarios should I rerun a task?
If you know which algorithms to process and which results to obtain for each offline transcription task that you submit, you can set the corresponding feature parameters once at submission. In this case, you do not need to rerun the task.
If you are developing a secondary encapsulation based on the Tingwu API, such as your own Software as a Service (SaaS) application that allows users to interactively display different algorithm results, you can use this API to rerun tasks and speed up execution.
What are the differences or benefits of rerunning a task compared to resubmitting an offline transcription task for the same audio or video file?
You can achieve the same functionality by resubmitting an offline transcription task with new parameters. However, if you rerun the task, the process does not start from scratch. It reuses some of the algorithm results from the previous run. This shortens the running time and avoids redundant calling costs.
How do I choose the language?
The following table provides examples of common language scenarios.
Language type
Corresponding parameter
Scenarios
Example
Single language
(Known language)
Parameter name: Input.SourceLanguage
Parameter value (select one): Chinese (cn), English (en), Cantonese (yue), Japanese (ja), Korean (ko)
Single-language speech recognition model.
Chinese and English support resolutions of 8K and 16K.
Japanese, Cantonese, and Korean support only 16 kHz.
Example 1:
If the language in the audio is known to be Chinese
Input.SourceLanguage="cn"
Recognition result: Thank you for using Tongyi Tingwu.
Example 2:
If the language in the audio is known to be English
Input.SourceLanguage="en"
Recognition result: Thank you for using Tongyi Tingwu.
Single language
(Unknown language)
Parameter name: Input.SourceLanguage
Parameter value: auto
Automatic single-language recognition.
Only 16 kHz is supported.
If you do not know the language in the audio, you can set this parameter to `auto` for automatic language detection. The system detects the language and then automatically selects the corresponding single-language model for speech recognition. This feature is available only for offline transcription tasks.
The `auto` mode can detect the following languages: Chinese, English, Japanese, Cantonese, and Korean.
Example 1:
If the language in the audio is unknown, the system must first detect and determine the language before recognition.
Input.SourceLanguage="auto"
Recognition result: Thank you for using Tongyi Tingwu.
Mixed languages
Parameter name 1: Input.SourceLanguage
Parameter value: multilingual
Parameter name 2: Input.LanguageHints
Parameter value (select one or more): Chinese (cn), English (en), Cantonese (yue), Japanese (ja), Korean (ko), German (de), French (fr), Russian (ru)
Multilingual speech recognition.
Only 16 kHz is supported.
Supports simultaneous recognition of Chinese, English, Japanese, Cantonese, Korean, German, French, and Russian.
Supports limiting the scope of languages.
Example 1:
If the audio contains multiple languages, directly recognize the text in the corresponding languages.
Input.SourceLanguage="multilingual"
Recognition result: Hello, everyone. Thank you for using Tongyi Tingwu.
Example 2:
If the audio contains multiple languages, but the scope of languages is known (such as only Chinese, English, and Cantonese), directly recognize the text in the corresponding languages while avoiding misidentification of languages irrelevant to the scenario, such as Japanese, French, or Russian.
Input.SourceLanguage="multilingual"
Input.LanguageHints=['cn', 'en', 'yue']