Offline transcription for audio and video files

更新时间:
复制 MD 格式

This topic describes the flow for ingesting audio and video recordings.

Audio and video file transcription is a service that performs offline processing on pre-recorded audio or video files. This processing includes features such as speech recognition, translation, key point extraction, summary generation, and PowerPoint (PPT) extraction and summarization.

Offline transcription is designed for non-real-time scenarios. You can submit a file by providing a URL that is accessible over HTTP or HTTPS. The service does not support submitting local files or files that use other protocols, such as FTP.

Usage notes

  • The service supports single-track or dual-track audio files in MP3, WAV, M4A, WMA, AAC, OGG, AMR, FLAC, and AIFF formats, and video files in MP4, WMV, M4V, FLV, RMVB, DAT, MOV, MKV, WEBM, AVI, MPEG, 3GP, and OGG formats.

  • The file size cannot exceed 6 GB.

  • The audio duration cannot exceed 6 hours.

  • The audio sampling rate can be 8 kHz, 16 kHz, 24 kHz, or 48 kHz.

  • The audio or video file for transcription must be accessible and downloadable from a public URL.

  • The access permissions for the submitted audio or video file URL must be set to public. The URL must use a domain name instead of an IP address. The URL cannot contain spaces, and we do not recommend using Chinese characters in the URL.

    • We recommend using Alibaba Cloud OSS. If file access permissions in OSS are set to public, you can obtain a file access URL. For more information, see Using Signed URLs to Download or Preview Files. If file access permissions in OSS are set to private, you can generate a time-limited access URL using the SDK. For more information, see Using Signed URLs to Download or Preview Files.

    • Alternatively, you can store the recording files on your own file server and provide a download link. Ensure that the `Content-Length` value in the HTTP response header matches the actual size of the data in the body. Otherwise, the download fails.

    • Offline tasks may be queued. To ensure that tasks are completed within 3 hours, if you use a time-limited access URL, you must set its validity period to at least 3 hours. This prevents the URL from expiring before the audio or video file can be downloaded and processed.

  • The service supports two invocation methods: polling and callback. You can choose either method.

  • The service supports speech recognition for multiple languages, including Chinese, English, Cantonese, Mixed Chinese and English (for long, mixed-language segments), Japanese, and Korean. For more information, see Audio and video file service parameter table.

  • The Queries Per Second (QPS) limits for the APIs are as follows:

Important
  • After you submit an offline transcription task, the task is typically completed within 3 hours. The recognized text and Large Language Model (LLM) results, if any, are then returned. This timeframe does not apply to large-scale data uploads, such as uploading more than 500 hours of recordings within 30 minutes.

  • To process large-scale offline files, contact our pre-sales experts for further discussion.

  • If your audio or video files are stored in the Alibaba Cloud OSS Beijing region, we strongly recommend that you generate an internally signed URL before you submit jobs. This practice reduces public network costs and speeds up job processing. For more information, see ECS Instances Access OSS Resources Through OSS Internal Endpoints.

Warning

If you exceed the QPS limit, the following error may occur: Throttling.User : Request was denied due to user flow control.. We recommend that you set a longer polling interval for the query API to avoid being rate-limited.

Interaction flow

The interaction flow between the client and the server is shown in the following figure:

image

Integration steps

The client submits a request to transcribe a recording file.

If the request is successful, the server returns a TaskId. You can use this TaskId to query the transcription result later.

If an error occurs, the response message includes error information and the RequestId. You can use this information to troubleshoot the request parameters.

You can query the processing result using the TaskId that is returned after you submit the task.

If you use this method to poll for results, do not set the polling frequency too high, which helps avoid rate limiting. For example, you can query the result at a frequency of every 1 minute or every 5 minutes.

You can use callback notifications to retrieve the results. Unlike active polling, this method lets you wait for the server to proactively send notifications about the task status after you submit the task.

We currently support HTTP callbacks to send notifications about the task processing status.

After a task is completed, you can add new parameters to trigger new algorithm processing and refresh the task result.

Prerequisites

Set AccessKey environment variables

Replace `YOUR_ACCESS_KEY_ID` and `YOUR_ACCESS_KEY_SECRET` in the following command with your AccessKey ID and AccessKey secret.

export ALIBABA_CLOUD_ACCESS_KEY_ID=YOUR_ACCESS_KEY_ID &&
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=YOUR_ACCESS_KEY_SECRET

Submit an offline audio or video transcription task

To create an audio or video offline transcription job, see API details. The common parameters are described in the following table.

Request parameters

Name

Parameter

Type

Default

Description

AppKey

AppKey

string

-

Required. Set this to the AppKey you created in the console.

Basic request information

(Input)

Input.FileUrl

string

-

Required. The download URL of the audio or video file. HTTP and HTTPS URLs are supported.

Input.SourceLanguage

string

-

Required, Configure the corresponding parameters based on the language categories in audio and video.

  • If you know the language in the audio, select the actual language: Chinese (cn), English (en), Cantonese (yue), Japanese (ja), or Korean (ko). This is the recommended method.

  • If you do not know the language in the audio, you can set this parameter to `auto` for automatic language detection. The system detects the language and then automatically switches to the corresponding model for speech recognition. This feature is available only for offline transcription tasks.

  • If the audio contains multiple languages, you can set this parameter to `multilingual` to recognize text in the corresponding languages. Use this in conjunction with `Input.LanguageHints`.

Input.LanguageHints

list[string]

null

List of languages for speech recognition when using a multilingual model.

When the audio contains multiple languages that all need to be recognized, this parameter is used to limit the language categories. It takes effect only when `Input.SourceLanguage` is set to `multilingual`.

Input.TaskKey

string

null

A custom ID set by the user.

Input.ProgressiveCallbacksEnabled

boolean

false

Specifies whether to enable the callback feature.

To enable the callback feature, configure the callback type and address in the console and set this parameter to `true` when creating the task.

Transcoding

Parameters.Transcoding.TargetAudioFormat

string

null

Currently, only conversion to MP3 is supported.

Parameters.Transcoding.TargetVideoFormat

string

null

Currently, only conversion to MP4 is supported. This parameter is meaningful only if the source file is a video.

Speech recognition

(Transcription)

Parameters.Transcription.DiarizationEnabled

boolean

false

Specifies whether to enable speaker diarization during speech recognition.

Parameters.Transcription.Diarization.SpeakerCount

int

-

The number of speakers to set when speaker diarization is enabled. A maximum of 100 speakers is supported.

  • Not set: Speaker roles are not differentiated.

  • 0: The number of speaker roles is indefinite.

If this parameter is set, it helps specify the number of speakers. The final number of separated tracks is based on the actual number of classified speakers.

Parameters.Transcription.PhraseId

String

-

The ID of the hotword vocabulary.

Translation

(Translation)

Parameters.TranslationEnabled

boolean

false

Specifies whether to enable the translation feature.

Parameters.Translation.TargetLanguages

list[string]

-

If translation is enabled, you must set the target translation languages.

Supported languages include Chinese (cn), English (en), Japanese (ja), Korean (ko), German (de), French (fr), and Russian (ru).

Chapter overview

Parameters.AutoChaptersEnabled

boolean

false

The chapter overview feature, which includes agenda titles and agenda summaries.

Key point extraction

(MeetingAssistance)

Parameters.MeetingAssistanceEnabled

boolean

false

The key point extraction feature, which includes keywords, key content, to-do items, and scenario recognition.

Parameters.MeetingAssistance.Types

list[string]

-

If you enable the key point extraction feature, you can set one or more of the following types:

  • Actions: To-do items

  • KeyInformation: Key information (including keywords and key content)

Note: If you only enable the feature but do not add any algorithm types, the model processes all algorithm models.

Summarization

(Summarization)

Parameters.SummarizationEnabled

boolean

false

Specifies whether to enable the summarization feature.

Parameters.Summarization.Types

list[string]

-

If you enable the summarization feature, you must set the summary types. You can set one or more types.

  • Paragraph (Full-text summary)

  • Conversational (Speaker summary)

  • QuestionsAnswering (Q&A review)

  • MindMap (Mind map)

PPT extraction and summarization

Parameters.PptExtractionEnabled

boolean

false

Specifies whether to enable the PPT feature, which includes PPT extraction and PPT summarization.

Note: This feature only supports PPTs in the main view (screen sharing or with a person's video nearby). It does not support scenarios where a person is walking or speaking in front of the PPT.

You can test the effect on the Tongyi Tingwu website. Click here to test

Spoken-to-written text conversion

Parameters.TextPolishEnabled

boolean

false

Specifies whether to enable the spoken-to-written text conversion feature.

Service quality inspection

Parameters.ServiceInspectionEnabled

boolean

false

Specifies whether to enable the service quality inspection feature.

Parameters.ServiceInspection

object

-

Refer to the Service Quality Inspection Parameters Object to conduct quality inspection of dialogues during service processes. It supports customizing multiple quality inspection dimensions to help customers improve their service levels.

Custom Prompt

Parameters.CustomPromptEnabled

boolean

false

Specifies whether to enable the custom prompt feature.

Parameters.CustomPrompt

object

-

Allows you to define custom prompts to guide the Large Language Model (LLM) in completing various tasks. For more information, see Custom prompt parameter object.

Global parameters for LLM post-processing tasks

Parameters.Model

String

-

The models supported for LLM post-processing tasks. Valid values: default, qwen-plus, and qwq. Note: This parameter does not overwrite the optional model parameter in Custom Prompt.

Parameters.LlmOutputLanguage

String

-

Specifies the output language for LLM tasks. Valid values:

  • cn: Chinese

  • en: English

Note: This parameter currently only takes effect for mind maps and chapter overviews.

Sample code

#!/usr/bin/env python
#coding=utf-8

import os
import json
import datetime
from aliyunsdkcore.client import AcsClient
from aliyunsdkcore.request import CommonRequest
from aliyunsdkcore.auth.credentials import AccessKeyCredential

def create_common_request(domain, version, protocolType, method, uri):
    request = CommonRequest()
    request.set_accept_format('json')
    request.set_domain(domain)
    request.set_version(version)
    request.set_protocol_type(protocolType)
    request.set_method(method)
    request.set_uri_pattern(uri)
    request.add_header('Content-Type', 'application/json')
    return request

def init_parameters():
    body = dict()
    body['AppKey'] = 'Enter the AppKey you created in the Tingwu console'

    # Basic request parameters
    input = dict()
    input['SourceLanguage'] = 'cn'
    input['TaskKey'] = 'task' + datetime.datetime.now().strftime('%Y%m%d%H%M%S')
    input['FileUrl'] = 'Enter the URL of the audio file to be tested'
    body['Input'] = input

    # AI-related parameters. Set them as needed.
    parameters = dict()

    # Audio and video conversion
    transcoding = dict()
    # Convert the source audio or video file to MP3 for playback in a browser
    # transcoding['TargetAudioFormat'] = 'mp3'
    # transcoding['SpectrumEnabled'] = False
    # parameters['Transcoding'] = transcoding

    # Speech recognition control
    transcription = dict()
    # Speaker diarization: Optional
    transcription['DiarizationEnabled'] = True
    diarization = dict()
    diarization['SpeakerCount'] = 2
    transcription['Diarization'] = diarization
    parameters['Transcription'] = transcription

    # Text translation control: Optional
    parameters['TranslationEnabled'] = True
    translation = dict()
    translation['TargetLanguages'] = ['en'] # Translate to English
    parameters['Translation'] = translation

    # Chapter overview: Optional. Includes titles and agenda summaries.
    parameters['AutoChaptersEnabled'] = True

    # Intelligent minutes: Optional. Includes to-do items and key information (keywords, key content, scenario recognition).
    parameters['MeetingAssistanceEnabled'] = True
    meetingAssistance = dict()
    meetingAssistance['Types'] = ['Actions', 'KeyInformation']
    parameters['MeetingAssistance'] = meetingAssistance

    # Summarization control: Optional. Includes full-text summary, speaker summary, and Q&A summary (Q&A review).
    parameters['SummarizationEnabled'] = True
    summarization = dict()
    summarization['Types'] = ['Paragraph', 'Conversational', 'QuestionsAnswering', 'MindMap']
    parameters['Summarization'] = summarization

    # PPT extraction and summarization: Optional
    parameters['PptExtractionEnabled'] = True
    
    # Spoken-to-written text conversion: Optional
    parameters['TextPolishEnabled'] = True
    
    # Global parameters for LLM post-processing tasks: Optional
    parameters['Model'] = 'qwq'
    parameters['LlmOutputLanguage'] = 'en'

    body['Parameters'] = parameters
    return body

body = init_parameters()
print(body)

# TODO  Set your AccessKeyId and AccessKeySecret using environment variables
credentials = AccessKeyCredential(os.environ['ALIBABA_CLOUD_ACCESS_KEY_ID'], os.environ['ALIBABA_CLOUD_ACCESS_KEY_SECRET'])
client = AcsClient(region_id='cn-beijing', credential=credentials)

request = create_common_request('tingwu.cn-beijing.aliyuncs.com', '2023-09-30', 'https', 'PUT', '/openapi/tingwu/v2/tasks')
request.add_query_param('type', 'offline')

request.set_content(json.dumps(body).encode('utf-8'))
response = client.do_action_with_exception(request)
print("response: \n" + json.dumps(json.loads(response), indent=4, ensure_ascii=False))
package com.alibaba.tingwu.client.demo.offlinetask;

import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import com.aliyuncs.CommonRequest;
import com.aliyuncs.CommonResponse;
import com.aliyuncs.DefaultAcsClient;
import com.aliyuncs.IAcsClient;
import com.aliyuncs.exceptions.ClientException;
import com.aliyuncs.http.FormatType;
import com.aliyuncs.http.MethodType;
import com.aliyuncs.http.ProtocolType;
import com.aliyuncs.profile.DefaultProfile;
import org.junit.Test;

/**
 * @author tingwu2023
 * @desc This demo shows how to call the OpenAPI to create an offline transcription task for an audio or video file.
 */
public class SubmitFiletransTaskTest {

    @Test
    public void summitTask() throws ClientException {
        CommonRequest request = createCommonRequest("tingwu.cn-beijing.aliyuncs.com", "2023-09-30", ProtocolType.HTTPS, MethodType.PUT, "/openapi/tingwu/v2/tasks");
        request.putQueryParameter("type", "offline");

        JSONObject root = new JSONObject();
        root.put("AppKey", "Enter the AppKey you created in the Tingwu console");

        String path = "Enter the URL of the audio file to be tested";
        JSONObject input = new JSONObject();
        input.fluentPut("FileUrl", path)
                .fluentPut("SourceLanguage", "cn")
                .fluentPut("TaskKey", "task" + System.currentTimeMillis());
        root.put("Input", input);

        JSONObject parameters = initRequestParameters();
        root.put("Parameters", parameters);

        System.out.println(root.toJSONString());
        request.setHttpContent(root.toJSONString().getBytes(), "utf-8", FormatType.JSON);

        // TODO Set your AccessKeyId and AccessKeySecret using environment variables
        DefaultProfile profile = DefaultProfile.getProfile("cn-beijing", System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
        IAcsClient client = new DefaultAcsClient(profile);
        CommonResponse response = client.getCommonResponse(request);
        System.out.println(response.getData());
        JSONObject body = JSONObject.parseObject(response.getData());
        JSONObject data = (JSONObject) body.get("Data");
        System.out.println("TaskId = " + data.getString("TaskId"));
    }

    public static JSONObject initRequestParameters() {
        JSONObject parameters = new JSONObject();

        // Audio and video conversion: Optional
        JSONObject transcoding = new JSONObject();
        transcoding.put("TargetAudioFormat", "mp3");
        transcoding.put("SpectrumEnabled", false);
        parameters.put("Transcoding", transcoding);

        // Speech recognition
        JSONObject transcription = new JSONObject();
        transcription.put("DiarizationEnabled", true);
        JSONObject speaker_count = new JSONObject();
        speaker_count.put("SpeakerCount", 2);
        transcription.put("Diarization", speaker_count);
        parameters.put("Transcription", transcription);

        // Translation: Optional
        JSONObject translation = new JSONObject();
        JSONArray langArry = new JSONArray();
        langArry.add("en");
        translation.put("TargetLanguages", langArry);
        parameters.put("Translation", translation);
        parameters.put("TranslationEnabled", true);

        // Chapter overview: Optional
        parameters.put("AutoChaptersEnabled", true);

        // Intelligent minutes: Optional
        parameters.put("MeetingAssistanceEnabled", true);
        JSONObject meetingAssistance = new JSONObject();
        JSONArray mTypes = new JSONArray().fluentAdd("Actions").fluentAdd("KeyInformation");
        meetingAssistance.put("Types", mTypes);
        parameters.put("MeetingAssistance", meetingAssistance);
      
        // Summarization: Optional
        parameters.put("SummarizationEnabled", true);
        JSONObject summarization = new JSONObject();
        JSONArray sTypes = new JSONArray().fluentAdd("Paragraph").fluentAdd("Conversational").fluentAdd("QuestionsAnswering").fluentAdd("MindMap");
        summarization.put("Types", sTypes);
        parameters.put("Summarization", summarization);

        // PPT extraction: Optional
        parameters.put("PptExtractionEnabled", true);
      
      	// Spoken-to-written text conversion: Optional
        parameters.put("TextPolishEnabled", true);
        
        // Global parameters for LLM post-processing tasks: Optional
        parameters.put("Model", "qwq"); 
        parameters.put("LlmOutputLanguage", "en");

        return parameters;
    }
  
    public static CommonRequest createCommonRequest(String domain, String version, ProtocolType protocolType, MethodType method, String uri) {
        // Create an API request and set parameters
        CommonRequest request = new CommonRequest();
        request.setSysDomain(domain);
        request.setSysVersion(version);
        request.setSysProtocol(protocolType);
        request.setSysMethod(method);
        request.setSysUriPattern(uri);
        request.setHttpContentType(FormatType.JSON);
        return request;
    }
}
package main

import (
	"encoding/json"
	"fmt"
	"log"
	"os"
	"time"

	"github.com/aliyun/alibaba-cloud-sdk-go/sdk"
	"github.com/aliyun/alibaba-cloud-sdk-go/sdk/requests"
)

type TranscodeingParam struct {
	TargetAudioFormat     string `json:"TargetAudioFormat,omitempty"`
	TargetVideoFormat     string `json:"TargetVideoFormat,omitempty"`
	VideoThumbnailEnabled bool   `json:"VideoThumbnailEnabled,omitempty"`
	SpectrumEnabled       bool   `json:"SpectrumEnabled,omitempty"`
}

type DiarizationParam struct {
	SpeakerCount *int `json:"SpeakerCount,omitempty"`
}

type TranscriptionParam struct {
	AudioEventDetectionEnabled bool              `json:"AudioEventDetectionEnabled,omitempty"`
	DiarizationEnabled         bool              `json:"DiarizationEnabled,omitempty"`
	Diarization                *DiarizationParam `json:"Diarization,omitempty"`
}

type TranslationParam struct {
	TargetLanguages []string `json:"TargetLanguages,omitempty"`
}

type SummarizationParam struct {
	Types []string `json:"Types,omitempty"`
}

type ExtraParamerters struct {
	Transcoding              *TranscodeingParam  `json:"Transcoding,omitempty"`
	Transcription            *TranscriptionParam `json:"Transcription,omitempty"`
	TranslationEnabled       bool                `json:"TranslationEnabled,omitempty"`
	Translation              *TranslationParam   `json:"Translation,omitempty"`
	AutoChaptersEnabled      bool                `json:"AutoChaptersEnabled,omitempty"`
	MeetingAssistanceEnabled bool                `json:"MeetingAssistanceEnabled,omitempty"`
	SummarizationEnabled     bool                `json:"SummarizationEnabled,omitempty"`
	Summarization            *SummarizationParam `json:"Summarization,omitempty"`
	TextPolishEnabled        bool                `json:"TextPolishEnabled,omitempty"`
	Model                    string              `json:"Model,omitempty"`           
        LlmOutputLanguage        string              `json:"LlmOutputLanguage,omitempty"` 
}

type InputParam struct {
	SourceLanguage string `json:"SourceLanguage"`
	FileUrl        string `json:"FileUrl,omitempty"`
	TaskKey        string `json:"TaskKey,omitempty"`
	Format         string `json:"Format,omitempty"`
	SampleRate     int    `json:"SampleRate,omitempty"`
}

type TaskBodyParam struct {
	Appkey      string            `json:"AppKey"`
	Input       InputParam        `json:"Input"`
	Paramerters *ExtraParamerters `json:"Parameters,omitempty"`
}

type GetTaskInfoResponse struct {
	RequestId string `json:"RequestId"`
	Code      string `json:"Code"`
	Message   string `json:"Message"`
	Data      struct {
		TaskId     string `json:"TaskId"`
		TaskKey    string `json:"TaskKey"`
		TaskStatus string `json:"TaskStatus"`
	} `json:"Data"`
}

func init_request_param_offline() *ExtraParamerters {
	param := new(ExtraParamerters)
	param.Transcoding = new(TranscodeingParam)
	transcription := new(TranscriptionParam)
	transcription.Diarization = new(DiarizationParam)
	transcription.Diarization.SpeakerCount = new(int)
        *transcription.Diarization.SpeakerCount = 0
	transcription.DiarizationEnabled = true
	param.Transcription = transcription

	param.TranslationEnabled = false
	param.AutoChaptersEnabled = false
	param.MeetingAssistanceEnabled = false
	param.SummarizationEnabled = true

	summarization := new(SummarizationParam)
	summarization.Types = []string{
		"Paragraph",
		"Conversational",
		"QuestionsAnswering",
                "MindMap",
	}
	param.Summarization = summarization
	param.TextPolishEnabled = false
	
	param.Model = "qwq"              
        param.LlmOutputLanguage = "en"

	return param
}

func test_submit_offline_task() {
	akkey := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_ID")
	aksecret := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET")
	client, err := sdk.NewClientWithAccessKey("cn-beijing", akkey, aksecret)
	if err != nil {
		log.Default().Fatalln(err)
		return
	}

	request := requests.NewCommonRequest()
	request.Method = "PUT"
	request.Domain = "tingwu.cn-beijing.aliyuncs.com"
	request.Version = "2023-09-30"
	request.SetContentType("application/json")
	request.PathPattern = "/openapi/tingwu/v2/tasks"
	request.QueryParams["type"] = "offline"

	param := new(TaskBodyParam)
	param.Appkey = "Enter the AppKey you created in the Tingwu console"
	param.Input.SourceLanguage = "cn"
	param.Input.FileUrl = "https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example.wav"
	param.Input.TaskKey = "task_" + fmt.Sprint(time.Now().Unix())

	param.Paramerters = init_request_param_offline()
        b, _ := json.Marshal(param)
	log.Default().Print("request body:\n", string(b))
	request.SetContent(b)
	request.SetScheme("https")

	response, err := client.ProcessCommonRequest(request)
	if err != nil {
		log.Default().Fatalln(err)
		return
	}

	log.Default().Print("response body:\n", string(response.GetHttpContentBytes()))
}

func main() {
	test_submit_offline_task()
}
#include <cstdlib>
#include <string>
#include <alibabacloud/core/AlibabaCloud.h>
#include <alibabacloud/core/CommonRequest.h>
#include <alibabacloud/core/CommonClient.h>
#include <alibabacloud/core/CommonResponse.h>
#include "jsoncpp/json.h"
/**
 * @author tingwu2023
 * @desc This demo shows how to call the OpenAPI to create an offline transcription task for an audio or video file.
 */
int main( int argc, char** argv ) {
    AlibabaCloud::InitializeSdk();
    AlibabaCloud::ClientConfiguration configuration( "cn-beijing" );
    // specify timeout when create client.
    configuration.setConnectTimeout(1500);
    configuration.setReadTimeout(4000);
    // Please ensure that the environment variables ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET are set.
    AlibabaCloud::Credentials credential( getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET") );
    AlibabaCloud::CommonClient client( credential, configuration );

    AlibabaCloud::CommonRequest request(AlibabaCloud::CommonRequest::RequestPattern::RoaPattern);
    request.setHttpMethod(AlibabaCloud::HttpRequest::Method::Put);
    request.setDomain("tingwu.cn-beijing.aliyuncs.com");
    request.setVersion("2023-09-30");
    request.setResourcePath("/openapi/tingwu/v2/tasks");
    request.setHeaderParameter("Content-Type", "application/json");
    request.setQueryParameter("type", "offline");

    Json::Value root;
    root["Appkey"] = "Enter the AppKey you created in the Tingwu console";

    Json::Value input;
    input["FileUrl"] = "Enter the URL of the audio file to be tested";
    input["SourceLanguage"] = "cn";
    input["TaskKey"] = "Enter a custom ID for this request";
    root["Input"] = input;

    Json::Value parameters;
    // Audio and video file conversion. Generally, you do not need to set this parameter.
    // Json::Value transcoding;
    // transcoding["TargetAudioFormat"] = "mp3";
    // parameters["Transcoding"] = transcoding;

    // Speech transcription: The following enables speaker diarization. You do not need to set this if it is not required.
    Json::Value transcription;
    transcription["DiarizationEnabled"] = true;
    Json::Value speakerCount;
    speakerCount["SpeakerCount"] = 2;
    transcription["Diarization"] = speakerCount;
    parameters["Transcription"] = transcription;

    // Translation: Optional
    Json::Value translation;
    Json::Value langauges;
    langauges.append("en");
    translation["TargetLanguages"] = langauges;
    parameters["Translation"] = translation;
    parameters["TranslationEnabled"] = true;

    // Chapter overview: Optional
    parameters["AutoChaptersEnabled"] = true;

    // Intelligent minutes: Optional. Includes to-do items and key information (keywords, key content, scenario recognition).
    parameters["MeetingAssistanceEnabled"] = true;
    Json::Value meetingAssistance;
    Json::Value meetingAssistanceTypeList;
    meetingAssistanceTypeList.append("Actions");
    meetingAssistanceTypeList.append("KeyInformation");
    meetingAssistance["Types"] = meetingAssistanceTypeList;
    parameters["MeetingAssistanceTypeList"] = meetingAssistanceTypeList;

    // Summarization: Optional. The following settings enable all three summary types. You can add or remove them as needed.
    parameters["SummarizationEnabled"] = true;
    Json::Value summarization;
    Json::Value summarizationTypeList;
    summarizationTypeList.append("Paragraph");
    summarizationTypeList.append("Conversational");
    summarizationTypeList.append("QuestionsAnswering");
    summarizationTypeList.append("MindMap");
    summarization["Types"] = summarizationTypeList;
    parameters["Summarization"] = summarization;

    // PPT extraction: Optional. We recommend enabling this only for video files that contain PPT-related video frames.
    //parameters["PptExtractionEnabled"] = true;
  	
    // Spoken-to-written text conversion: Optional
    parameters["TextPolishEnabled"] = true;
    
    # Global parameters for LLM post-processing tasks: Optional
    parameters["Model"] = "qwq";
    parameters["LlmOutputLanguage"] = "en";

    root["Parameters"] = parameters;

    Json::FastWriter writer;
    std::string body = writer.write(root);
    printf("input json: [%s]\n", body.c_str());
    request.setContent(body.c_str(), body.size());
      
    auto response = client.commonResponse(request);
    if (response.isSuccess()) {
        printf("request success.\n");
        printf("result: %s\n", response.result().payload().c_str());
    } else {
        printf("error: %s\n", response.error().errorMessage().c_str());
        printf("request id: %s\n", response.error().requestId().c_str());
    }
      
    AlibabaCloud::ShutdownSdk();
    return 0;
} 

Sample output

When the task is running:

{
    "Code":"0",
    "Data":{
        "TaskId":"e8adc0b3vc4b45d898fcadb*********",
        "TaskKey":"task16988********",
        "TaskStatus":"ONGOING"
    },
    "Message":"success",
    "RequestId":"2001dc2a-9b46-4a2f-9822-b140********"
}

Protocol parsing

The fields are defined as follows.

Parameter Name

Type

Description

TaskId

string

The ID generated when the task was created. Use it to query the task status and result, and for troubleshooting.

TaskStatus

string

The task status.

TaskKey

string

The TaskKey you set when creating the task.

RequestId

string

The ID of the request. Use it for troubleshooting.

After you submit an offline audio or video transcription task, the task enters the running state.

Query task status and result

  • When you query a task, you must use the TaskId that was returned when you submitted the task. You can then initiate a query request and check the returned status to determine whether the task is complete.

Request parameters

Parameter

Type

Required

Description

TaskId

string

Yes

The TaskId returned when you submitted the task.

Sample code

#!/usr/bin/env python
#coding=utf-8

import os
import json
from aliyunsdkcore.client import AcsClient
from aliyunsdkcore.request import CommonRequest
from aliyunsdkcore.auth.credentials import AccessKeyCredential

def create_common_request(domain, version, protocolType, method, uri):
    request = CommonRequest()
    request.set_accept_format('json')
    request.set_domain(domain)
    request.set_version(version)
    request.set_protocol_type(protocolType)
    request.set_method(method)
    request.set_uri_pattern(uri)
    request.add_header('Content-Type', 'application/json')
    return request

# TODO  Set your AccessKeyId and AccessKeySecret using environment variables
credentials = AccessKeyCredential(os.environ['ALIBABA_CLOUD_ACCESS_KEY_ID'], os.environ['ALIBABA_CLOUD_ACCESS_KEY_SECRET'])
client = AcsClient(region_id='cn-beijing', credential=credentials)

uri = '/openapi/tingwu/v2/tasks' + '/' + 'Enter the TaskId returned when you submitted the task'
request = create_common_request('tingwu.cn-beijing.aliyuncs.com', '2023-09-30', 'https', 'GET', uri)

response = client.do_action_with_exception(request)
print("response: \n" + json.dumps(json.loads(response), indent=4, ensure_ascii=False))
package com.alibaba.tingwu.client.demo.common;

import com.aliyuncs.CommonRequest;
import com.aliyuncs.CommonResponse;
import com.aliyuncs.DefaultAcsClient;
import com.aliyuncs.IAcsClient;
import com.aliyuncs.exceptions.ClientException;
import com.aliyuncs.http.MethodType;
import com.aliyuncs.http.ProtocolType;
import com.aliyuncs.profile.DefaultProfile;
import org.junit.Test;

/**
 * @author tingwu2023
 * @desc This demo shows how to call the OpenAPI to query the task status and result by TaskId.
 */
public class GetTaskInfoTest {
    @Test
    public void getTaskInfo() throws ClientException {
        String taskId = "Enter the TaskId of the created task (offline transcription or real-time meeting)";
        String queryUrl = String.format("/openapi/tingwu/v2/tasks" + "/%s", taskId);

        CommonRequest request = createCommonRequest("tingwu.cn-beijing.aliyuncs.com", "2023-09-30", ProtocolType.HTTPS, MethodType.GET, queryUrl);
        // TODO Set your AccessKeyId and AccessKeySecret using environment variables
        DefaultProfile profile = DefaultProfile.getProfile("cn-beijing", System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
        IAcsClient client = new DefaultAcsClient(profile);
        CommonResponse response = client.getCommonResponse(request);
        System.out.println(response.getData());
    }
  
    public static CommonRequest createCommonRequest(String domain, String version, ProtocolType protocolType, MethodType method, String uri) {
        // Create an API request and set parameters
        CommonRequest request = new CommonRequest();
        request.setSysDomain(domain);
        request.setSysVersion(version);
        request.setSysProtocol(protocolType);
        request.setSysMethod(method);
        request.setSysUriPattern(uri);
        request.setHttpContentType(FormatType.JSON);
        return request;
    }
}
package main

import (
	"encoding/json"
	"log"
	"os"

	"github.com/aliyun/alibaba-cloud-sdk-go/sdk"
	"github.com/aliyun/alibaba-cloud-sdk-go/sdk/requests"
)

type GetTaskInfoResponse struct {
	RequestId string `json:"RequestId"`
	Code      string `json:"Code"`
	Message   string `json:"Message"`
	Data      struct {
		TaskId     string `json:"TaskId"`
		TaskKey    string `json:"TaskKey"`
		TaskStatus string `json:"TaskStatus"`
	} `json:"Data"`
}

func get_task_info(taskid string, akkey string, aksecret string) (*GetTaskInfoResponse, string, error) {
	client, err := sdk.NewClientWithAccessKey("cn-beijing", akkey, aksecret)
	if err != nil {
		return nil, "", err
	}

	request := requests.NewCommonRequest()
	request.Method = "GET"
	request.Domain = "tingwu.cn-beijing.aliyuncs.com"
	request.Version = "2023-09-30"
	request.PathPattern = "/openapi/tingwu/v2/tasks/" + taskid
	request.SetScheme("https")

	log.Default().Print("request task:", taskid)
	response, err := client.ProcessCommonRequest(request)
	if err != nil {
		return nil, "", err
	}

	log.Default().Print("response body:\n", string(response.GetHttpContentBytes()))
	resp := new(GetTaskInfoResponse)
	err = json.Unmarshal(response.GetHttpContentBytes(), resp)
	if err != nil {
		return nil, response.GetHttpContentString(), err
	}
	return resp, response.GetHttpContentString(), nil
}

func main() {
	akkey := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_ID")
	aksecret := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET")
	_, raw, _ := get_task_info("Enter the TaskId of the created task (offline transcription or real-time meeting)", akkey, aksecret)
	log.Default().Println("response :", raw)
}
#include <cstdlib>
#include <iostream>
#include <string>
#include <alibabacloud/core/AlibabaCloud.h>
#include <alibabacloud/core/CommonRequest.h>
#include <alibabacloud/core/CommonClient.h>
#include <alibabacloud/core/CommonResponse.h>
/**
 * @author tingwu2023
 * @desc This demo shows how to call the OpenAPI to query the task status and result by TaskId.
 */
int main( int argc, char** argv ) {
    std::string taskId = "Enter the TaskId of the created task (offline transcription or real-time meeting)";

    AlibabaCloud::InitializeSdk();
    AlibabaCloud::ClientConfiguration configuration( "cn-beijing" );
    // specify timeout when create client.
    configuration.setConnectTimeout(1500);
    configuration.setReadTimeout(4000);
    // Please ensure that the environment variables ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET are set.
    AlibabaCloud::Credentials credential( getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET") );
    AlibabaCloud::CommonClient client( credential, configuration );
    AlibabaCloud::CommonRequest request(AlibabaCloud::CommonRequest::RequestPattern::RoaPattern);
    request.setHttpMethod(AlibabaCloud::HttpRequest::Method::Get);
    request.setDomain("tingwu.cn-beijing.aliyuncs.com");
    request.setVersion("2023-09-30");
    request.setHeaderParameter("Content-Type", "application/json");
    request.setResourcePath("/openapi/tingwu/v2/tasks/" + taskId);

    auto response = client.commonResponse(request);
    if (response.isSuccess()) {
        printf("request success.\n");
        printf("result: %s\n", response.result().payload().c_str());
    } else {
        printf("error: %s\n", response.error().errorMessage().c_str());
        printf("request id: %s\n", response.error().requestId().c_str());
    }
      
    AlibabaCloud::ShutdownSdk();
    return 0;
} 

Sample output

When the task is running:

{
    "Code":"0",
    "Data":{
        "TaskId":"e8adc0b3bc4b42d898fcadb0********",
        "TaskStatus":"ONGOING"
    },
    "Message":"success",
    "RequestId":"5fa32fc6-441f-4dd1-bb86-c030********"
}

When the task is running and partial results are available:

{
    "Code":"0",
    "Data":{
        "TaskId":"e8adc0b3bc4b42d898fcadb0********",
        "TaskStatus":"ONGOING",
        "OutputMp3Path":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_20231101141801.mp3?ExLTAI****************AccessKeyId=LTAI****************&amp;Signature=********JBMijH7wLq0xX6aivHc%3D",
        "Result":{
            "Transcription":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu_data/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_Transcription_20231101141926.json?Expires=1698906034&amp;OSSAcceLTAI****************&amp;Signature=********NJlqSEWJxfkMwjwsHCA%3D"
        }
    },
    "Message":"success",
    "RequestId":"1b20e0d9-c55c-4cc3-85af-80b4********"
}

When the task is completed:

{
    "Code":"0",
    "Data":{
        "TaskId":"e8adc0b3bc4b42d898fcadb0********",
        "TaskStatus":"COMPLETED",
        "OutputMp3Path":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_20231101141801.mp3?ExLTAI****************AccessKeyId=LTAI****************&amp;Signature=********JBMijH7wLq0xX6aivHc%3D",
        "Result":{
            "AutoChapters":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu_data/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_AutoChapters_20231101141955.json?Expires=1698906034&amp;OSSAcceLTAI****************&amp;Signature=********Ax9FvifYAO8dj4qzWg%3D",
            "Transcription":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu_data/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_Transcription_20231101141926.json?Expires=1698906034&amp;OSSAccessKeyId=LTAI****************&amp;Signature=********NJlqSEWJxfkMwjwsHCA%3D"
        }
    },
    "Message":"success",
    "RequestId":"1b20e0d9-c55c-4cc3-85af-80b4********"
}

When the task has failed:

{
    "Code":"0",
    "Data":{
        "TaskId":"b76389677b1441fa82165cb1********",
        "TaskStatus":"FAILED",
        "ErrorCode":"TSC.AudioFileLink",
        "ErrorMessage":"Audio file link invalid."
    },
    "Message":"success",
    "RequestId":"d181d898-b627-4040-b7c9-9563********"
}

Callback notifications for task status and results

If you want to receive task status and results through callbacks, you must configure the callback type and address in the console. When you create an offline transcription task, you must set the `AppKey` parameter to the corresponding project AppKey and set the `Input.ProgressiveCallbacksEnabled` parameter to `true`.

Important

When you use the HTTP callback method, a maximum request timeout period is set. This means that if the service does not receive a response from your server within 5 seconds after sending the request, the service stops waiting for a response. The system then determines that the callback request failed and retries the request later.

We recommend that you review and adjust your system to ensure that it can respond to callback requests within 5 seconds. This may include, but is not limited to, the following actions:

  • Optimize your processing logic for efficient execution.

  • Handle unnecessarily long operations asynchronously instead of waiting for them in the main callback response path.

  • Implement error handling and retry mechanisms to cope with timeouts or failures.

Note

Callback retry mechanism:

If a callback message fails to be sent, the Tingwu server resends the callback message after 5 seconds. If the second attempt also fails, the message is resent every 5 minutes thereafter, up to a maximum of 3 retries.

Sample output

When a single AI model sub-task is completed:

{
    "Code":"0",
    "Data":{
        "TaskId":"e8adc0b3bc4b42d898fcadb0********",
        "TaskStatus":"ONGOING",
        "TaskKey":"TingwuDemo",
        "Result":{
            "Transcription":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu_data/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_Transcription_20231101141926.json?Expires=1698906034&amp;OSSAcceLTAI****************&amp;Signature=********NJlqSEWJxfkMwjwsHCA%3D"
        }
    },
    "Message":"success",
    "RequestId":"5cf2b587ac6e41e6a1f06ca1********"
}

When the task is completed:

{
    "Code":"0",
    "Data":{
        "TaskId":"e8adc0b3bc4b42d898fcadb0********",
        "TaskStatus":"COMPLETED",
        "TaskKey":"TingwuDemo",
        "Result":{
            "AutoChapters":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu_data/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_AutoChapters_20231101141955.json?Expires=1698906034&amp;OSSAcceLTAI****************&amp;Signature=********Ax9FvifYAO8dj4qzWg%3D",
            "Transcription":"http://speech-swap.oss-cn-zhangjiakou.aliyuncs.com/tingwu_data/output/1738248129743478/e8adc0b3bc4b42d898fcadb0a1710635/e8adc0b3bc4b42d898fcadb0a1710635_Transcription_20231101141926.json?Expires=1698906034&amp;OSSAccessKeyId=LTAI****************&amp;Signature=********NJlqSEWJxfkMwjwsHCA%3D"
        }
    },
    "Message":"success",
    "RequestId":"9c60b07f152445349eaa6161********"
}

When the task has failed:

{
    "Code":"0",
    "Data":{
        "TaskId":"e8adc0b3bc4b42d898fcadb0********",
        "TaskStatus":"FAILED",
        "TaskKey":"TingwuDemo",
        "ErrorCode":"TSC.AudioFileLink",
        "ErrorMessage":"Audio file link invalid."
    },
    "Message":"success",
    "RequestId":"5e1c0babe36844e49f1b7bee********"
}

Rerun a task to refresh the result

To refresh a completed task, for example, to regenerate a specific algorithm result or add a new one, you can initiate the task processing request again. Use the TaskId that was returned when you submitted the task and the new control parameters as input, and then wait for the result.

For example, if you did not enable the chapter overview feature when you first created the task, you can add the new parameter to continue the task. This way, you do not need to resubmit the audio or video file and start from scratch.

Important

You can only rerun a task that is already completed (COMPLETED). You cannot rerun a task that is running or has failed.

Request parameters

Feature name

Parameter

Type

Default

Description

Basic request information

(Input)

TaskId

string

-

Required. The TaskId of the task you want to rerun.

Translation

(Translation)

TranslationEnabled

boolean

false

Specifies whether to enable the translation feature.

Translation.TargetLanguages

list[string]

[en]

  • If translation is enabled, you must set the target translation languages.

  • Supported languages include Chinese (cn), English (en), Japanese (ja), Korean (ko), German (de), French (fr), and Russian (ru).

Chapter overview

AutoChaptersEnabled

boolean

false

The chapter overview feature, which includes agenda titles and agenda summaries.

Key point extraction

MeetingAssistanceEnabled

boolean

false

Keywords, action items, to-do items, and scenario recognition.

Summarization

SummarizationEnabled

boolean

false

Specifies whether to enable the summarization feature.

Summarization.Types

list[string]

[]

You can set one or more types.

  • Paragraph (Full-text summary)

  • Conversational (Speaker summary)

  • QuestionsAnswering (Q&A review, key point review)

  • MindMap (Mind map)

PPT extraction and summarization

PptExtractionEnabled

boolean

false

Specifies whether to enable the PPT feature, which includes PPT extraction and PPT summarization.

Spoken-to-written text conversion

TextPolishEnabled

boolean

false

Specifies whether to enable the spoken-to-written text conversion feature.

Service quality inspection

ServiceInspectionEnabled

boolean

false

Specifies whether to enable the service quality inspection feature.

ServiceInspection

object

-

Refer to the Service Quality Inspection Parameter Object to perform quality inspection on conversations during service processes, support customizing multiple quality inspection dimensions, and help customers improve service levels.

Custom Prompt

CustomPromptEnabled

boolean

false

Specifies whether to enable the custom prompt feature.

CustomPrompt

object

-

Allows you to define custom prompts to guide the Large Language Model (LLM) to complete various tasks that you specify. For more information, see CustomPrompt parameter object.

Sample code

#!/usr/bin/env python
#coding=utf-8

import os
import json
import datetime
from aliyunsdkcore.client import AcsClient
from aliyunsdkcore.request import CommonRequest
from aliyunsdkcore.auth.credentials import AccessKeyCredential

def create_common_request(domain, version, protocolType, method, uri):
    request = CommonRequest()
    request.set_accept_format('json')
    request.set_domain(domain)
    request.set_version(version)
    request.set_protocol_type(protocolType)
    request.set_method(method)
    request.set_uri_pattern(uri)
    request.add_header('Content-Type', 'application/json')
    return request

def init_parameters():
    root = dict()
    root['AppKey'] = 'Enter the AppKey you created in the Tingwu console'
    # Basic request parameters
    input = dict()
    input['TaskId'] = 'Enter the TaskId of the completed task'
    root['Input'] = input

    # AI-related parameters. Set them as needed.
    parameters = dict()
    # Audio and video conversion: Rerunning is not supported.
    # Speech recognition: Rerunning the speech recognition algorithm is not supported.

    # Translation: Optional
    parameters['TranslationEnabled'] = True
    translation = dict()
    translation['TargetLanguages'] = ['en'] # Translate to English
    parameters['Translation'] = translation

    # Chapter overview: Optional
    parameters['AutoChaptersEnabled'] = True

    # Intelligent minutes: Optional
    parameters['MeetingAssistanceEnabled'] = True

    # Summarization: Optional
    parameters['SummarizationEnabled'] = True
    summarization = dict()
    summarization['Types'] = ['Paragraph', 'Conversational', 'QuestionsAnswering', 'MindMap']
    parameters['Summarization'] = summarization
    root['Parameters'] = parameters

    # PPT extraction: Optional (Rerunning is not supported for real-time meetings or offline transcription tasks where the source file is an audio format).
    parameters['PptExtractionEnabled'] = True
    
    # Spoken-to-written text conversion: Optional
    parameters['TextPolishEnabled'] = True
    
    root['Parameters'] = parameters
    return root

body = init_parameters()
print(body)

# TODO  Set your AccessKeyId and AccessKeySecret using environment variables
credentials = AccessKeyCredential(os.environ['ALIBABA_CLOUD_ACCESS_KEY_ID'], os.environ['ALIBABA_CLOUD_ACCESS_KEY_SECRET'])
client = AcsClient(region_id='cn-beijing', credential=credentials)

request = create_common_request('tingwu.cn-beijing.aliyuncs.com', '2023-09-30', 'https', 'PUT', '/openapi/tingwu/v2/tasks')
request.add_query_param('type', 'offline')

request.set_content(json.dumps(body).encode('utf-8'))
response = client.do_action_with_exception(request)
print("response: \n" + json.dumps(json.loads(response), indent=4, ensure_ascii=False))
package com.alibaba.tingwu.client.demo.common;

import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import com.aliyuncs.CommonRequest;
import com.aliyuncs.CommonResponse;
import com.aliyuncs.DefaultAcsClient;
import com.aliyuncs.IAcsClient;
import com.aliyuncs.exceptions.ClientException;
import com.aliyuncs.http.FormatType;
import com.aliyuncs.http.MethodType;
import com.aliyuncs.http.ProtocolType;
import com.aliyuncs.profile.DefaultProfile;
import org.junit.Test;

/**
 * @author tingwu2023
 * @desc This demo shows how to call the OpenAPI to rerun a completed task by TaskId.
 */
public class RerunTaskTest {

    @Test
    public void testRerunTask() throws ClientException {
        CommonRequest request = createCommonRequest("tingwu.cn-beijing.aliyuncs.com", "2023-09-30", ProtocolType.HTTPS, MethodType.PUT, "/openapi/tingwu/v2/tasks");
        request.putQueryParameter("type", "offline");

        JSONObject root = new JSONObject();
        JSONObject input = new JSONObject();
        input.put("TaskId", "Enter the TaskId of the completed task");
        root.put("Input", input);
        root.put("AppKey", "Enter the AppKey you created in the Tingwu console");

        JSONObject parameters = initRequestParametersForReRun();
        root.put("Parameters", parameters);

        System.out.println(root.toJSONString());
        request.setHttpContent(root.toJSONString().getBytes(), "utf-8", FormatType.JSON);

        // TODO Set your AccessKeyId and AccessKeySecret using environment variables
        DefaultProfile profile = DefaultProfile.getProfile("cn-beijing", System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
        IAcsClient client = new DefaultAcsClient(profile);
        CommonResponse response = client.getCommonResponse(request);
        System.out.println(response.getData());
    }

    public static JSONObject initRequestParametersForReRun() {
        JSONObject parameters = new JSONObject();

        // Audio and video conversion: Rerunning is not supported.

        // Speech recognition: Rerunning the speech recognition algorithm is not supported.

        // Translation: Optional
        JSONObject translation = new JSONObject();
        JSONArray langArry = new JSONArray();
        langArry.add("en");
        translation.put("TargetLanguages", langArry);
        parameters.put("Translation", translation);
        parameters.put("TranslationEnabled", true);

        // Chapter overview: Optional
        parameters.put("AutoChaptersEnabled", true);

        // Intelligent minutes: Optional
        parameters.put("MeetingAssistanceEnabled", true);

        // Summarization: Optional
        parameters.put("SummarizationEnabled", true);
        JSONObject summarization = new JSONObject();
        JSONArray types = new JSONArray().fluentAdd("Paragraph").fluentAdd("Conversational").fluentAdd("QuestionsAnswering").fluentAdd("MindMap");
        summarization.put("Types", types);
        parameters.put("Summarization", summarization);

        // PPT extraction: Optional (Rerunning is not supported for real-time meetings or offline transcription tasks where the source file is an audio format).
        //parameters.put("PptExtractionEnabled", false);
      
      	// Spoken-to-written text conversion: Optional
        parameters.put("TextPolishEnabled", true);

        return parameters;
    }
  
    public static CommonRequest createCommonRequest(String domain, String version, ProtocolType protocolType, MethodType method, String uri) {
        CommonRequest request = new CommonRequest();
        request.setSysDomain(domain);
        request.setSysVersion(version);
        request.setSysProtocol(protocolType);
        request.setSysMethod(method);
        request.setSysUriPattern(uri);
        request.setHttpContentType(FormatType.JSON);
        return request;
    }
}
package main

import (
	"encoding/json"
	"fmt"
	"log"
	"os"
	"time"

	"github.com/aliyun/alibaba-cloud-sdk-go/sdk"
	"github.com/aliyun/alibaba-cloud-sdk-go/sdk/requests"
)

type TranscodeingParam struct {
	TargetAudioFormat     string `json:"TargetAudioFormat,omitempty"`
	TargetVideoFormat     string `json:"TargetVideoFormat,omitempty"`
	VideoThumbnailEnabled bool   `json:"VideoThumbnailEnabled,omitempty"`
	SpectrumEnabled       bool   `json:"SpectrumEnabled,omitempty"`
}

type DiarizationParam struct {
	SpeakerCount int `json:"SpeakerCount,omitempty"`
}

type TranscriptionParam struct {
	AudioEventDetectionEnabled bool              `json:"AudioEventDetectionEnabled,omitempty"`
	DiarizationEnabled         bool              `json:"DiarizationEnabled,omitempty"`
	Diarization                *DiarizationParam `json:"Diarization,omitempty"`
}

type TranslationParam struct {
	TargetLanguages []string `json:"TargetLanguages,omitempty"`
}

type SummarizationParam struct {
	Types []string `json:"Types,omitempty"`
}

type ExtraParamerters struct {
	Transcoding              *TranscodeingParam  `json:"Transcoding,omitempty"`
	Transcription            *TranscriptionParam `json:"Transcription,omitempty"`
	TranslationEnabled       bool                `json:"TranslationEnabled,omitempty"`
	Translation              *TranslationParam   `json:"Translation,omitempty"`
	AutoChaptersEnabled      bool                `json:"AutoChaptersEnabled,omitempty"`
	MeetingAssistanceEnabled bool                `json:"MeetingAssistanceEnabled,omitempty"`
	SummarizationEnabled     bool                `json:"SummarizationEnabled,omitempty"`
	Summarization            *SummarizationParam `json:"Summarization,omitempty"`
  TextPolishEnabled        bool                `json:"TextPolishEnabled,omitempty"`
}

type InputParam struct {
	SourceLanguage string `json:"SourceLanguage"`
	FileUrl        string `json:"FileUrl,omitempty"`
	TaskKey        string `json:"TaskKey,omitempty"`
	TaskId         string `json:"TaskId,omitempty"`
	Format         string `json:"Format,omitempty"`
	SampleRate     int    `json:"SampleRate,omitempty"`
}

type TaskBodyParam struct {
	Appkey      string            `json:"AppKey"`
	Input       InputParam        `json:"Input"`
	Paramerters *ExtraParamerters `json:"Parameters,omitempty"`
}

type GetTaskInfoResponse struct {
	RequestId string `json:"RequestId"`
	Code      string `json:"Code"`
	Message   string `json:"Message"`
	Data      struct {
		TaskId     string `json:"TaskId"`
		TaskKey    string `json:"TaskKey"`
		TaskStatus string `json:"TaskStatus"`
	} `json:"Data"`
}

func init_request_param_offline() *ExtraParamerters {
	param := new(ExtraParamerters)
	param.Transcoding = new(TranscodeingParam)

	transcription := new(TranscriptionParam)
	transcription.Diarization = new(DiarizationParam)
	transcription.Diarization.SpeakerCount = 2
	transcription.DiarizationEnabled = true
	param.Transcription = transcription

	// Set translation language
	translation := new(TranslationParam)
	translation.TargetLanguages = []string{"en"}
	param.Translation = translation
	param.TranslationEnabled = true
	param.AutoChaptersEnabled = false
	param.MeetingAssistanceEnabled = false
	param.SummarizationEnabled = true

	summarization := new(SummarizationParam)
	summarization.Types = []string{"Paragraph", "Conversational", "QuestionsAnswering", "MindMap"}
	param.Summarization = summarization
  param.TextPolishEnabled = true

	return param
}

func test_rerun_task() {
	akkey := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_ID")
	aksecret := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET")
	client, err := sdk.NewClientWithAccessKey("cn-beijing", akkey, aksecret)
	if err != nil {
		log.Default().Fatalln(err)
		return
	}

	request := requests.NewCommonRequest()
	request.Method = "PUT"
	request.Domain = "tingwu.cn-beijing.aliyuncs.com"
	request.Version = "2023-09-30"
	request.SetContentType("application/json")
	request.PathPattern = "/openapi/tingwu/v2/tasks"
	request.QueryParams["type"] = "offline"

	param := new(TaskBodyParam)
	param.Appkey = "Enter the AppKey you created in the Tingwu console"
	param.Input.SourceLanguage = "cn"
	param.Input.TaskId = "Enter the TaskId of the completed task"
	param.Input.FileUrl = "https://isv-data.oss-cn-hangzhou.aliyuncs.com/ics/MaaS/ASR/test_audio/asr_example.wav"
	param.Input.TaskKey = "task_" + fmt.Sprint(time.Now().Unix())

	b, _ := json.Marshal(param)
	log.Default().Print("request body:\n", string(b))
	request.SetContent(b)
	request.SetScheme("https")

	response, err := client.ProcessCommonRequest(request)
	if err != nil {
		log.Default().Fatalln(err)
		return
	}

	log.Default().Print("response body:\n", string(response.GetHttpContentBytes()))
}

func main() {
	test_rerun_task()
}

Sample output

After you rerun the task, you can continue to query the task status and result using the same TaskId. The input request is almost identical to the request that you used to submit the task. Pass a JSON-formatted string to the body of the request object. The following code shows a sample JSON message:

{
    "Input":{
        "TaskId":"be602978409b47ecb1788e36********"
    },
    "Parameters":{
        "Translation":{
            "TargetLanguages":[
                "en"
            ]
        },
        "TranslationEnabled":true
    }
}

Protocol parsing

The fields are defined as follows.

Parameter name

Type

Description

TaskId

string

The TaskId generated when the task was created.

TaskStatus

string

A task can have one of the following statuses:

  • ONGOING: The task is running.

  • COMPLETED: The task is completed.

  • FAILED: The task failed.

  • INVALID: The task is invalid, possibly because the TaskId does not exist.

OutputMp3Path

string

If you set the request parameter to convert the source audio or video to MP3 format, this field returns the HTTP download link for the generated MP3 file.

Result

list[]

A collection of multiple algorithm results:

Note: The returned values are download links. You can refer to the corresponding model's documentation to process the results.

ErrorCode

string

The cause of the error.

ErrorMessage

string

The detailed error message.

FAQ

The generated HTTP link is valid. Why do I receive an 'AudioFileLink' error?

  • Results are often returned in minutes, but this is not guaranteed. Offline transcription is an asynchronous process, and tasks may be queued. Therefore, the time limit for offline tasks is 3 hours. If the validity period of the audio HTTP link that you set is less than 3 hours, the link may expire before the task is processed. This makes the file impossible to download.

Do the HTTP links in the result have an expiration time?

  • Yes, they do. The HTTP links within the `Result` parameter are valid for 30 days. After 30 days, you cannot access the task result through these links. Other HTTP links within the `Result` content also have a 30-day validity period. For example, the `Result` of PPT extraction includes an HTTP link to the PPT thumbnail image, which also expires after 30 days.

  • If an HTTP link in the result expires, you can call the task query API again to obtain a new task result with a 30-day validity period.

In which scenarios should I rerun a task?

  • If you know which algorithms to process and which results to obtain for each offline transcription task that you submit, you can set the corresponding feature parameters once at submission. In this case, you do not need to rerun the task.

  • If you are developing a secondary encapsulation based on the Tingwu API, such as your own Software as a Service (SaaS) application that allows users to interactively display different algorithm results, you can use this API to rerun tasks and speed up execution.

What are the differences or benefits of rerunning a task compared to resubmitting an offline transcription task for the same audio or video file?

  • You can achieve the same functionality by resubmitting an offline transcription task with new parameters. However, if you rerun the task, the process does not start from scratch. It reuses some of the algorithm results from the previous run. This shortens the running time and avoids redundant calling costs.

How do I choose the language?

  • The following table provides examples of common language scenarios.

    Language type

    Corresponding parameter

    Scenarios

    Example

    Single language

    (Known language)

    Parameter name: Input.SourceLanguage

    Parameter value (select one): Chinese (cn), English (en), Cantonese (yue), Japanese (ja), Korean (ko)

    • Single-language speech recognition model.

    • Chinese and English support resolutions of 8K and 16K.

    • Japanese, Cantonese, and Korean support only 16 kHz.

    Example 1:

    If the language in the audio is known to be Chinese

    Input.SourceLanguage="cn"

    Recognition result: Thank you for using Tongyi Tingwu.

    Example 2:

    If the language in the audio is known to be English

    Input.SourceLanguage="en"

    Recognition result: Thank you for using Tongyi Tingwu.

    Single language

    (Unknown language)

    Parameter name: Input.SourceLanguage

    Parameter value: auto

    • Automatic single-language recognition.

    • Only 16 kHz is supported.

    • If you do not know the language in the audio, you can set this parameter to `auto` for automatic language detection. The system detects the language and then automatically selects the corresponding single-language model for speech recognition. This feature is available only for offline transcription tasks.

    • The `auto` mode can detect the following languages: Chinese, English, Japanese, Cantonese, and Korean.

    Example 1:

    If the language in the audio is unknown, the system must first detect and determine the language before recognition.

    Input.SourceLanguage="auto"

    Recognition result: Thank you for using Tongyi Tingwu.

    View language detection results.

    Mixed languages

    • Parameter name 1: Input.SourceLanguage

      Parameter value: multilingual

    • Parameter name 2: Input.LanguageHints

      Parameter value (select one or more): Chinese (cn), English (en), Cantonese (yue), Japanese (ja), Korean (ko), German (de), French (fr), Russian (ru)

    • Multilingual speech recognition.

    • Only 16 kHz is supported.

    • Supports simultaneous recognition of Chinese, English, Japanese, Cantonese, Korean, German, French, and Russian.

    • Supports limiting the scope of languages.

    Example 1:

    If the audio contains multiple languages, directly recognize the text in the corresponding languages.

    Input.SourceLanguage="multilingual"

    Recognition result: Hello, everyone. Thank you for using Tongyi Tingwu.

    Example 2:

    If the audio contains multiple languages, but the scope of languages is known (such as only Chinese, English, and Cantonese), directly recognize the text in the corresponding languages while avoiding misidentification of languages irrelevant to the scenario, such as Japanese, French, or Russian.

    Input.SourceLanguage="multilingual"

    Input.LanguageHints=['cn', 'en', 'yue']