API and Implementation

更新时间:
复制 MD 格式

This topic describes the access process for real-time recording.

Notes

  • Supported input formats: PCM (uncompressed PCM or WAV files), OPUS, SPEEX, MP3, and AAC, with 16-bit audio bit depth and mono.

  • Supported audio sampling rates: 16000 Hz and 8000 Hz.

  • Maximum single recording duration: 24 hours.

  • Supports configuring whether to return intermediate recognition results.

  • Supports multilingual recognition: Chinese, English, Cantonese, Japanese, Korean, and free-form Chinese-English (long mixed segments).

  • Supports enabling translation: bidirectional translation between Chinese, English, Japanese, Korean, German, French, and Russian. For free-form Chinese-English, the target translation languages can be Chinese, English, or both. You can enable or disable this feature during meetings.

  • Supports synchronous audio transcoding: transcodes audio to MP3 format at 128 kb/s and writes it to the specified OSS in Near Real-Time.

  • Supports various Large Language Model (LLM) capabilities: Chapter Overview, LLM Summary Generation (Full Text Summary, Speaker Summary, Q&A Review, Mind Map), Key Point Extraction (Keywords, To-Do Items, Key Content, Scenario Recognition), Service Quality Inspection, PPT Extraction and Summary, Spoken Language Formalization, and custom prompt.

  • API-level Queries Per Second (QPS) limits are as follows:

Interaction Flow

image

Access Steps

Step 1: Call the CreateTask API to create a real-time recording, corresponding to interaction flow steps 1–2.

Note

Transcription is managed at the recording level. After creating a recording, you can perform real-time recording audio stream ingest transcription during the meeting based on this recording.

Step 2: Perform real-time recording audio stream ingest. After creating a recording, use the interaction flow and implementation provided by Tingwu to ingest audio in real time during the meeting and receive recognition and translation results.

Step 3: Call the CreateTask API to end the real-time recording.

Step 4: Call the GetTaskInfo API to query the real-time recording status.

Note

After ending the real-time recording, the system enters the AI processing stage, such as intelligent meeting minute extraction. At this point, you can query or retrieve results via callback.

Prerequisites

AccessKey Environment Variable Settings

Replace YOUR_ACCESS_KEY_ID and YOUR_ACCESS_KEY_SECRET in the following command with your AccessKey ID and secret.

export ALIBABA_CLOUD_ACCESS_KEY_ID=YOUR_ACCESS_KEY_ID &&
export ALIBABA_CLOUD_ACCESS_KEY_SECRET=YOUR_ACCESS_KEY_SECRET

Create a Real-time Recording

Refer to API details to create a real-time recording task. Common parameters are listed below. Then, use the obtained recording stream ingest URL to establish a WebSocket connection, perform real-time stream ingest, and recognize speech.

Request Parameters

Name

Parameter

Type

Default Value

Description

AppKey

AppKey

string

-

Required. Set this to the AppKey you created in the console.

Basic Request Information

(body.Input)

Input.Format

string

-

Required. The audio format for real-time binary audio stream ingest. Supports pcm, opus, aac, speex, and mp3.

Input.SampleRate

int

-

Required. The sample rate for real-time binary audio stream ingest. Supports 16000 and 8000. Most recording scenarios typically use 16000.

Input.SourceLanguage

string

-

Required. Configure the corresponding parameters based on the language category in the audio and video.

  • If the speech is a single language, the language type corresponds to the audio and video file. Supports cn (Chinese), en (English), yue (Cantonese), ja (Japanese), and ko (Korean).

  • If the speech contains multiple languages, pass in `multilingual` to recognize text in the corresponding languages. Use this in conjunction with Input.LanguageHints.

Input.LanguageHints

list[string]

null

Specify the list of languages for speech recognition when using a multilingual model.

When speech contains multiple languages that all require text recognition, this parameter limits the language categories. This configuration takes effect only when Input.SourceLanguage is set to 'multilingual'.

Input.TaskKey

string

null

A custom ID set by the user.

Input.ProgressiveCallbacksEnabled

boolean

false

Enable callback feature.

To enable the callback feature, configure the callback type and address in the console, and set this parameter to `true` when creating the task.

Transcoding

(Transcoding)

Parameters.Transcoding.TargetAudioFormat

string

null

Currently, only MP3 conversion is supported. It is disabled by default.

Speech Recognition

(Transcription)

Parameters.Transcription.OutputLevel

int

1

Set the return level for speech recognition results.

  • 1: Return recognition results when a complete sentence is recognized.

  • 2: Return recognition results when intermediate results and complete sentences are recognized.

Set this as needed only in real-time recording scenarios. No need to set it in offline transcription scenarios.

Parameters.Transcription.DiarizationEnabled

boolean

false

Enable speaker diarization during speech recognition.

Parameters.Transcription.Diarization.SpeakerCount

int

-

When speaker diarization is enabled, set the number of speakers. Supports up to 100 speakers.

  • Not set: Do not distinguish speaker roles.

  • 0: Speaker role distinction results in an indefinite number of speakers.

If this parameter is present, use it to assist in specifying the number of speakers. The final number of separated speakers is based on the actual classification.

Parameters.Transcription.PhraseId

String

-

Hotword list ID.

Translation

(Translation)

Parameters.TranslationEnabled

boolean

false

Enable translation feature.

Parameters.Translation.OutputLevel

int

1

Set the return level for real-time language translation recognition results.

  • 1: Return recognition results when a complete sentence is recognized.

  • 2: Return recognition results when intermediate results and complete sentences are recognized.

Set this as needed only in real-time recording scenarios. No need to set it in offline transcription scenarios.

Parameters.Translation.TargetLanguages

list[string]

-

If translation is enabled, set the target translation languages. Supports Chinese (cn), English (en), Japanese (ja), Korean (ko), German (de), French (fr), and Russian (ru).

Chapter Overview

Parameters.AutoChaptersEnabled

boolean

false

Chapter Overview feature, including agenda titles and agenda summaries.

Key Point Extraction

Parameters.MeetingAssistanceEnabled

boolean

false

Keywords, To-Do Items, Key Content, Scenario Recognition.

Summary

Parameters.SummarizationEnabled

boolean

false

Enable summary generation feature.

Parameters.Summarization.Types

list[string]

-

If summary generation is enabled, set the summary types. Supports setting one or more.

  • Paragraph (Full Text Summary)

  • Conversational (Speaker Summary)

  • QuestionsAnswering (Q&A Review)

  • MindMap (Mind Map)

Spoken Language Formalization

Parameters.TextPolishEnabled

boolean

false

Enable spoken language formalization feature.

Service Quality Inspection

Parameters.ServiceInspectionEnabled

boolean

false

Enable service quality inspection feature.

Parameters.ServiceInspection

object

-

Refer to Service Quality Inspection parameter object to perform quality inspection on conversations during the service process. Supports customizing multiple inspection dimensions to help customers improve service levels.

Custom Prompt

Parameters.CustomPromptEnabled

boolean

false

Enable custom prompt feature.

Parameters.CustomPrompt

object

-

Refer to Custom Prompt parameter object for customers to define Large Language Model (LLM) prompts to guide the LLM in completing various customer-defined tasks.

Code Example

package com.alibaba.tingwu.client.demo.realtimemeeting;

import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import com.aliyuncs.CommonRequest;
import com.aliyuncs.CommonResponse;
import com.aliyuncs.DefaultAcsClient;
import com.aliyuncs.IAcsClient;
import com.aliyuncs.exceptions.ClientException;
import com.aliyuncs.http.FormatType;
import com.aliyuncs.http.MethodType;
import com.aliyuncs.http.ProtocolType;
import com.aliyuncs.profile.DefaultProfile;
import org.junit.Test;

/**
 * @author tingwu2023
 * @desc Demonstrates how to call the OpenAPI to create a real-time recording.
 */
public class SubmitRealtimeMeetingTaskTest {

    @Test
    public void testSummitRealtimeMeetingTask() throws ClientException {
        CommonRequest request = createCommonRequest("tingwu.cn-beijing.aliyuncs.com", "2023-09-30", ProtocolType.HTTPS, MethodType.PUT, "/openapi/tingwu/v2/tasks");
        request.putQueryParameter("type", "realtime");

        JSONObject root = new JSONObject();
        root.put("AppKey", "Enter the AppKey you created in the Tingwu console");

        JSONObject input = new JSONObject();
        input.fluentPut("SourceLanguage", "cn").fluentPut("Format", "pcm").fluentPut("SampleRate", 16000).fluentPut("TaskKey", "task" + System.currentTimeMillis());
        root.put("Input", input);

        JSONObject parameters = initRequestParameters();
        root.put("Parameters", parameters);

        System.out.println(root.toJSONString());
        request.setHttpContent(root.toJSONString().getBytes(), "utf-8", FormatType.JSON);

        // TODO Set your AccessKeyId and AccessKeySecret through environment variables
        DefaultProfile profile = DefaultProfile.getProfile("cn-beijing", System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
        IAcsClient client = new DefaultAcsClient(profile);
        CommonResponse response = client.getCommonResponse(request);
        System.out.println(response.getData());
        JSONObject body = JSONObject.parseObject(response.getData());
        JSONObject data = (JSONObject) body.get("Data");
        System.out.println("TaskId = " + data.getString("TaskId"));
        System.out.println("MeetingJoinUrl = " + data.getString("MeetingJoinUrl"));
    }

    private static JSONObject initRequestParameters() {
        JSONObject parameters = new JSONObject();

        // Audio and video conversion: Optional
        JSONObject transcoding = new JSONObject();
        //transcoding.put("TargetAudioFormat", "mp3");
        //transcoding.put("SpectrumEnabled", false);
        parameters.put("Transcoding", transcoding);

        // Speech recognition
        JSONObject transcription = new JSONObject();
        transcription.put("DiarizationEnabled", true);
        JSONObject speakerCount = new JSONObject();
        speakerCount.put("SpeakerCount", 2);
        transcription.put("Diarization", speakerCount);
        parameters.put("Transcription", transcription);

        // Translation: Optional
        JSONObject translation = new JSONObject();
        JSONArray langArry = new JSONArray();
        langArry.add("en");
        translation.put("TargetLanguages", langArry);
        parameters.put("Translation", translation);
        parameters.put("TranslationEnabled", true);

        // Chapter Overview: Optional
        parameters.put("AutoChaptersEnabled", false);

        // Intelligent Meeting Minutes: Optional
        parameters.put("MeetingAssistanceEnabled", false);

        // Summary related: Optional
        parameters.put("SummarizationEnabled", true);
        JSONObject summarization = new JSONObject();
        JSONArray types = new JSONArray().fluentAdd("Paragraph").fluentAdd("Conversational").fluentAdd("QuestionsAnswering").fluentAdd("MindMap");
        summarization.put("Types", types);
        parameters.put("Summarization", summarization);

        // PPT extraction: Not optional, real-time recording has no video information, so PPT functionality is not available
        // parameters.put("PptExtractionEnabled", false);
      	
	// Spoken language formalization: Optional
        parameters.put("TextPolishEnabled", false);

        return parameters;
    }
    public static CommonRequest createCommonRequest(String domain, String version, ProtocolType protocolType, MethodType method, String uri) {
        CommonRequest request = new CommonRequest();
        request.setSysDomain(domain);
        request.setSysVersion(version);
        request.setSysProtocol(protocolType);
        request.setSysMethod(method);
        request.setSysUriPattern(uri);
        request.setHttpContentType(FormatType.JSON);
        return request;
    }
}
package main

import (
	"encoding/json"
	"fmt"
	"log"
	"os"
	"time"

	"github.com/aliyun/alibaba-cloud-sdk-go/sdk"
	"github.com/aliyun/alibaba-cloud-sdk-go/sdk/requests"
)

type TranscodeingParam struct {
	TargetAudioFormat     string `json:"TargetAudioFormat,omitempty"`
	TargetVideoFormat     string `json:"TargetVideoFormat,omitempty"`
	VideoThumbnailEnabled bool   `json:"VideoThumbnailEnabled,omitempty"`
	SpectrumEnabled       bool   `json:"SpectrumEnabled,omitempty"`
}

type DiarizationParam struct {
	SpeakerCount int `json:"SpeakerCount"`
}

type TranscriptionParam struct {
	AudioEventDetectionEnabled bool              `json:"AudioEventDetectionEnabled,omitempty"`
	DiarizationEnabled         bool              `json:"DiarizationEnabled,omitempty"`
	Diarization                *DiarizationParam `json:"Diarization,omitempty"`
}

type TranslationParam struct {
	TargetLanguages []string `json:"TargetLanguages,omitempty"`
}

type SummarizationParam struct {
	Types []string `json:"Types,omitempty"`
}

type ExtraParamerters struct {
	Transcoding              *TranscodeingParam  `json:"Transcoding,omitempty"`
	Transcription            *TranscriptionParam `json:"Transcription,omitempty"`
	TranslationEnabled       bool                `json:"TranslationEnabled,omitempty"`
	Translation              *TranslationParam   `json:"Translation,omitempty"`
	AutoChaptersEnabled      bool                `json:"AutoChaptersEnabled,omitempty"`
	MeetingAssistanceEnabled bool                `json:"MeetingAssistanceEnabled,omitempty"`
	SummarizationEnabled     bool                `json:"SummarizationEnabled,omitempty"`
	Summarization            *SummarizationParam `json:"Summarization,omitempty"`
  TextPolishEnabled        bool                `json:"TextPolishEnabled,omitempty"`
}

type InputParam struct {
	SourceLanguage string `json:"SourceLanguage"`
	FileUrl        string `json:"FileUrl,omitempty"`
	TaskKey        string `json:"TaskKey,omitempty"`
	Format         string `json:"Format,omitempty"`
	SampleRate     int    `json:"SampleRate,omitempty"`
}

type TaskBodyParam struct {
	Appkey      string            `json:"AppKey"`
	Input       InputParam        `json:"Input"`
	Paramerters *ExtraParamerters `json:"Parameters,omitempty"`
}

type CreateTaskResponse struct {
	RequestId string `json:"RequestId"`
	Code      string `json:"Code"`
	Message   string `json:"Message"`
	Data      struct {
		TaskId         string `json:"TaskId"`
		TaskKey        string `json:"TaskKey"`
		MeetingJoinUrl string `json:"MeetingJoinUrl,omitempty"`
	} `json:"Data"`
}

func init_request_param() *ExtraParamerters {
	param := new(ExtraParamerters)
	param.Transcoding = new(TranscodeingParam)

	transcription := new(TranscriptionParam)
	transcription.Diarization = new(DiarizationParam)
	transcription.Diarization.SpeakerCount = 2
	transcription.DiarizationEnabled = true
	param.Transcription = transcription

	translation := new(TranslationParam)
	translation.TargetLanguages = []string{"en"}
	param.Translation = translation
	param.TranslationEnabled = true
	param.AutoChaptersEnabled = false
	param.MeetingAssistanceEnabled = false
	param.SummarizationEnabled = true

	summarization := new(SummarizationParam)
	summarization.Types = []string{"Paragraph", "Conversational", "QuestionsAnswering", "MindMap"}
	param.Summarization = summarization
  param.TextPolishEnabled = false

	return param
}

func test_submit_realtime_meeting_task() string {
	akkey := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_ID")
	aksecret := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET")
	client, err := sdk.NewClientWithAccessKey("cn-beijing", akkey, aksecret)
	if err != nil {
		log.Default().Fatalln(err)
		return ""
	}

	request := requests.NewCommonRequest()
	request.Method = "PUT"
	request.Domain = "tingwu.cn-beijing.aliyuncs.com"
	request.Version = "2023-09-30"
	request.SetContentType("application/json")
	request.PathPattern = "/openapi/tingwu/v2/tasks"
	request.QueryParams["type"] = "realtime"

	param := new(TaskBodyParam)
	param.Appkey = "Enter the AppKey you created in the Tingwu console"
	param.Input.SourceLanguage = "cn"
	param.Input.Format = "pcm"
	param.Input.SampleRate = 16000
	param.Input.TaskKey = "task_" + fmt.Sprint(time.Now().Unix())
	param.Paramerters = init_request_param()

	b, _ := json.Marshal(param)
	log.Default().Print("request body:\n", string(b))
	request.SetContent(b)
	request.SetScheme("https")

	response, err := client.ProcessCommonRequest(request)
	if err != nil {
		log.Default().Fatalln(err)
		return ""
	}

	log.Default().Print("response body:\n", string(response.GetHttpContentBytes()))

	var resp CreateTaskResponse
	err = json.Unmarshal(response.GetHttpContentBytes(), &resp)
	if err != nil {
		log.Default().Fatalln(err)
		return ""
	}

	log.Default().Println("TaskId:", resp.Data.TaskId)
	log.Default().Println("MeetingJoinUrl:", resp.Data.MeetingJoinUrl)

	return resp.Data.MeetingJoinUrl
}

func main() {
	test_submit_realtime_meeting_task()
}
#include <cstdlib>
#include <iostream>
#include <string>
#include <alibabacloud/core/AlibabaCloud.h>
#include <alibabacloud/core/CommonRequest.h>
#include <alibabacloud/core/CommonClient.h>
#include <alibabacloud/core/CommonResponse.h>
#include "jsoncpp/json.h"
/**
 * @author tingwu2023
 * @desc Demonstrates how to call the OpenAPI to create a real-time transcription recording.
 */
int main( int argc, char** argv ) {
    AlibabaCloud::InitializeSdk();
    AlibabaCloud::ClientConfiguration configuration( "cn-beijing" );
    // specify timeout when create client.
    configuration.setConnectTimeout(1500);
    configuration.setReadTimeout(4000);
    // Please ensure that the environment variables ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET are set.
    AlibabaCloud::Credentials credential( getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET") );
    AlibabaCloud::CommonClient client( credential, configuration );
    AlibabaCloud::CommonRequest request(AlibabaCloud::CommonRequest::RequestPattern::RoaPattern);
    request.setHttpMethod(AlibabaCloud::HttpRequest::Method::Put);
    request.setDomain("tingwu.cn-beijing.aliyuncs.com");
    request.setVersion("2023-09-30");
    request.setResourcePath("/openapi/tingwu/v2/tasks");
    request.setQueryParameter("type", "realtime");
    request.setHeaderParameter("Content-Type", "application/json");
    
    Json::Value root;
    root["Appkey"] = "Enter the AppKey you created in the Tingwu console";
    root["Appkey"] = "tEA3SDcxTKire0ut";

    Json::Value input;
    input["SourceLanguage"] = "cn";
    input["Format"]         = "pcm";
    input["SampleRate"]     = "16000";
    input["SourceLanguage"] = "cn";
    input["TaskKey"] = "Enter the custom ID for this request";
    root["Input"] = input;

    Json::Value parameters;
    // Audio and video file conversion: Generally, you do not need to set this parameter.
    // Json::Value transcoding;
    // transcoding["TargetAudioFormat"] = "mp3";
    // parameters["Transcoding"] = transcoding;

    // Speech transcription: The following enables speaker diarization (role separation). If you do not need it, do not set it.
    Json::Value transcription;
    transcription["DiarizationEnabled"] = true;
    Json::Value speakerCount;
    speakerCount["SpeakerCount"] = 2;
    transcription["Diarization"] = speakerCount;
    parameters["Transcription"] = transcription;

    // Translation: Optional
    Json::Value translation;
    Json::Value langauges;
    langauges.append("en");
    translation["TargetLanguages"] = langauges;
    parameters["Translation"] = translation;
    parameters["TranslationEnabled"] = true;

    // Chapter Overview: Optional
    parameters["AutoChaptersEnabled"] = true;

    // Intelligent Meeting Minutes: Optional, including the following: To-Do, Key Information (Keywords, Key Content, Scenario Recognition)
    parameters["MeetingAssistanceEnabled"] = true;
    Json::Value meetingAssistance;
    Json::Value meetingAssistanceTypeList;
    meetingAssistanceTypeList.append("Actions");
    meetingAssistanceTypeList.append("KeyInformation");
    meetingAssistance["Types"] = meetingAssistanceTypeList;
    parameters["MeetingAssistanceTypeList"] = meetingAssistanceTypeList;

    // Summary related: Optional. The following settings enable all three summary types. You can add or remove them as needed.
    parameters["SummarizationEnabled"] = true;
    Json::Value summarization;
    Json::Value summarizationTypeList;
    summarizationTypeList.append("Paragraph");
    summarizationTypeList.append("Conversational");
    summarizationTypeList.append("QuestionsAnswering");
    summarizationTypeList.append("MindMap");
    summarization["Types"] = summarizationTypeList;
    parameters["Summarization"] = summarization;
  	
    // Spoken language formalization: Optional
    parameters["TextPolishEnabled"] = true;

    root["Parameters"] = parameters;

    Json::FastWriter writer;
    std::string body = writer.write(root);
    printf("input json: [%s]\n", body.c_str());
    request.setContent(body.c_str(), body.size());

    auto response = client.commonResponse(request);
    if (response.isSuccess()) {
        printf("request success.\n");
        printf("result: %s\n", response.result().payload().c_str());
    } else {
        printf("error: %s\n", response.error().errorMessage().c_str());
        printf("request id: %s\n", response.error().requestId().c_str());
    }
      
    AlibabaCloud::ShutdownSdk();
    return 0;
}
#!/usr/bin/env python
#coding=utf-8

import os
import json
import datetime
from aliyunsdkcore.client import AcsClient
from aliyunsdkcore.request import CommonRequest
from aliyunsdkcore.auth.credentials import AccessKeyCredential

def create_common_request(domain, version, protocolType, method, uri):
    request = CommonRequest()
    request.set_accept_format('json')
    request.set_domain(domain)
    request.set_version(version)
    request.set_protocol_type(protocolType)
    request.set_method(method)
    request.set_uri_pattern(uri)
    request.add_header('Content-Type', 'application/json')
    return request

def init_parameters():
    body = dict()
    body['AppKey'] = 'Enter the AppKey you created in the Tingwu console'

    # Basic request parameters
    input = dict()

    # Ensure the audio stream format and sample rate are consistent with the following parameter settings
    input['Format'] = 'pcm'
    input['SampleRate'] = 16000
    input['SourceLanguage'] = 'cn'
    input['TaskKey'] = 'task' + datetime.datetime.now().strftime('%Y%m%d%H%M%S')
    input['ProgressiveCallbacksEnabled'] = False
    body['Input'] = input

    # AI-related parameters, set as needed
    parameters = dict()

    # Audio and video conversion related
    transcoding = dict()
    # Convert the original audio and video file to an MP3 file for subsequent browser playback
    # transcoding['TargetAudioFormat'] = 'mp3'
    # transcoding['SpectrumEnabled'] = False
    # parameters['Transcoding'] = transcoding

    # Speech recognition control related
    transcription = dict()
    # Speaker diarization: Optional
    transcription['DiarizationEnabled'] = True
    diarization = dict()
    diarization['SpeakerCount'] = 2
    transcription['Diarization'] = diarization
    parameters['Transcription'] = transcription

    # Text translation control related: Optional
    parameters['TranslationEnabled'] = True
    translation = dict()
    translation['TargetLanguages'] = ['en'] # Assume translation to English
    parameters['Translation'] = translation

    # Chapter Overview related: Optional, including the following: titles, agenda summaries
    parameters['AutoChaptersEnabled'] = True

    # Intelligent Meeting Minutes related: Optional, including the following: To-Do, Key Information (Keywords, Key Content, Scenario Recognition)
    parameters['MeetingAssistanceEnabled'] = True
    meetingAssistance = dict()
    meetingAssistance['Types'] = ['Actions', 'KeyInformation']
    parameters['MeetingAssistance'] = meetingAssistance

    # Summary control related: Optional, including the following: full text summary, speaker summary, Q&A summary (Q&A review)
    parameters['SummarizationEnabled'] = True
    summarization = dict()
    summarization['Types'] = ['Paragraph', 'Conversational', 'QuestionsAnswering', 'MindMap']
    parameters['Summarization'] = summarization

    # PPT extraction and PPT summary: Optional
    parameters['PptExtractionEnabled'] = True
    
    # Spoken language formalization: Optional
    parameters['TextPolishEnabled'] = True

    body['Parameters'] = parameters
    return body

body = init_parameters()
print(body)

# TODO Set your AccessKeyId and AccessKeySecret through environment variables
credentials = AccessKeyCredential(os.environ['ALIBABA_CLOUD_ACCESS_KEY_ID'], os.environ['ALIBABA_CLOUD_ACCESS_KEY_SECRET'])
client = AcsClient(region_id='cn-beijing', credential=credentials)

request = create_common_request('tingwu.cn-beijing.aliyuncs.com', '2023-09-30', 'https', 'PUT', '/openapi/tingwu/v2/tasks')
request.add_query_param('type', 'realtime')

request.set_content(json.dumps(body).encode('utf-8'))
response = client.do_action_with_exception(request)
print("response: \n" + json.dumps(json.loads(response), indent=4, ensure_ascii=False))

Example Output

{
    "Code":"0",
    "Data":{
        "TaskId":"3190978427bb43z09c01dfff********",
        "TaskKey":"task16988********",
        "MeetingJoinUrl":"wss://tingwu-realtime-cn-beijing.aliyuncs.com/api/ws/v1?mc=g9ySw5kiwXM4K7tBnIajKq6Fh9G1aUokzkptBIFixj7e7zv6c8AKxUDTW2Oz8AFFONWXtTQedh-NpKZUffqIYdW7yAlivqlo9B0TdeM88fzgWaYk2Ifg********"
    },
    "Message":"success",
    "RequestId":"6582c654-cc37-4f2d-b80d-e5e7********"
}

Protocol Parsing

Specific field definitions are as follows.

Parameter Name

Type

Description

TaskId

string

The TaskId generated when creating the task, used to query task status, results, and troubleshoot issues.

TaskKey

string

The TaskKey you set when creating the task.

MeetingJoinUrl

string

The audio stream ingest address generated in real-time recording scenarios. Use this address for subsequent real-time audio stream recognition.

RequestId

string

The RequestId is used for troubleshooting.

Real-time Recording Audio Stream Ingest

After creating a recording, use the interaction flow and implementation provided by Tingwu to perform real-time audio stream ingest during the meeting and receive real-time recognition and translation results.

End Real-time Recording

When the recording ends, you must refer to the following content to promptly call the API to end the recording. If you previously enabled features such as summary generation, chapter overview, or intelligent meeting minutes when creating the real-time recording, the recording status will be ONGOING—not COMPLETED—after ending the recording. This indicates that the system has entered the post-processing stage.

Note

Ending the real-time recording triggers the post-processing stage, which incurs billing for AI model capabilities used in post-processing. Repeatedly ending the same recording will trigger multiple Large Language Model (LLM) capability billings. Avoid performing the "end real-time recording" operation multiple times for the same TaskId.

Request Parameters

Name

Parameter

Type

Default Value

Description

TaskId

TaskId

string

-

Required. The TaskId information returned when you submit the task.

Code Example

package com.alibaba.tingwu.client.demo.realtimemeeting;

import com.alibaba.fastjson.JSONObject;
import com.aliyuncs.CommonRequest;
import com.aliyuncs.CommonResponse;
import com.aliyuncs.DefaultAcsClient;
import com.aliyuncs.IAcsClient;
import com.aliyuncs.exceptions.ClientException;
import com.aliyuncs.http.FormatType;
import com.aliyuncs.http.MethodType;
import com.aliyuncs.http.ProtocolType;
import com.aliyuncs.profile.DefaultProfile;
import org.junit.Test;

/**
 * @author tingwu2023
 * @desc Demonstrates how to call the OpenAPI to end a real-time meeting.
 */
public class StopRealtimeMeetingTaskTest {

    @Test
    public void stopTask() throws ClientException {
        CommonRequest request = createCommonRequest("tingwu.cn-beijing.aliyuncs.com", "2023-09-30", ProtocolType.HTTPS, MethodType.PUT, "/openapi/tingwu/v2/tasks");
        request.putQueryParameter("type", "realtime");
        // Must set operation=stop
        request.putQueryParameter("operation", "stop");

        JSONObject root = new JSONObject();
        JSONObject input = new JSONObject();
        input.put("TaskId", "Enter the TaskId of the real-time meeting");
        root.put("Input", input);
        System.out.println(root.toJSONString());
        request.setHttpContent(root.toJSONString().getBytes(), "utf-8", FormatType.JSON);

        // TODO Set your AccessKeyId and AccessKeySecret through environment variables
        DefaultProfile profile = DefaultProfile.getProfile("cn-beijing", System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
        IAcsClient client = new DefaultAcsClient(profile);
        CommonResponse response = client.getCommonResponse(request);
        System.out.println(response.getData());
    }
  
    public static CommonRequest createCommonRequest(String domain, String version, ProtocolType protocolType, MethodType method, String uri) {
        CommonRequest request = new CommonRequest();
        request.setSysDomain(domain);
        request.setSysVersion(version);
        request.setSysProtocol(protocolType);
        request.setSysMethod(method);
        request.setSysUriPattern(uri);
        request.setHttpContentType(FormatType.JSON);
        return request;
    }
}
package main

import (
	"encoding/json"
	"log"
	"os"

	"github.com/aliyun/alibaba-cloud-sdk-go/sdk"
	"github.com/aliyun/alibaba-cloud-sdk-go/sdk/requests"
)

type TranscodeingParam struct {
	TargetAudioFormat     string `json:"TargetAudioFormat,omitempty"`
	TargetVideoFormat     string `json:"TargetVideoFormat,omitempty"`
	VideoThumbnailEnabled bool   `json:"VideoThumbnailEnabled,omitempty"`
	SpectrumEnabled       bool   `json:"SpectrumEnabled,omitempty"`
}

type DiarizationParam struct {
	SpeakerCount int `json:"SpeakerCount,omitempty"`
}

type TranscriptionParam struct {
	AudioEventDetectionEnabled bool              `json:"AudioEventDetectionEnabled,omitempty"`
	DiarizationEnabled         bool              `json:"DiarizationEnabled,omitempty"`
	Diarization                *DiarizationParam `json:"Diarization,omitempty"`
}

type TranslationParam struct {
	TargetLanguages []string `json:"TargetLanguages,omitempty"`
}

type SummarizationParam struct {
	Types []string `json:"Types,omitempty"`
}

type ExtraParamerters struct {
	Transcoding              *TranscodeingParam  `json:"Transcoding,omitempty"`
	Transcription            *TranscriptionParam `json:"Transcription,omitempty"`
	TranslationEnabled       bool                `json:"TranslationEnabled,omitempty"`
	Translation              *TranslationParam   `json:"Translation,omitempty"`
	AutoChaptersEnabled      bool                `json:"AutoChaptersEnabled,omitempty"`
	MeetingAssistanceEnabled bool                `json:"MeetingAssistanceEnabled,omitempty"`
	SummarizationEnabled     bool                `json:"SummarizationEnabled,omitempty"`
	Summarization            *SummarizationParam `json:"Summarization,omitempty"`
}

type InputParam struct {
	SourceLanguage string `json:"SourceLanguage"`
	FileUrl        string `json:"FileUrl,omitempty"`
	TaskKey        string `json:"TaskKey,omitempty"`
	TaskId         string `json:"TaskId,omitempty"`
	Format         string `json:"Format,omitempty"`
	SampleRate     int    `json:"SampleRate,omitempty"`
}

type TaskBodyParam struct {
	Appkey      string            `json:"AppKey"`
	Input       InputParam        `json:"Input"`
	Paramerters *ExtraParamerters `json:"Parameters,omitempty"`
}

type CreateTaskResponse struct {
	RequestId string `json:"RequestId"`
	Code      string `json:"Code"`
	Message   string `json:"Message"`
	Data      struct {
		TaskId         string `json:"TaskId"`
		TaskKey        string `json:"TaskKey"`
		MeetingJoinUrl string `json:"MeetingJoinUrl,omitempty"`
	} `json:"Data"`
}

func test_stop_realtime_meeting_task() {
	akkey := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_ID")
	aksecret := os.Getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET")
	client, err := sdk.NewClientWithAccessKey("cn-beijing", akkey, aksecret)
	if err != nil {
		log.Default().Fatalln(err)
		return
	}

	request := requests.NewCommonRequest()
	request.Method = "PUT"
	request.Domain = "tingwu.cn-beijing.aliyuncs.com"
	request.Version = "2023-09-30"
	request.SetContentType("application/json")
	request.PathPattern = "/openapi/tingwu/v2/tasks"
	request.QueryParams["type"] = "realtime"
	request.QueryParams["operation"] = "stop"

	param := new(TaskBodyParam)
	param.Appkey = "Enter the AppKey you created in the Tingwu console"
	param.Input.TaskId = "Enter the TaskId of the real-time meeting"

	b, _ := json.Marshal(param)
	log.Default().Print("request body:\n", string(b))
	request.SetContent(b)
	request.SetScheme("https")

	response, err := client.ProcessCommonRequest(request)
	if err != nil {
		log.Default().Fatalln(err)
		return
	}

	log.Default().Print("response body:\n", string(response.GetHttpContentBytes()))
	return
}

func main() {
	test_stop_realtime_meeting_task()
}
#include <cstdlib>
#include <iostream>
#include <string>
#include <string.h>
#include <alibabacloud/core/AlibabaCloud.h>
#include <alibabacloud/core/CommonRequest.h>
#include <alibabacloud/core/CommonClient.h>
#include <alibabacloud/core/CommonResponse.h>
#include "jsoncpp/json.h"
/**
 * @author tingwu2023
 * @desc Demonstrates how to call the OpenAPI to end a real-time recording.
 */
int main( int argc, char** argv ) {
    std::string taskId = "Enter the TaskId of the created task (including offline transcription and real-time meeting)";

    AlibabaCloud::InitializeSdk();
    AlibabaCloud::ClientConfiguration configuration( "cn-beijing" );
    // specify timeout when create client.
    configuration.setConnectTimeout(1500);
    configuration.setReadTimeout(4000);
    // Please ensure that the environment variables ALIBABA_CLOUD_ACCESS_KEY_ID and ALIBABA_CLOUD_ACCESS_KEY_SECRET are set.
    AlibabaCloud::Credentials credential( getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET") );
    AlibabaCloud::CommonClient client( credential, configuration );
    AlibabaCloud::CommonRequest request(AlibabaCloud::CommonRequest::RequestPattern::RoaPattern);
    request.setHttpMethod(AlibabaCloud::HttpRequest::Method::Put);
    request.setDomain("tingwu.cn-beijing.aliyuncs.com");
    request.setVersion("2023-09-30");
    request.setResourcePath("/openapi/tingwu/v2/tasks");
    request.setQueryParameter("type", "realtime");
    request.setQueryParameter("operation", "stop");
    request.setHeaderParameter("Content-Type", "application/json");

    Json::Value root;
    Json::Value input;
    input["TaskId"] = taskId;
    root["Input"] = input;

    Json::FastWriter writer;
    std::string body = writer.write(root);
    printf("input json: [%s]\n", body.c_str());
    request.setContent(body.c_str(), body.size());

    auto response = client.commonResponse(request);
    if (response.isSuccess()) {
        printf("request success.\n");
        printf("result: %s\n", response.result().payload().c_str());
    } else {
        printf("error: %s\n", response.error().errorMessage().c_str());
        printf("request id: %s\n", response.error().requestId().c_str());
    }
      
    AlibabaCloud::ShutdownSdk();
    return 0;
}
#!/usr/bin/env python
#coding=utf-8

import os
import json
import datetime
from aliyunsdkcore.client import AcsClient
from aliyunsdkcore.request import CommonRequest
from aliyunsdkcore.auth.credentials import AccessKeyCredential

def create_common_request(domain, version, protocolType, method, uri):
    request = CommonRequest()
    request.set_accept_format('json')
    request.set_domain(domain)
    request.set_version(version)
    request.set_protocol_type(protocolType)
    request.set_method(method)
    request.set_uri_pattern(uri)
    request.add_header('Content-Type', 'application/json')
    return request

def init_parameters():
    body = dict()
    body['AppKey'] = 'Enter the AppKey you created in the Tingwu console'

    # Basic request parameters
    input = dict()

    # Ensure the audio stream format and sample rate are consistent with the following parameter settings
    input['TaskId'] = 'Enter the TaskId of the real-time meeting'
    body['Input'] = input

    return body

body = init_parameters()
print(body)

# TODO Set your AccessKeyId and AccessKeySecret through environment variables
credentials = AccessKeyCredential(os.environ['ALIBABA_CLOUD_ACCESS_KEY_ID'], os.environ['ALIBABA_CLOUD_ACCESS_KEY_SECRET'])
client = AcsClient(region_id='cn-beijing', credential=credentials)

request = create_common_request('tingwu.cn-beijing.aliyuncs.com', '2023-09-30', 'https', 'PUT', '/openapi/tingwu/v2/tasks')
request.add_query_param('type', 'realtime')
request.add_query_param('operation', 'stop')

request.set_content(json.dumps(body).encode('utf-8'))
response = client.do_action_with_exception(request)
print("response: \n" + json.dumps(json.loads(response), indent=4, ensure_ascii=False))

Example Output

When only real-time speech recognition is used:

{
    "Code":"0",
    "Data":{
        "TaskId":"e8adc0b3bc4b42d898fcadb0a*******",
        "TaskStatus":"COMPLETED"
    },
    "Message":"success",
    "RequestId":"1b20e0d9-c55c-4cc3-85af-80b********"
}

When speech recognition is enabled along with features such as summary generation, chapter overview, and intelligent meeting minutes:

{
    "Code":"0",
    "Data":{
        "TaskId":"e8adc0b3bc4b42d898fcadb0*******",
        "TaskStatus":"ONGOING"
    },
    "Message":"success",
    "RequestId":"5fa32fc6-441f-4dd1-bb86-c030*******"
}

Protocol Parsing

Specific field definitions are as follows.

Parameter Name

Type

Description

TaskId

string

The TaskId generated when creating the task.

TaskStatus

string

Task status, including the following:

  • NEW: Newly created status, no audio stream has been sent yet.

  • ONGOING: Running

  • PAUSED: Real-time recording paused. The status is PAUSED after sending the audio stream and then stopping the transmission.

  • COMPLETED: Completed

  • FAILED: Task failed

  • INVALID: Invalid, possibly because the TaskId does not exist.

ErrorCode

string

Cause

ErrorMessage

string

Error message

FAQ

Why are PPT extraction feature parameters missing in real-time recording compared to offline transcription?

PPT extraction (including PPT extraction and PPT summary) is primarily designed for offline transcription scenarios where the source file is a video. Neither audio files nor binary audio streams in real-time recording contain video information, so no PPT-related results are generated.

Why is the real-time recording status PAUSED when queried?

Ensure your real-time recording interaction sequence is correct. The standard interaction flow is: first create a real-time recording, send a StartTranscription instruction to the MeetingJoinUrl returned by the real-time recording service to begin recognition, push audio frames, send a StopTranscription instruction to end recognition, close the MeetingJoinUrl connection, and finally end the real-time recording operation and retrieve the final results. The MeetingJoinUrl connection can repeatedly send StartTranscription instructions, push audio frames, and send StopTranscription instructions to pause the real-time recording.

How do I select a language?

  • The following are common scenarios:

    Language Type

    Corresponding Parameters

    Applicable Scenarios (Real-time Recording)

    Example

    Single Language

    (Known Language)

    Parameter Name: Input.SourceLanguage

    Configurable parameter values (single selection): Chinese (cn), English (en), Cantonese (yue), Japanese (ja), Korean (ko)

    • Single-language speech recognition model.

    • Chinese supports 8K and 16K.

    • English, Japanese, Cantonese, and Korean only support 16K.

    Example 1:

    If the speech language is known to be Chinese

    Input.SourceLanguage="cn"

    Recognition result: Thank you for using Tongyi Tingwu.

    Example 2:

    If the speech language is known to be English

    Input.SourceLanguage="en"

    Recognition result: Thank you for using Tongyi Tingwu.

    Multilingual Mix

    • Parameter Name 1: Input.SourceLanguage

      Parameter Value: multilingual

    • Parameter Name 2: Input.LanguageHints

      Configurable parameter values (multiple selection): Chinese (cn), English (en), Cantonese (yue), Japanese (ja), Korean (ko), German (de), French (fr), Russian (ru)

    • Multilingual speech recognition.

    • Only supports 16K.

    • Supports simultaneous recognition of Chinese, English, Japanese, Cantonese, Korean, German, French, and Russian.

    • Supports limiting the range of languages.

    Example 1:

    If the speech contains multiple languages, directly recognize text in the corresponding languages.

    Input.SourceLanguage="multilingual"

    Recognition result: hello, everyone. Thank you for using Tongyi Tingwu.

    Example 2:

    If the speech contains multiple languages, but the language range is known (such as only Chinese, English, and Cantonese), directly recognize text in the corresponding languages while avoiding misrecognition of irrelevant languages such as Japanese, French, or Russian.

    Input.SourceLanguage="multilingual"

    Input.LanguageHints=['cn', 'en', 'yue']