Audio and video translation

Updated at:

qwen3-livetranslate-flash translates audio and video through the OpenAI-compatible chat completions endpoint. All requests are streamed.

Related documentation: Audio and video file translation - Qwen

The DashScope interface is not supported.

OpenAI compatible

POSThttps://WorkspaceId.cn-beijing.maas.aliyuncs.com/compatible-mode/v1/chat/completions
POSThttps://WorkspaceId.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/chat/completions

Beijing region

The base_url configured for SDK calls is: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1

The endpoint configured for HTTP calls: POST https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1/chat/completions

Singapore region

The base_url configured for SDK calls is: https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1

The endpoint configured for HTTP calls: POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/chat/completions

ImportantAlibaba Cloud Model Studio has launched workspace-specific dedicated domains for the China (Beijing) and Singapore regions, which deliver superior performance and higher stability for inference requests. We recommend that you migrate to the new domains:

  • China (Beijing) region: migrate from https://dashscope.aliyuncs.com to https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com
  • Singapore region: migrate from https://dashscope-intl.aliyuncs.com to https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com

{WorkspaceId} is your workspace ID, which you can view on the Workspace Details page in the Alibaba Cloud Model Studio console. The existing domains remain fully functional.

You must have obtained and configured an API key and configured the API key as an environment variable. If you call the API through the OpenAI SDK, you must install the SDK.

Request body

modelstring(required)

Model name. Supported models: qwen3-livetranslate-flash, qwen3-livetranslate-flash-2025-12-01.

messagesarray(required)

An array of messages used to pass context to the model. Only one user message is supported.

Message types

User Messageobject(required)

The user message.

Properties

contentarray(required)

The message content.

Properties

typestring(required)

Valid values:

  • input_audio

    Set it to input_audio when inputting audio.

  • video_url

    Set it to video_url when inputting a video file.

input_audioobject

The audio information to input. Required when type is input_audio.

Properties

datastring(required)

The URL or Base64 data URL of the audio. For local files, see Input a Base64-encoded local file.

formatstring(required)

The format of the input audio, such as mp3 or wav.

video_urlobject

The video file information to input. Required when type is video_url.

Properties

urlstring(required)

The public URL or Base64 data URL of the video file. For local video files, see Input a Base64-encoded local file.

rolestring(required)

The role of the user message, fixed to user.

streamboolean(required)

Default value: false

Whether to return responses in streaming mode. The model only supports streaming output, so it can only be set to true.

stream_optionsobject(optional)

Streaming output configuration. Takes effect only when stream is true.

Properties

include_usageboolean(optional)

Default value: false

Whether to include token usage information in the last data chunk.

Valid values:

  • true: included;
  • false: not included.
modalitiesarray(optional)

Default value: ["text"]

The modality of the output data. Valid values:

  • ["text","audio"]: outputs both text and audio;
  • ["text"]: outputs text only.
audioobject(optional)

The voice and format of the output audio. The modalities parameter must be ["text","audio"].

Properties

voicestring(required)

The voice of the output audio. See Supported voices.

formatstring(required)

The format of the output audio. Only wav is supported.

max_tokensinteger(optional)

Limits the maximum number of tokens the model can output. If the generated content exceeds this value, the response is truncated.

The default value and the maximum value are both the maximum output length of the model. See Model selection.

seedinteger(optional)

The random number seed. It is used to ensure that the generated results are reproducible with the same input and parameters. If the same seed is passed and the other parameters remain unchanged, the model tries to return the same result.

Value range: [0,2 31 −1].

speech_ratefloat(optional)

Default value: 1.0

Controls the speech rate of the output audio. 1.0 is normal speed, less than 1.0 is slower, and greater than 1.0 is faster.

Value range: [0.5, 2.0].

temperaturefloat(optional)

Default value: 0.000001

The sampling temperature, which controls the diversity of the generated content. A higher temperature produces more diverse content, and vice versa.

Value range: [0, 2)

For translation accuracy, we do not recommend changing this value.

top_pfloat(optional)

Default value: 0.8

The probability threshold for nucleus sampling, which controls the diversity of the generated content.

A higher top_p produces more diverse content, and vice versa.

Value range: (0,1.0]

For translation accuracy, we do not recommend changing this value.

presence_penaltyfloat(optional)

Default value: 0

Controls the repetition of content when the model generates text.

Value range: [-2.0, 2.0]. Positive values reduce repetition, and negative values increase repetition. For translation accuracy, we do not recommend changing this value.

top_kinteger(optional)

Default value: 1

The size of the sampling candidate set during generation. For example, when the value is 50, only the 50 highest-scoring tokens in a single generation form the random sampling candidate set. A larger value increases randomness, and a smaller value increases determinism. When the value is None or top_k is greater than 100, the top_k strategy is disabled and only the top_p strategy takes effect.

The value must be greater than or equal to 0. For translation accuracy, we do not recommend changing this value.

This parameter is not a standard OpenAI parameter. When calling through the Python SDK, put it in the extra_body object as follows: extra_body={"top_k": xxx}. When calling through the Node.js SDK or HTTP, pass it as a top-level parameter.

repetition_penaltyfloat(optional)

Default value: 1.05

The repetition of consecutive sequences when the model generates content. Increasing repetition_penalty reduces the repetition in the generated content, and 1.0 means no penalty. The value only needs to be greater than 0. For translation accuracy, we do not recommend changing this value.

This parameter is not a standard OpenAI parameter. When calling through the Python SDK, put it in the extra_body object as follows: extra_body={"repetition_penalty": xxx}. When calling through the Node.js SDK or HTTP, pass it as a top-level parameter.

translation_optionsobject(required)

The translation parameters to configure.

Properties

source_langstring(optional)

The full English name of the source language. See Supported languages. If not set, the model automatically detects the language of the input.

target_langstring(required)

The full English name of the target language. See Supported languages.

This parameter is not a standard OpenAI parameter. When calling through the Python SDK, put it in the extra_body object as follows: extra_body={"translation_options": xxx}. When calling through the Node.js SDK or HTTP, pass it as a top-level parameter.

import os
from openai import OpenAI

client = OpenAI(
    # 新加坡和北京地域的API Key不同。获取API Key:https://help.aliyun.com/zh/model-studio/get-api-key
    # 若没有配置环境变量,请用阿里云百炼API Key将下行替换为:api_key="sk-xxx",
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    base_url="https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1",
)

# ----------------音频输入 ----------------
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "input_audio",
                "input_audio": {
                    "data": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250211/tixcef/cherry.wav",
                    "format": "wav",
                },
            }
        ],
    }
]

# ----------------视频输入(需取消注释)----------------
# messages = [
#     {
#         "role": "user",
#         "content": [
#             {
#                 "type": "video_url",
#                 "video_url": {
#                     "url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241115/cqqkru/1.mp4"
#                 },
#             }
#         ],
#     },
# ]

completion = client.chat.completions.create(
    model="qwen3-livetranslate-flash",
    messages=messages,
    modalities=["text", "audio"],
    audio={"voice": "Cherry", "format": "wav"},
    stream=True,
    stream_options={"include_usage": True},
    extra_body={"translation_options": {"source_lang": "zh", "target_lang": "en"}},
)

for chunk in completion:
    print(chunk)
import OpenAI from "openai";

const client = new OpenAI({
    // 若没有配置环境变量,请用阿里云百炼API Key将下行替换为:apiKey: "sk-xxx",
    // 新加坡和北京地域的API Key不同。获取API Key:https://help.aliyun.com/zh/model-studio/get-api-key
    apiKey: process.env.DASHSCOPE_API_KEY,
    baseURL: "https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1",
});

// ---------------- 音频输入 ----------------
const messages = [
    {
        role: "user",
        content: [
            {
                type: "input_audio",
                input_audio: {
                    data: "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250211/tixcef/cherry.wav",
                    format: "wav",
                },
            },
        ],
    },
];

// ---------------- 视频输入(需取消注释) ----------------
// const messages = [
//     {
//         role: "user",
//         content: [
//             {
//                 type: "video_url",
//                 video_url: {
//                     url: "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241115/cqqkru/1.mp4",
//                 },
//             },
//         ],
//     },
// ];

async function main() {
    const completion = await client.chat.completions.create({
        model: "qwen3-livetranslate-flash",
        messages: messages,
        modalities: ["text", "audio"],
        audio: { voice: "Cherry", format: "wav" },
        stream: true,
        stream_options: { include_usage: true },
        translation_options: { source_lang: "zh", target_lang: "en" },
    });

    for await (const chunk of completion) {
        console.log(JSON.stringify(chunk));
    }
}

main();
# ======= 重要提示 =======
# 新加坡和北京地域的API Key不同。获取API Key:https://help.aliyun.com/zh/model-studio/get-api-key
# === 执行时请删除该注释 ===
curl -X POST https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/compatible-mode/v1/chat/completions \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "qwen3-livetranslate-flash",
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "input_audio",
                    "input_audio": {
                        "data": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20250211/tixcef/cherry.wav",
                        "format": "wav"
                    }
                }
            ]
        }
    ],
    "modalities": ["text", "audio"],
    "audio": {
        "voice": "Cherry",
        "format": "wav"
    },
    "stream": true,
    "stream_options": {
        "include_usage": true
    },
    "translation_options": {
        "source_lang": "zh",
        "target_lang": "en"
    }
}'

chat response chunk object (streaming output)

idstring

The unique identifier of this call. Every chunk object has the same id.

choicesarray

The array of model-generated content. If the include_usage parameter is set to true, choices is an empty array in the last chunk.

Properties

deltaobject

The incremental object of the request.

Properties

contentstring

The incremental message content.

reasoning_contentstring

This value is fixed to null.

function_callobject

This value is fixed to null.

audioobject

The output audio information.

Properties

datastring

The incremental Base64-encoded audio data.

expires_atinteger

The timestamp when the request was created.

idstring

The unique identifier of the output audio.

refusalobject

This parameter is currently fixed to null.

rolestring

The role of the incremental message object, present only in the first chunk.

tool_callsarray

This value is fixed to null.

finish_reasonstring

The reason the model stopped generating. Possible cases:

  • stop when the output stops naturally;
  • null when the generation is not finished;
  • length when the generation ends due to excessive length.
indexinteger

The index of the current response in the choices array, fixed to 0.

logprobsobject

This value is fixed to null.

createdinteger

The timestamp when this request was created. Every chunk has the same timestamp.

modelstring

The model used for this request.

objectstring

Always chat.completion.chunk.

service_tierstring

This value is fixed to null.

system_fingerprintstring

This value is fixed to null.

usageobject

The tokens consumed by this request. Displayed only in the last chunk when include_usage is true.

Properties

completion_tokensinteger

The number of tokens output by the model.

prompt_tokensinteger

The number of input tokens.

total_tokensinteger

The total number of tokens, which is the sum of prompt_tokens and completion_tokens.

completion_tokens_detailsobject

The details of the output tokens.

Properties

audio_tokensinteger

The number of output audio tokens.

reasoning_tokensinteger

This value is fixed to null.

text_tokensinteger

The number of output text tokens.

prompt_tokens_detailsobject

The fine-grained breakdown of the input tokens.

Properties

audio_tokensinteger

The number of input audio tokens.

The number of audio tokens in the video file is returned through this parameter.

text_tokensinteger

The number of input text tokens. This value is fixed to 0.

video_tokensinteger

The number of input video tokens.

{
  "id": "chatcmpl-c22a54b8-40cc-4a1d-988b-f84cdf86868f",
  "choices": [
    {
      "delta": {
        "content": " of",
        "function_call": null,
        "refusal": null,
        "role": null,
        "tool_calls": null
      },
      "finish_reason": null,
      "index": 0,
      "logprobs": null
    }
  ],
  "created": 1764755440,
  "model": "qwen3-livetranslate-flash",
  "object": "chat.completion.chunk",
  "service_tier": null,
  "system_fingerprint": null,
  "usage": null
}
{
  "id": "chatcmpl-c22a54b8-40cc-4a1d-988b-f84cdf86868f",
  "choices": [
    {
      "delta": {
        "content": null,
        "function_call": null,
        "refusal": null,
        "role": null,
        "tool_calls": null,
        "audio": {
          "data": "///+//7////+////////////AAAAAAAAAAABA......",
          "expires_at": 1764755440,
          "id": "audio_c22a54b8-40cc-4a1d-988b-f84cdf86868f"
        }
      },
      "finish_reason": null,
      "index": 0,
      "logprobs": null
    }
  ],
  "created": 1764755440,
  "model": "qwen3-livetranslate-flash",
  "object": "chat.completion.chunk",
  "service_tier": null,
  "system_fingerprint": null,
  "usage": null
}
{
  "id": "chatcmpl-c22a54b8-40cc-4a1d-988b-f84cdf86868f",
  "choices": [],
  "created": 1764755440,
  "model": "qwen3-livetranslate-flash",
  "object": "chat.completion.chunk",
  "service_tier": null,
  "system_fingerprint": null,
  "usage": {
    "completion_tokens": 242,
    "prompt_tokens": 415,
    "total_tokens": 657,
    "completion_tokens_details": {
      "accepted_prediction_tokens": null,
      "audio_tokens": 191,
      "reasoning_tokens": null,
      "rejected_prediction_tokens": null,
      "text_tokens": 51
    },
    "prompt_tokens_details": {
      "audio_tokens": 415,
      "cached_tokens": null,
      "text_tokens": 0
    }
  }
}