cosyvoice, android, sdk, speech synthesis, tts, text to speech, alibaba cloud-Alibaba Cloud Model Studio(Model Studio)-阿里云帮助中心

Convert text to expressive, high-quality speech in your Android app with the CosyVoice text-to-speech (TTS) SDK.

User guide: For model details and selection guidance, see Speech synthesis.

NativeNui

The SDK is built on the NativeNui singleton and uses callbacks to deliver synthesis events.

Architecture highlights:

Singleton: Call NativeNui.GetInstance() to get the global, unique instance.
Callback-driven: Receive events and audio data through the INativeStreamInputTtsCallback interface.
Event types:
- STREAM_INPUT_TTS_EVENT_SYNTHESIS_STARTED: The synthesis task has started.
- onStreamInputTtsDataCallback: Audio data is delivered.
- STREAM_INPUT_TTS_EVENT_SYNTHESIS_COMPLETE: The synthesis task has finished.
- STREAM_INPUT_TTS_EVENT_TASK_FAILED: The synthesis task has failed.

Usage flow

CosyVoice supports two invocation modes: one-shot input and streaming input.

One-shot input is suited to short text and any scenario that uses SSML markup.

startStreamInputTts() — initialize the SDK and configure the callback and connection parameters.
playStreamInputTts() or asyncPlayStreamInputTts() — submit the text and start synthesis.
onStreamInputTtsDataCallback() — receive audio data.
STREAM_INPUT_TTS_EVENT_SYNTHESIS_COMPLETE — synthesis is complete.

Streaming input is suited to real-time conversation and long-form "speak as you go" scenarios. SSML is not supported in this mode.

startStreamInputTts() — initialize the SDK and configure the callback and connection parameters.
sendStreamInputTts() — send text continuously as it becomes available.
onStreamInputTtsDataCallback() — receive audio data.
stopStreamInputTts() or asyncStopStreamInputTts() — signal that all text has been sent and wait for synthesis to finish.
STREAM_INPUT_TTS_EVENT_SYNTHESIS_COMPLETE — synthesis is complete.

startStreamInputTts

Starts bidirectional streaming synthesis, opens the connection to the service, and registers the callback for events and audio data.

This method can block. Call it on a non-UI thread.

Method signature:

public synchronized int startStreamInputTts(INativeStreamInputTtsCallback callback,
                                            String ticket,
                                            String parameters,
                                            String session_id,
                                            int log_level,
                                            boolean save_log)

Parameters:

Parameter	Type	Description
`callback`	`INativeStreamInputTtsCallback`	Your implementation of the event and data callback interface.
`ticket`	`String`	A JSON string that holds authentication, connection, and debug settings. See the ticket parameter reference below.
`parameters`	`String`	A JSON string that controls the speech synthesis output. See the parameters reference below.
`session_id`	`String`	A client-specified session ID. If omitted, the server generates one automatically.
`log_level`	`int`	Controls the verbosity of the SDK's internal logs. Valid values: 0: LOG_LEVEL_VERBOSE 1: LOG_LEVEL_DEBUG 2: LOG_LEVEL_INFO 3: LOG_LEVEL_WARNING 4: LOG_LEVEL_ERROR 5: LOG_LEVEL_NONE (disables logging)
`save_log`	`boolean`	Whether to write logs to a local file. If true, set `debug_path` in the ticket to specify the destination, and optionally set `max_log_file_size` to cap the file size.

Return value:

Returns an error code.

ticket JSON example:

{
    "url": "wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference",
    "apikey": "sk-****",
    "device_id": "my_device_id"
}

ticket parameters:

Parameter	Type	Required	Description
`url`	`String`	Yes	The service endpoint. Fixed value: `wss://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api-ws/v1/inference`. Replace `WorkspaceId` with your actual workspace ID.
`apikey`	`String`	Yes	Your API key. To limit the blast radius if a long-lived key is exposed, use a short-lived temporary API key instead.
`device_id`	`String`	Yes	A unique identifier for the end user. Set it to your in-app user ID or to a client-generated device identifier. This ID is used mainly for log correlation and troubleshooting.
`complete_waiting_ms`	`int`	No	After you call the stop method, the time in milliseconds to wait for the STREAM_INPUT_TTS_EVENT_SYNTHESIS_COMPLETE event before timing out. Default: 10000.
`debug_path`	`String`	No	The local path where log files are written. This parameter takes effect only when `save_log` is set to true for startStreamInputTts, playStreamInputTts, or asyncPlayStreamInputTts. In that case, the path is required; otherwise the call returns an error. At most two log files are kept on the device.
`max_log_file_size`	`int`	No	The maximum log-file size in bytes. This parameter takes effect only when `save_log` is set to true for startStreamInputTts, playStreamInputTts, or asyncPlayStreamInputTts. Default: 104857600 (100 × 1024 × 1024 bytes, or 100 MiB).
`log_track_level`	`int`	No	Filter level for logs that are delivered through the log callback (`onStreamInputTtsLogTrackCallback`). Default: 2. Valid values: 0: LOG_LEVEL_VERBOSE 1: LOG_LEVEL_DEBUG 2: LOG_LEVEL_INFO 3: LOG_LEVEL_WARNING 4: LOG_LEVEL_ERROR 5: LOG_LEVEL_NONE (disables logging) Note: `log_track_level` works together with `log_level` (set on startStreamInputTts, playStreamInputTts, or asyncPlayStreamInputTts) to decide which logs reach the callback. A log entry must be at or above both `log_track_level` and `log_level` to be delivered. For example, if `log_track_level` is 2 (INFO) and `log_level` is 3 (WARNING), only WARNING and above (level >= 3) reach the callback.

parameters JSON example:

{
    "model": "cosyvoice-v2",
    "voice": "longxiaochun",
    "format": "mp3",
    "volume": 50,
    "rate": 1.0,
    "pitch": 1.0
}

parameters reference:

Parameter	Type	Required	Description
`model`	`String`	Yes	The model name.
`voice`	`String`	Yes	The voice used for speech synthesis. System voices: See CosyVoice Voice list Cloned voices: Custom voices created through voice cloning Custom voices: Custom voices created through voice design
`format`	`String`	No	The audio encoding format. Valid values: pcm wav mp3 (default) opus Important `cosyvoice-v1` doesn't support the opus format.
`volume`	`int`	No	The volume level. Default value: 50. Valid values: [0, 100].
`sample_rate`	`int`	No	The audio sample rate in Hz. Valid values: 8000, 16000, 22050 (default), 24000, 44100, 48000.
`rate`	`float`	No	The speech rate. Default value: 1.0. Valid values: [0.5, 2.0].
`pitch`	`float`	No	The pitch. Default value: 1.0. Valid values: [0.5, 2.0].
`bit_rate`	`int`	No	The audio bit rate in kbps. When the audio format is opus, use `bit_rate` to adjust the bit rate. Default value: 32. Valid values: [6, 510]. `cosyvoice-v1` doesn't support this parameter.
`enable_ssml`	`boolean`	No	Whether to enable SSML. Default: false. true: enabled. false: disabled.
`word_timestamp_enabled`	`boolean`	No	Specifies whether to enable word-level timestamps. Default value: false. Available only in streaming output mode. Supported voices: cloned voices of cosyvoice-v3.5-plus, cosyvoice-v3.5-flash, cosyvoice-v3-flash, cosyvoice-v3-plus, and cosyvoice-v2, and system voices marked as supported in CosyVoice Voice list. Cloned voices of other models do not support this feature. Word timestamps are returned in the all_response field of INativeStreamInputTtsCallback.
`seed`	`int`	No	A random seed for controlling variation in the synthesis output. When the model version, text, voice, and other parameters are unchanged, using the same seed produces identical results. Default value: 0. Valid values: [0, 65535]. cosyvoice-v1 doesn't support this parameter.
`language_hints`	`String[]`	No	Important This parameter is an array, but the current version only processes the first element. Pass a single value. This parameter specifies the target language for speech synthesis. It's unrelated to the language of the audio sample used in voice cloning. To set the source language for a cloning task, see the voice cloning API reference. Specifies the target language for speech synthesis to improve output quality. cosyvoice-v1 doesn't support this feature. When digit pronunciation, abbreviation expansion, symbol reading, or minority-language synthesis doesn't meet expectations, use this parameter. For example: Unexpected digit pronunciation: "hello, this is 110" is read as "hello, this is one zero" instead of the expected Chinese pronunciation Inaccurate symbol pronunciation: "@" is read as the Chinese equivalent instead of "at" Poor minor language synthesis quality with unnatural results Valid values: zh: Chinese en: English fr: French de: German ja: Japanese ko: Korean ru: Russian pt: Portuguese th: Thai id: Indonesian vi: Vietnamese
`instruction`	`String`	No	Controls synthesis characteristics such as dialect, emotion, or speaking style. For usage details, see Instruction-based control.
`enable_aigc_tag`	`boolean`	No	Specifies whether to embed an AIGC watermark in the generated audio. When set to true, the watermark is embedded in audio files of supported formats (wav/mp3/opus). Default value: false. Only cosyvoice-v3-flash, cosyvoice-v3-plus, and cosyvoice-v2 support this feature.
`aigc_propagator`	`String`	No	Sets the `ContentPropagator` field in the AIGC watermark, identifying the content propagator. Takes effect only when `enable_aigc_tag` is `true`. Default value: Alibaba Cloud UID. Only cosyvoice-v3-flash, cosyvoice-v3-plus, and cosyvoice-v2 support this feature.
`aigc_propagate_id`	`String`	No	Sets the `PropagateID` field in the AIGC watermark, uniquely identifying a specific propagation action. Takes effect only when `enable_aigc_tag` is `true`. Default value: The request ID of the current speech synthesis request. Only cosyvoice-v3-flash, cosyvoice-v3-plus, and cosyvoice-v2 support this feature.

sendStreamInputTts

Sends text to be synthesized. Use this method together with startStreamInputTts.

After you call startStreamInputTts, call this method to push text continuously.

When you have sent all the text, call stopStreamInputTts or asyncStopStreamInputTts to end the input.

Method signature:

public synchronized int sendStreamInputTts(String text)

Parameters:

Parameter	Type	Description
`text`	`String`	Text to synthesize. SSML is not supported. SSML tags in the input are read aloud as plain text rather than parsed.

Return value:

Returns an error code.

stopStreamInputTts

Synchronous method that tells the server all text has been sent and blocks until all audio has been synthesized and the STREAM_INPUT_TTS_EVENT_SYNTHESIS_COMPLETE event is received.

The wait timeout is controlled by complete_waiting_ms.

Method signature:

public synchronized int stopStreamInputTts()

Return value:

Returns an error code.

asyncStopStreamInputTts

Asynchronous method that tells the server all text has been sent. The call returns immediately, and synthesis continues in the background.

Use the STREAM_INPUT_TTS_EVENT_SYNTHESIS_COMPLETE event to detect when synthesis is finished.

Method signature:

public synchronized int asyncStopStreamInputTts()

Return value:

Returns an error code.

cancelStreamInputTts

Immediately closes the connection to the server and aborts the current synthesis task. No further audio callbacks are delivered after this call.

Method signature:

public synchronized int cancelStreamInputTts()

Return value:

Returns an error code.

playStreamInputTts

Synchronous one-shot synthesis method. The call sends the text, blocks while audio is received, and returns only after synthesis is complete. You don't need to call the stop method afterward.

SSML is enabled by default for this method. To override, set enable_ssml explicitly.

Call this method on a non-UI thread.

Method signature:

public synchronized int playStreamInputTts(INativeStreamInputTtsCallback callback,
                                           String ticket,
                                           String parameters,
                                           String text,
                                           String session_id,
                                           int log_level,
                                           boolean save_log)

Parameters:

The callback, ticket, and other shared parameters have the same meaning as in startStreamInputTts.

Parameter	Type	Description
`text`	`String`	Text to synthesize. Supports SSML.

Return value:

Returns an error code.

asyncPlayStreamInputTts

Asynchronous one-shot synthesis method. The call returns immediately, synthesis runs in the background, and results are delivered through the callback. You don't need to call the stop method afterward.

SSML is enabled by default for this method. To override, set enable_ssml explicitly.

Method signature:

public synchronized int asyncPlayStreamInputTts(INativeStreamInputTtsCallback callback,
                                           String ticket,
                                           String parameters,
                                           String text,
                                           String session_id,
                                           int log_level,
                                           boolean save_log)

Parameters:

The callback, ticket, and other shared parameters have the same meaning as in startStreamInputTts.

Parameter	Type	Description
`text`	`String`	Text to synthesize. Supports SSML.

Return value:

Returns an error code.

INativeStreamInputTtsCallback

The CosyVoice streaming TTS callback interface delivers synthesis events, audio data, and logs.

onStreamInputTtsEventCallback: receive events

Method signature:

void onStreamInputTtsEventCallback(StreamInputTtsEvent event,
                                   String task_id,
                                   String session_id,
                                   int ret_code,
                                   String error_msg,
                                   String timestamp,
                                   String all_response);

Parameters:

Parameter	Type	Description
`event`	StreamInputTtsEvent	The event being delivered.
`task_id`	`String`	The synthesis task ID.
`session_id`	`String`	The session ID. Returned as supplied by the client; generated by the server if omitted.
`ret_code`	`int`	Error code. Valid only with the STREAM_INPUT_TTS_EVENT_TASK_FAILED event.
`error_msg`	`String`	Error message. Valid only with the STREAM_INPUT_TTS_EVENT_TASK_FAILED event.
`timestamp`	`String`	Timestamp information for the synthesis result.
`all_response`	`String`	The full JSON response. Parse it to extract any additional data you need.

onStreamInputTtsDataCallback: receive audio data

The SDK invokes this callback repeatedly during synthesis. Read the audio data from the callback.

Method signature:

void onStreamInputTtsDataCallback(byte[] data);

Parameters:

Parameter

Type

Description

data

byte[]

Audio data for the current segment. Use it to:

Assemble a complete audio file and play it back.
Play it in real time with a streaming-capable player.

Notes:

For compressed formats such as mp3 and opus, segmented playback requires a streaming player. Decoding can fail if you play the segments frame by frame.
To assemble a complete file, open it in append mode and write each segment in order.
For wav and mp3, only the first onStreamInputTtsDataCallback invocation contains the file header. Subsequent invocations carry raw audio. Concatenate all the buffer chunks in order. For opus, each frame is a self-contained Ogg page and can be concatenated directly.

onStreamInputTtsLogTrackCallback: receive trace logs

Use this callback to receive detailed internal logs from the SDK, which helps with troubleshooting and debugging.

Method signature:

default void onStreamInputTtsLogTrackCallback(Constants.LogLevel level, String log)

StreamInputTtsEvent

Event type enumeration for CosyVoice streaming speech synthesis.

Event	Description
STREAM_INPUT_TTS_EVENT_SYNTHESIS_STARTED	The server has accepted the request and started processing. Shortly after this event, `onStreamInputTtsDataCallback` starts returning the first audio data.
STREAM_INPUT_TTS_EVENT_SENTENCE_SYNTHESIS	Runtime information about the synthesis, including billing data.
STREAM_INPUT_TTS_EVENT_SYNTHESIS_COMPLETE	The server has sent all audio data, and `onStreamInputTtsEventCallback` won't be called again. This event is the explicit signal that the audio stream has ended.
STREAM_INPUT_TTS_EVENT_TASK_FAILED	The task failed. Read the all_response field of INativeStreamInputTtsCallback for task_id, error_code, and error_message to identify the cause. `{ "header": { "task_id": "2bf83b9a-baeb-4fda-8d9a-xxxxxxxxxxxx", "event": "task-failed", "error_code": "InvalidParameter", "error_message": "[tts:]Engine return error code: 418", "attributes": {} }, "payload": {} }`

Sample code

Get an API key: Obtain an API key. For security, store the API key in an environment variable.

Note
Use a temporary API key when you need to grant temporary access to a third-party app or user, or when you want to tightly control high-risk operations such as accessing or deleting sensitive data. A temporary API key is valid for 60 seconds, after which you must request a new one.
Download the SDK and run the sample code:
- Download the latest SDK package.
- Unzip the package. The AAR-format SDK is in app/libs — add it to your project dependencies. For Android C++ integration, get the shared libraries from android_libs and the headers from android_include inside the ZIP.
- Open the project in Android Studio. The sample code is in DashCosyVoiceStreamTtsActivity.java — replace the API key with your own to try the feature.

Invocation modes

Mode

Description

One-shot text input

Steps:

Initialize the SDK and the player components.
Configure the parameters for your scenario.
Call playStreamInputTts or asyncPlayStreamInputTts to submit the text and start synthesis.
Receive the STREAM_INPUT_TTS_EVENT_SYNTHESIS_COMPLETE event — synthesis is complete.

Use cases:

Short text synthesis
Scenarios that require SSML markup

Streaming text input

Steps:

Initialize the SDK and the player components.
Configure the parameters for your scenario.
Call startStreamInputTts to start streaming synthesis.
Call sendStreamInputTts to send text continuously as it becomes available.
Receive binary audio data in onStreamInputTtsDataCallback.
Call stopStreamInputTts or asyncStopStreamInputTts to end the input and wait for synthesis to finish.
Receive the STREAM_INPUT_TTS_EVENT_SYNTHESIS_COMPLETE event — synthesis is complete.

Use cases:

Real-time conversation and long-form "speak as you go" scenarios
SSML markup is not supported in this mode

Advanced features

SSML markup

Purpose: Embed XML tags in the text to fine-tune pronunciation, speaking rate, pauses, and other prosodic details.

Limitations: SSML is supported only with One-shot text input (the playStreamInputTts or asyncPlayStreamInputTts methods). Streaming text input (the sendStreamInputTts method) does not support SSML.

Usage: When you call playStreamInputTts or asyncPlayStreamInputTts, the SDK enables SSML automatically. Pass text that contains SSML tags through the text parameter.

For more information, see SSML markup language.

Math expressions

Purpose: Let the model read common math formulas and expressions aloud correctly.

Usage: Pass text that contains LaTeX-formatted math expressions through the text parameter. For more information, see Convert LaTeX formulas to speech (Chinese language only).