SSML-based synthesis
This topic describes Speech Synthesis Markup Language (SSML), its supported tags, and examples.
Overview
SSML is an XML-based markup language for speech synthesis. It controls sentence and word boundaries, pronunciation, speech rate, pauses, pitch, and volume, and can add background music.
Intelligent Speech Interaction implements W3C SSML 1.0 and supports the tags described in this topic, rather than all W3C tags.
Usage
SSML is currently supported only for Chinese and English voices. Supported tags and content differ by language, as described in the following tag reference.
-
For short-text speech synthesis, enclose all text in a single
<speak>...</speak>root element. The XML declaration before<speak>can be omitted. -
Long-text tasks, including real-time and asynchronous long-text synthesis, can contain multiple
<speak>...</speak>pairs and mix SSML with plain text. The following text can be sent in a single long-text request. For real-time long-text synthesis with the Java SDK, set the text by usingsetLongText.<speak><say-as interpret-as="telephone">114</say-as>查询号码 <say-as interpret-as="cardinal">123</say-as>开始干。加起来为<say-as interpret-as="digits">1234</say-as>。<say-as interpret-as="name">张三</say-as>的快递。<say-as interpret-as="address">富路国际1号楼3单元304</say-as><say-as interpret-as="nick">李四6689</say-as></speak>
Escape XML special characters in tag content and attributes: use " for ", ' for ', & for &, < for <, and > for >.
Pass SSML text in the text parameter of a speech synthesis request. The following example shows a Java SDK snippet and the corresponding request. For client initialization and the complete calling procedure, see Java SDK.
Java SDK snippet:
SpeechSynthesizer synthesizer = new SpeechSynthesizer(client, getSynthesizerListener());
String text = "<speak>请闭上眼睛休息一下<break time=\"500ms\"/>好了，请睁开眼睛。</speak>";
synthesizer.setText(text);Corresponding request:
{
"payload": {
"volume": 50,
"sample_rate": 16000,
"format": "wav",
"text": "<speak>请闭上眼睛休息一下<break time=\"500ms\"/>好了，请睁开眼睛。</speak>"
},
"context": {
"sdk": {
"name": "nls-sdk-java",
"version": "2.0.4"
}
},
"header": {
"namespace": "SpeechSynthesizer",
"name": "StartSynthesis",
"message_id": "5fdf78c0dd574b6897f3cb204dd0****",
"appkey": "fd4er4aa****",
"task_id": "6e1be78ef5804c50a2c5a8b92de1****"
}
} Tags
<speak>
The <speak> tag is the root element for all supported Speech Synthesis Markup Language (SSML) tags and must enclose all SSML-formatted text.
<speak>SSML text</speak>Attributes
The <speak> tag supports the following properties.
|
Attribute |
Type |
Value |
Required |
Description |
|
|
String |
The name of the online speaker that can be called. The value is the lowercase value of the |
No |
A proprietary tag of Alibaba Cloud speech synthesis. It specifies the speaker during synthesis and has a higher priority than the speaker specified by the "xiaoyue" cannot currently be set by |
|
|
String |
PCM, WAV, or MP3 |
No |
A proprietary tag of Alibaba Cloud speech synthesis. It specifies the audio file format during synthesis and has a higher priority than the format specified by the The |
|
|
String |
8000, 16000, 24000, or 48000 |
No |
A proprietary tag of Alibaba Cloud speech synthesis. It specifies the audio sample rate during synthesis and has a higher priority than the rate specified by the The |
|
|
String |
An integer from -500 to 500. The default value is 0.
|
No |
A proprietary tag of Alibaba Cloud speech synthesis. It specifies the audio speed during synthesis and has a higher priority than the speed specified by the |
|
|
String |
An integer from -500 to 500. The default value is 0.
|
No |
A proprietary tag of Alibaba Cloud speech synthesis. It specifies the audio pitch during synthesis and has a higher priority than the pitch specified by the |
|
|
String |
An integer from 0 to 100. The default value is 50.
|
No |
A proprietary tag of Alibaba Cloud speech synthesis. It specifies the audio volume during synthesis and has a higher priority than the volume specified by the |
|
|
String |
robot, lolita, lowpass, echo, eq, lpfilter, or hpfilter |
No |
A proprietary tag of Alibaba Cloud speech synthesis. Use this tag to produce different sound effects for the synthesized speech.
|
|
|
String |
When the |
No |
|
|
|
String |
The name of an available background music track. For details, see the bgm attribute. |
No |
A proprietary tag of Alibaba Cloud speech synthesis. It adds the specified background music to the synthesized speech. |
|
|
String |
An integer from 0 to 100. The default value is 50.
|
No |
A proprietary tag of Alibaba Cloud speech synthesis. It controls the volume of the background music. |
bgm attribute
Built-in background music
The Alibaba Cloud speech synthesis service has several built-in background music tracks for you to try:
Custom background music
Store custom audio in OSS and ensure that its URL is accessible over HTTP or HTTPS. To create a bucket, see Create a bucket. For file upload and sharing, see Console quick start.
Audio requirements:
-
16 kHz sample rate, single-channel WAV format.
-
Short-text synthesis: no larger than 2.8 MB. Long-text synthesis: no larger than 7.3 MB.
-
If the synthesis duration is longer than the background music duration, the background music will loop. If the background music is not in WAV format, you can use FFmpeg to convert it:
ffmpeg -i input_audio -acodec pcm_s16le -ac 1 -ar 16000 target.wav. -
If the URL in the tag contains special XML characters, you must escape them.
-
The bit depth must be 16-bit.
You are legally responsible for the copyright of the audio you upload.
Examples
No attributes
<speak>
需要调用SSML标签的文本
</speak>Audio effect: SSML-speak1.mp3
voice property
<speak voice="xiaogang">
我是男声。
</speak>Audio effect: SSML-speak2.mp3
encodeType property
<speak encodeType="mp3">
我可以生成压缩格式的音频。
</speak>Audio effect: SSML-encode.mp3
sampleRate property
<speak sampleRate="8000">
看看我的文件大小吧，是16000采样率音频的一半。
</speak>Audio effect: SSML-speak4.mp3
rate property
<speak rate="200">
我的语速比正常人快。
</speak>Audio effect: SSML-speak5.mp3
pitch property
<speak pitch="-100">
我的音高却比别人低。
</speak>Audio effect: SSML-speak6.mp3
volume property
<speak volume="80">
我的音量也很大。
</speak>Audio effect: SSML-speak7.mp3
Property combination (separated by spaces)
<speak rate="200" pitch="-100" volume="80">
所以放在一起，我的声音是这样的。
</speak>Audio effect: SSML-speak8.mp3
effect property
<speak effect="robot">
你喜欢机器人瓦力吗？
</speak>Audio effect: SSML-speak9.mp3
bgm property
<speak bgm="http://nls.alicdn.com/bgm/2.wav" backgroundMusicVolume="30" rate="-500" volume="40">
<break time="2s"/>
阴崖老木苍苍烟
<break time="700ms"/>
雨声犹在竹林间
<break time="700ms"/>
绵蕝固知裨国计
<break time="700ms"/>
绵州风物总堪怜
<break time="2s"/>
</speak>Audio effect: SSML-speak10.mp3
Tag relationships
The <speak> tag can contain text and the following tags:
-
<break>
-
<s>
-
<w>
-
<phoneme>
-
<say-as>
-
<vhml/>
<emotion>
The <emotion> tag is used for multi-emotion speech synthesis. This tag is optional. If you use this tag with a speaker that does not support multi-emotion speech synthesis, the synthesis request fails.
<emotion category="happy" intensity="1.0">今天天气真不错！</emotion>Attributes
The <emotion> tag supports the following properties.
|
Attribute |
Type |
Value |
Required |
Description |
|
|
String |
Enumeration value, such as neutral or happy. |
Yes |
Specifies the emotion of the speech. For the emotions supported by each speaker, see the following table. |
|
|
String |
Floating-point value. The range is [0.01, 2.0]. |
No |
Specifies the intensity of the emotion. The default value is 1.0, which indicates the predefined emotion intensity. The minimum value is 0.01, which results in a slight tendency toward the target emotion. The maximum value is 2.0, which doubles the intensity of the target emotion. |
This topic describes the support for multi-emotion voices.
|
Voice name |
|
Emotion category |
|
Zhifeng_Multi-emotion |
zhifeng_emo |
angry, fear, happy, neutral, sad, surprise |
|
Zhibing_Multi-emotion |
zhibing_emo |
angry, fear, happy, neutral, sad, surprise |
|
Zhimiao_Multi-emotion |
zhimiao_emo |
serious, sad, disgust, jealousy, embarrassed, happy, fear, surprise, neutral, frustrated, affectionate, gentle, angry, newscast, customer-service, story, living |
|
Zhimi_Multi-emotion |
zhimi_emo |
angry, fear, happy, hate, neutral, sad, surprise |
|
Zhiyan_Multi-emotion |
zhiyan_emo |
neutral, happy, angry, sad, fear, hate, surprise, arousal |
|
Zhibei_Multi-emotion |
zhibei_emo |
neutral, happy, angry, sad, fear, hate, surprise |
|
Zhitian_Multi-emotion |
zhitian_emo |
neutral, happy, angry, sad, fear, hate, surprise |
Examples
<speak voice="zhitian_emo">
<emotion category="happy" intensity="1.0">今天天气真不错！</emotion>
</speak>Audio effect: SSML-emotion.wav
Tag relationships
The <emotion> tag can contain text and the following tags:
-
<s>
-
<sub>
-
<say-as>
-
<w>
-
<phoneme>
-
<soundEvent/>
-
<break/>
-
<vhml/>
<break>
You can use this tag to insert a pause in the text. This tag is optional.
<!-- Without attributes -->
<break/>
<!-- With the time attribute -->
<break time="string"/>Attributes
If you use the <break> tag without any properties, the default pause duration is 1s.
|
Attribute |
Type |
Value |
Required |
Description |
|
|
String |
[number] s / [number] ms |
No |
Sets the duration of the pause in seconds (s) or milliseconds (ms), such as "2s" or "50ms".
|
If multiple <break> tags appear consecutively, the total pause duration is the sum of the durations of each tag. If the total duration exceeds 10 seconds, the effective pause duration is 10 seconds.
For example, in the following code, the total pause duration of the three consecutive <break> tags is 15 seconds. Because this exceeds 10 seconds, the final effective pause duration is 10 seconds.
<speak>
Close your eyes and take a rest.<break time="5s"/><break time="5s"/><break time="5s"/>Okay, now open your eyes.
</speak>Examples
<speak>
请闭上眼睛休息一下<break time="500ms"/>好了，请睁开眼睛。
</speak>Audio effect: SSML-break.mp3
Tag relationships
The <break> tag is an empty tag and cannot contain other tags. If an SSML structure contains an <s> tag, place the <break> tag inside it to set a pause for that paragraph or sentence.
<s>
You can use this tag to represent the sentence structure of the text. This tag is optional.
<s>text</s>Attributes
None.
Examples
<speak><s>这是第一句话</s><s>这是第二句话</s></speak>Audio effect: SSML-s.mp3
Tag relationships
The <s> tag can contain text and the following tags:
-
<break>
-
<w>
-
<phoneme>
-
<say-as>
-
<vhml/>
<sub>
This tag replaces the text within it with an alias.
<sub alias="string"></sub>Attributes
|
Attribute |
Type |
Value |
Required |
Description |
|
|
String |
The replacement content. |
Yes |
Used to replace the text within the tag. |
Examples
<speak><sub alias="网络协议标准">W3C</sub></speak>Audio effect: SSML-sub.mp3
Tag relationships
The <sub> tag can contain text and the <vhml/> tag.
<w>
You can use this tag to define the word structure of the text. This tag is optional. Because English text is typically tokenized by spaces, this tag is usually not required for English. The content within a <w> tag must be a single word or phrase. The word or phrase cannot be a mix of Chinese and other languages.
<w>text</w>Attributes
None.
Examples
<speak>南京市长<w>江大桥</w>今天发表了演讲。</speak>Audio effect: SSML-w.mp3
Tag relationships
The <w> tag can contain text and the <vhml/> tag.
<vhml>
To obtain timestamps for animation markers, set enable_subtitle=true and enable_phoneme_timestamp=true in the speech synthesis request.
The <vhml/> tag is a Virtual Human Markup Language tag that inserts markers for body and facial animation into text. Its src attribute is a user-defined string. The value is returned in the subtitle timestamp data and can be used to drive facial or body animation. The marker appears in text, phoneme is the string "null", and begin_index, end_index, begin_time, and end_time are all 0.
Usually, <vhml> markers appear in timestamp data in the same order as in the original text. If a marker is inside or adjacent to text that requires normalization (for example, 123 is normalized to “一百二十三”), its returned position may differ from its position in the original text. The relative order of multiple <vhml> markers is preserved.
<vhml src="content"/>Attributes
None.
Examples
<speak><vhml src="start"/>我们今天来<vhml src="t0"/>介绍一下这个新商品<vhml src="end"/></speak>Timestamp data:
{
"subtitles":[
{
"text":"start",
"phoneme":"null",
"begin_index":0,
"end_index":0,
"begin_time":0,
"end_time":0
},
{
"text":"我",
"phoneme":"null",
"begin_index":0,
"end_index":1,
"begin_time":0,
"end_time":88
},
{
"text":"们",
"phoneme":"null",
"begin_index":1,
"end_index":2,
"begin_time":88,
"end_time":238
},
{
"text":"今",
"phoneme":"null",
"begin_index":2,
"end_index":3,
"begin_time":238,
"end_time":450
},
{
"text":"天",
"phoneme":"null",
"begin_index":3,
"end_index":4,
"begin_time":450,
"end_time":650
},
{
"text":"来",
"phoneme":"null",
"begin_index":4,
"end_index":5,
"begin_time":650,
"end_time":825
},
{
"text":"t0",
"phoneme":"null",
"begin_index":0,
"end_index":0,
"begin_time":0,
"end_time":0
},
{
"text":"介",
"phoneme":"null",
"begin_index":5,
"end_index":6,
"begin_time":825,
"end_time":988
},
{
"text":"绍",
"phoneme":"null",
"begin_index":6,
"end_index":7,
"begin_time":988,
"end_time":1213
},
{
"text":"一",
"phoneme":"null",
"begin_index":7,
"end_index":8,
"begin_time":1213,
"end_time":1325
},
{
"text":"下",
"phoneme":"null",
"begin_index":8,
"end_index":9,
"begin_time":1325,
"end_time":1525
},
{
"text":"这",
"phoneme":"null",
"begin_index":9,
"end_index":10,
"begin_time":1525,
"end_time":1663
},
{
"text":"个",
"phoneme":"null",
"begin_index":10,
"end_index":11,
"begin_time":1663,
"end_time":1750
},
{
"text":"新",
"phoneme":"null",
"begin_index":11,
"end_index":12,
"begin_time":1750,
"end_time":1950
},
{
"text":"商",
"phoneme":"null",
"begin_index":12,
"end_index":13,
"begin_time":1950,
"end_time":2188
},
{
"text":"品",
"phoneme":"null",
"begin_index":13,
"end_index":14,
"begin_time":2188,
"end_time":2538
},
{
"text":"end",
"phoneme":"null",
"begin_index":0,
"end_index":0,
"begin_time":0,
"end_time":0
}
]
}Tag relationships
<vhml/> is an empty tag.
<phoneme>
You can use this tag to control the pronunciation of the text within it. This tag is optional and is not supported for English text.
<phoneme alphabet="string" ph="string">Chinese text</phoneme>Attributes
|
Attribute |
Type |
Value |
Required |
Description |
|
|
String |
py |
Yes |
"py" indicates Pinyin. |
|
|
String |
The Pinyin string corresponding to the text within the tag. |
Yes |
Pinyin assignment rules:
|
Examples
<speak>
去<phoneme alphabet="py" ph="dian3 dang4 hang2">典当行</phoneme>把这个玩意<phoneme alphabet="py" ph="dang4 diao4">当掉</phoneme>
</speak>Audio effect: SSML-phoneme.mp3
Tag relationships
The <phoneme> tag can contain text and the <vhml/> tag.
<soundEvent>
A sound cue tag. You can use this tag to insert a sound cue at any position during SSML synthesis.
<soundEvent src="URL"/>Attributes
|
Attribute |
Type |
Value |
Required |
Description |
|
|
String |
URL path of the sound cue resource. |
Yes |
Store custom audio in OSS and ensure that its URL is accessible over HTTP or HTTPS. To create a bucket, see Create a bucket. For file upload and sharing, see Console quick start. Audio requirements:
Important
You are legally responsible for the copyright of the audio you upload. |
Examples
<speak>
一匹马受了惊吓<soundEvent src="http://nls.alicdn.com/sound-event/horse-neigh.wav"/>人们四散躲避
</speak>Audio effect: SSML-sound-event.mp3
Tag relationships
<soundEvent> is an empty tag and cannot contain any other tags.
<say-as>
This tag indicates the information type of the text within it, which allows the text to be pronounced according to the default method for that type.
<say-as interpret-as="string">text</say-as>Attributes
|
Attribute |
Type |
Value |
Required |
Description |
|
|
String |
cardinal, digits, telephone, name, address, id, characters, punctuation, date, time, currency, or measure |
Yes |
Indicates the information type of the text within the tag:
|
Examples
cardinal
<speak>
<say-as interpret-as="cardinal">12345</say-as>
</speak>Audio effect in Chinese: SSML-say-as_Cardinal.mp3
<speak>
<say-as interpret-as="cardinal">10234</say-as>
</speak>Audio effect in English: en-SSML-say-as_cardinal.mp3
digits
<speak>
<say-as interpret-as="digits">12345</say-as>
</speak>Audio effect in Chinese: SSML-say-as_digit.mp3
<speak>
<say-as interpret-as="digits">10234</say-as>
</speak>Audio effect in English: en-SSML-say-as_digits.mp3
telephone
<speak>
<say-as interpret-as="telephone">12345</say-as>
</speak>Audio effect in Chinese: SSML-say-as_Telephone.mp3
<speak>
<say-as interpret-as="telephone">10234</say-as>
</speak>Audio effect in English: en-SSML-say-as_telephone.mp3
name
<speak>
她的曾用名是<say-as interpret-as="name">曾小凡</say-as>
</speak>Audio effect: SSML-say-as_Name.mp3
address
<speak>
<say-as interpret-as="address">富路国际1号楼3单元304</say-as>
</speak>Audio effect: SSML-say-as_Address.mp3
id
<speak>
<say-as interpret-as="id">myid_1998</say-as>
</speak>Audio effect: SSML-say-as_id.mp3
characters
<speak>
<say-as interpret-as="characters">希腊字母αβ</say-as>
</speak>Audio effect in Chinese: SSML-say-as_characters.mp3
<speak>
<say-as interpret-as="characters">*b+3.c$=α</say-as>
</speak>Audio effect in English: en-SSML-say-as_characters.mp3
punctuation
<speak>
<say-as interpret-as="punctuation"> -./:;</say-as>
</speak>Audio effect: SSML-say-as_punctuation.mp3
date
<speak>
<say-as interpret-as="date">1000-10-10</say-as>
</speak>Audio effect in Chinese: SSML-say-as_date.mp3
<speak>
<say-as interpret-as="date">10-01-2020</say-as>
</speak>Audio effect in English: en-SSML-say-as_date.mp3
time
<speak>
<say-as interpret-as="time">5:00am</say-as>
</speak>Audio effect in Chinese: SSML-say-as_time.mp3
<speak>
<say-as interpret-as="time">0500</say-as>
</speak>Audio effect in English: en-SSML-say-as_time.mp3
currency
<speak>
<say-as interpret-as="currency">13,000,000.00RMB</say-as>
</speak>Audio effect in Chinese: SSML-say-as_currency.mp3
<speak>
<say-as interpret-as="currency">$1,000.01</say-as>
</speak>Audio effect in English: en-SSML-say-as_currency.mp3
measure
<speak>
<say-as interpret-as="measure">100m12cm6mm</say-as>
</speak>Audio effect in Chinese: SSML-say-as_measure.mp3
<speak>
<say-as interpret-as="measure">1,000.01kg</say-as>
</speak>Audio effect in English: en-SSML-say-as_measure.mp3
Tag relationships
The <say-as> tag can contain text and the <vhml/> tag.
Comprehensive example
The following example combines multiple SSML tags. Audio effect: Comprehensive example.mp3.
-
The narrator's voice in the sample audio is a custom voice that is not yet publicly available. To try custom voices, see Speech Synthesis Voice Customization.
-
Each short-text synthesis request can contain no more than 300 characters and one
<speak>root element. The following example contains multiple<speak>fragments. Submit the fragments separately for short-text synthesis.
<speak>
相传北宋年间，
<say-as interpret-as="date">1121-10-10</say-as>，
<say-as interpret-as="address">开封城</say-as>
郊外的早晨笼罩在一片
<sub alias="双十一">11.11</sub>
前买买买的欢乐海洋中。一支运货的骡队刚进入城门
<soundEvent src="http://nls.alicdn.com/sound-event/bell.wav"/>
一个肤白貌美
<phoneme alphabet="py" ph="de5">地</phoneme>
姑娘便拦下第一排的小哥<say-as interpret-as="name">阿发。</say-as>
</speak>
<speak voice="xiaomei">
“亲，本店今日特惠，鞋履全场
<say-as interpret-as="digits">199</say-as>
减
<say-as interpret-as="cardinal">100</say-as>，
走过路过不要错过”。
</speak>
<speak voice="sicheng" rate="150">
“不啦不啦，赶着上货，已经
<say-as interpret-as="time">09:59:59</say-as>
了，再晚就供应链断裂了”。
</speak>
<speak>
<say-as interpret-as="name">阿发</say-as>
擦了擦汗，带着运货队伍，径直穿过闹巷，耳边充斥着各种叫卖声：
</speak>
<speak voice="ninger" rate="200">
最新花色现染布匹，买两尺送一尺；
</speak>
<speak voice="xiaobei">
爆款纱帽头盔，7天无理由退货；
</speak>
<speak voice="sijia">
专治大小方脉，调理男人妇人疑难杂症。
</speak>
<speak>
突然，一匹马不知怎么受了惊，在路上嘶鸣狂奔
<soundEvent src="http://nls.alicdn.com/sound-event/horse-neigh.wav"/>
一个孩子也吓坏了，跌跌撞撞地扑向大人怀里
<break time="50ms"/>大喊道：
</speak>
<speak voice="sitong" rate="150">
“妈妈，妈妈！”
</speak>
<speak>
这时，
<say-as interpret-as="name">阿发</say-as>
心想
</speak>
<speak effect="robot" pitch="-100">
“吓死宝宝了！”
</speak>
<speak>
于是他赶紧捂住了
<phoneme alphabet="py" ph="he2 bao1">钱包</phoneme>，
继续赶路送货。一路上，
<say-as interpret-as="address">开封城</say-as>
的繁荣景象给
<say-as interpret-as="name">阿发</say-as>
留下了深刻的印象。
</speak>
<speak bgm="http://nls.alicdn.com/bgm/2.wav" backgroundMusicVolume="30" rate="-200">
物换星移，繁华落尽，于是他在购物狂欢之余握起画笔，勾勒出一幅长卷，并命名为《清明上河图》。
</speak>