Sentences
This document provides technical details about the sentence speech evaluation feature to assist developers with product integration.
Question type description
Scores English sentences and supports audio comparison and real-time feedback.
Provides feedback on the overall score, fluency score, accuracy score, and integrity score. It also evaluates the pronunciation of each word, checking for issues with word stress, intonation, thought-group pauses, liaisons, extra or missed words, and pauses.
Evaluation request parameters
Field | Type | Optional | Description |
coreType | string | Required | en.sent.score indicates English sentence evaluation. |
refText | string | Required | The reference text for scoring. It supports full sentences with punctuation, such as "I want to know the past and present of Hong Kong." |
rank | int | Optional | The scoring scale. You can set this to any value. The final score is recalculated based on its ratio to 100. |
attachAudioUrl | int | Optional | Specifies whether to return the audio URL in the evaluation result. |
symbol | int | Optional | If there are multiple consecutive punctuation marks before a word, only the first one is displayed. If there are multiple consecutive punctuation marks at the end of a word, only the three closest to the word are output. Set to 0 or 1. The default is 0, which disables this feature. |
outputPhones | int | Optional | The output includes the phoneme score for each word. Set to 0 or 1. The default is 0, which disables this feature. |
phdet | int | Optional | Phoneme error detection. This field is only valid if outputPhones is set to 1. Set to 0 or 1. The default is 0, which disables this feature. |
precision | double | Optional | Sets the scoring precision. Only 0.1, 0.5, or 1 are supported. The default is 1. If you set a value other than 0.1 or 0.5, it is treated as 1. |
typeThres | int | Optional | Scoring strictness. A higher score has less impact. 1: Strict. 2: Loose. 3: Very strict. 4: Very loose. The default is 0, which disables this feature. |
feedback | int | Optional | 1: Enables real-time feedback. This provides the text position corresponding to the currently read audio in real time. 0: Default value, which disables this feature. |
enableCompareAudio | int | Optional | Enables the audio comparison feature. Returns discrete point information. You must also set std_audio_url. |
std_audio_url | string | Optional | The path to the standard audio file. You must coordinate with the platform to obtain the corresponding list. |
accent | string | Optional | "accent":"en" specifies evaluation for British English pronunciation. |
auto_rhythm | int | Optional | 1: Enables liaison detection. 0: Default value, which disables this feature. When this feature is enabled, words in the sentence that previously had low liaison scores will be scored normally. |
Parameter example
{
"coreType": "en.sent.score",
"refText": "let us play basketball, are you ok?(t:1).",
"accent": 1,
"rank": 100,
"outputPhones":1,
"phdet": 1,
"attachAudioUrl":1
}Example of a returned result
{
"tokenId": "60ffd5bc88bf5b0000040001", // The unique ID of the client request
"recordId": "11ebeebf6124c82ebcf3a14812619e1a", // The unique ID of the data in the cloud. Save this to your product database to help with troubleshooting.
"applicationId": "aXXX", // The evaluation AppKey
"audioUrl": "http:\/\/xxx.xxx.ssapi.cn:8080\/axxx\/11ebeebf6124c82ebcf3a14812619e1a", // The download URL for the user's audio
"dtLastResponse": "2021-07-27 17:45:34:615", // The timestamp when the data was returned
"connect": {
"param": {
"app": {
"timestamp": "1627379132",
"applicationId": "aXXX", // The evaluation AppKey
"sig": "7cb761d87ec0078199312fa4eff63f07ef1e72ea"
},
"sdk": {
"os": "6.2(9200)1",
"product": "",
"source": 7,
"protocol": 1,
"os_version": "6.2(9200)",
"arch": "586",
"version": 16778752
}
},
"cmd": "connect"
},
"params": {
"app": {
"timestamp": "1627379132",
"userId": "ssound_text", // A custom string from the product. We recommend using a different user ID for each user to help locate issues.
"sig": "7cb761d87ec0078199312fa4eff63f07ef1e72ea",
"connect_id": "60ffd5bc88bf5b0000060002",
"clientId": "",
"applicationId": "aXXX" // The product's AppKey
},
"audio":{ // The audio format parameters for the scoring request
"sampleRate":16000, // Sample rate
"channel":1, // Sound channel. 1 is mono, 2 is stereo. Evaluation only supports mono.
"sampleBytes":2,
"audioType":"ogg" // Audio type
},
"request": {
"request_id": "60ffd5bc88bf5b0000050001",
"phdet": 1, // Phoneme error detection. 1 enables this feature. The default is 0, which disables it. Only 0 and 1 are valid.
"rank": 100, // Scoring scale. The default is 100 for a percentage-based scale. You can set this to any value. The final score is recalculated based on its ratio to 100.
"tokenId": "60ffd5bc88bf5b0000040001",
"outputPhones": 1, // The output includes the phoneme score for each word. Set to 0 or 1. The default is 0, which disables this feature.
"refText": " let us play basketball, are you ok?(t:1).", // The text for the evaluation request
"coreType": "en.sent.score" // The question type for the evaluation request
}
},
"refText": " let us play basketball, are you ok?(t:1).",
"result": {
"overall": 76, // The overall evaluation score
"forceout": 0,
"precision": 1,
"systime": 4063, // The duration of the evaluation calculation in the cloud
"res": "eng.snt.online.1.0",
"rank": 100,
"rhythm": { // Rhythm-related
"stress": 100, // The total stress score. Checks if words that require stress are stressed.
"overall": 50, // A comprehensive score for stress, thought-group pauses, and intonation.
"tone": 100, // The total tone score. Checks if the pronunciation at the end of the sentence has the required rising intonation.
"sense": 0 // The total thought-group score. Checks if the pronunciation within thought groups is fluent.
},
"fluency": { // Fluency-related
"pause": 0, // The number of pauses
"overall": 94, // The fluency score
"speed": 1 // Speech rate. The default is 0. It is also 0 for missed words. 0: Slow, 1: Normal, 2: Fast.
},
"pron": 73, // Pronunciation score (same as accuracy)
"wavetime": 3990, // The duration of the user's recording
"accuracy": 73, // Pronunciation score (same as pron)
"details": [ // The scoring details for each word
{
"stressref": 0,
"tonescore": 0,
"dur": 520,
"char": "let",
"liaisonref": 0, // Liaison flag. 1 indicates that a liaison is set for the word. 0 indicates that no liaison is needed. The default is 0 for all words.
"liaisonscore": 0, // Returns 1 or 0. 1 means the actual pronunciation matches the setting. 0 means it does not match. Check if the value is null before getting it.
"senseref": 0, // Thought-group pause flag. 1 indicates a pause is set for the word. 0 indicates no pause is needed. The default is 1 at the end of a sentence.
"start": 620, // The start time of the word in the audio, in ms
"fluency": 99,
"score": 90, // The pronunciation score of the word
"stressscore": 0, // Stress feedback (0 or 1)
"toneref": 0, // Intonation feedback
"sensescore": 1, // Thought-group pause feedback (0 or 1)
"phone": [ // The phoneme scoring details for the current word
{
"phid": "1", // The position of this phoneme in the current word's phonetic transcription. This field is output only when phdet is set.
"ph2alpha": "l", // The letter in the word that corresponds to the current phoneme. Developers do not need to calculate this from phid. This field is output only when phdet is set.
"start": 620,
"char": "l", // Phoneme
"score": 83, // Phoneme score
"end": 880,
"pherr": 0 // Checks for phoneme errors (0 or 1). This field is output only when phdet is set.
},
{
"phid": "2",
"ph2alpha": "e",
"start": 880,
"char": "eh",
"score": 95,
"end": 1020,
"pherr": 0
},
{
"phid": "3",
"ph2alpha": "t",
"start": 1020,
"char": "t",
"score": 93,
"end": 1140,
"pherr": 0
}
],
"end": 1140, // The end time of the word in the audio, in ms
"accent": "ea" // If the accent parameter is specified in the scoring request for British/American English, this parameter is not returned. Otherwise, the accent field is included in the result.
},
{
"stressref": 0,
"tonescore": 0,
"dur": 320,
"char": "us",
"liaisonref": 0,
"liaisonscore": 0,
"senseref": 0,
"start": 1200,
"fluency": 69,
"score": 90,
"stressscore": 0,
"toneref": 0,
"sensescore": 1,
"phone": [
{
"phid": "1",
"ph2alpha": "u",
"start": 1200,
"char": "ah",
"score": 92,
"end": 1400,
"pherr": 0
},
{
"phid": "2",
"ph2alpha": "s",
"start": 1400,
"char": "s",
"score": 88,
"end": 1520,
"pherr": 0
}
],
"end": 1520,
"accent": "ea"
},
{
"stressref": 0,
"tonescore": 0,
"dur": 240,
"char": "play",
"liaisonref": 0,
"liaisonscore": 0,
"senseref": 0,
"start": 1580,
"fluency": 95,
"score": 42,
"stressscore": 0,
"toneref": 0,
"sensescore": 0,
"phone": [
{
"phid": "1",
"ph2alpha": "p",
"start": 1580,
"char": "p",
"score": 0,
"end": 1640,
"pherr": 1
},
{
"phid": "2",
"ph2alpha": "l",
"start": 1640,
"char": "l",
"score": 79,
"end": 1700,
"pherr": 0
},
{
"phid": "3_4",
"ph2alpha": "ay",
"start": 1700,
"char": "ey",
"score": 89,
"end": 1820,
"pherr": 0
}
],
"end": 1820,
"accent": "ea"
},
{
"stressref": 0,
"tonescore": 0,
"dur": 760,
"char": "basketball",
"liaisonref": 0,
"liaisonscore": 0,
"senseref": 0,
"start": 1820,
"fluency": 96,
"score": 72,
"stressscore": 0,
"toneref": 0,
"sensescore": 1,
"phone": [
{
"phid": "1",
"ph2alpha": "b",
"start": 1820,
"char": "b",
"score": 98,
"end": 1900,
"pherr": 0
},
{
"phid": "2",
"ph2alpha": "a",
"start": 1900,
"char": "aa",
"score": 97,
"end": 2000,
"pherr": 0
},
{
"phid": "3",
"ph2alpha": "s",
"start": 2000,
"char": "s",
"score": 97,
"end": 2120,
"pherr": 0
},
{
"phid": "4",
"ph2alpha": "k",
"start": 2120,
"char": "k",
"score": 96,
"end": 2200,
"pherr": 0
},
{
"phid": "5",
"ph2alpha": "e",
"start": 2200,
"char": "ih",
"score": 95,
"end": 2260,
"pherr": 0
},
{
"phid": "6",
"ph2alpha": "t",
"start": 2260,
"char": "t",
"score": 80,
"end": 2400,
"pherr": 0
},
{
"phid": "7",
"ph2alpha": "b",
"start": 2400,
"char": "b",
"score": 61,
"end": 2460,
"pherr": 0
},
{
"phid": "8",
"ph2alpha": "a",
"start": 2460,
"char": "ao",
"score": 25,
"end": 2520,
"pherr": 0
},
{
"phid": "9_10",
"ph2alpha": "ll",
"start": 2520,
"char": "l",
"score": 49,
"end": 2580,
"pherr": 0
}
],
"end": 2580,
"accent": "en"
},
{
"stressref": 0,
"tonescore": 0,
"dur": 120,
"char": "are",
"liaisonref": 0,
"liaisonscore": 0,
"senseref": 0,
"start": 2700,
"fluency": 100,
"score": 93,
"stressscore": 0,
"toneref": 0,
"sensescore": 0,
"phone": [
{
"phid": "1_2_3",
"ph2alpha": "are",
"start": 2700,
"char": "aa",
"score": 93,
"end": 2820,
"pherr": 0
}
],
"end": 2820,
"accent": "en"
},
{
"stressref": 0,
"tonescore": 0,
"dur": 200,
"char": "you",
"liaisonref": 0,
"liaisonscore": 0,
"senseref": 0,
"start": 2820,
"fluency": 100,
"score": 46,
"stressscore": 0,
"toneref": 0,
"sensescore": 0,
"phone": [
{
"phid": "1",
"ph2alpha": "y",
"start": 2820,
"char": "y",
"score": 74,
"end": 2940,
"pherr": 0
},
{
"phid": "2_3",
"ph2alpha": "ou",
"start": 2940,
"char": "uh",
"score": 32,
"end": 3020,
"pherr": 0
}
],
"end": 3020,
"accent": "ea"
},
{
"stressref": 0,
"tonescore": 1,
"dur": 560,
"char": "ok",
"liaisonref": 0,
"liaisonscore": 0,
"senseref": 1,
"start": 3020,
"fluency": 100,
"score": 75,
"stressscore": 0,
"toneref": 1,
"sensescore": 1,
"phone": [
{
"phid": "1",
"ph2alpha": "o",
"start": 3020,
"char": "ow",
"score": 49,
"end": 3160,
"pherr": 0
},
{
"phid": "2",
"ph2alpha": "k",
"start": 3160,
"char": "k",
"score": 94,
"end": 3320,
"pherr": 0
},
{
"phid": "",
"char": "ey",
"start": 3320,
"score": 84,
"end": 3580,
"pherr": 0
}
],
"end": 3580,
"accent": "ea"
}
],
"info": {
"tipId": 0,
"clip": 0,
"snr": 33.928673,
"volume": 88
},
"statics": [
{
"score": 70,
"char": "l",
"count": 3
},
{
"score": 95,
"char": "eh",
"count": 1
},
{
"score": 87,
"char": "t",
"count": 2
},
{
"score": 92,
"char": "ah",
"count": 1
},
{
"score": 92,
"char": "s",
"count": 2
},
{
"score": 0,
"char": "p",
"count": 1
},
{
"score": 86,
"char": "ey",
"count": 2
},
{
"score": 79,
"char": "b",
"count": 2
},
{
"score": 95,
"char": "aa",
"count": 2
},
{
"score": 95,
"char": "k",
"count": 2
},
{
"score": 95,
"char": "ih",
"count": 1
},
{
"score": 25,
"char": "ao",
"count": 1
},
{
"score": 74,
"char": "y",
"count": 1
},
{
"score": 32,
"char": "uh",
"count": 1
},
{
"score": 49,
"char": "ow",
"count": 1
}
],
"delaytime": 30,
"integrity": 100,
"pretime": 1,
"version": "0.0.80.2021.6.29.10:28:00"
},
"eof": 1,
"cloud_platform": {
"origin_audio_length": 15366
}
}Description of returned results
Key | Type | Description | Usage suggestion |
recordId | string | The unique ID of the audio file. | For reference. |
tokenId | string | The unique ID of the user request. | For reference. |
audioUrl | string | The URL of the user's audio. By default, it is stored in the cloud for only one month. We recommend saving it to your product server. | For reference. |
applicationId | string | The authorized account, which is the provided AppKey. | Internal parameter. |
dtLastResponse | string | The time when the cloud responded. | Internal parameter. |
eof | int | 0: The response is not complete. More results will follow. 1: All results for this evaluation have been returned. | |
refText | string | The input text for evaluation. | For reference. |
params | object | Echoes the evaluation request parameters sent by the client. | |
- app | object | ||
- - applicationId | string | The authorized account, which is the provided evaluation AppKey. | |
- - userId | string | The user ID from the product. This can be a custom string. We recommend using a unique user ID for each user to help with troubleshooting. | |
- - timestamp | string | The start time. | |
- - sig | string | The verification code. | |
- request | object | The node for scoring request parameters. | |
- audio | object | The node for audio parameters in the scoring request. | |
result | object | The evaluation result. | |
- version | string | The version of the engine, including the release date. | Internal parameter. |
- rank | int | The scoring scale. | Internal parameter. |
- precision | float | The scoring precision. | Internal parameter. |
- res | string | The name of the resource used for evaluating this question type. | Internal parameter. |
- forceout | int | Internal parameter. | |
- pretime | Int | The time taken by the cloud to call the Start interface, in ms. | Internal parameter. |
- systime | int | The total time taken for the entire evaluation process in the cloud, in ms. | Internal parameter. |
- delaytime | int | The time from when the audio feed ends to when the result is obtained in the cloud, in ms. | Internal parameter. |
- wavetime | int | The duration of the audio, in ms. | For reference. |
- overall | float | The overall score. | Recommended field. |
- pron | float | The pronunciation score. | Internal parameter. |
- integrity | float | The integrity score. | Recommended field. |
- accuracy | float | The pronunciation score (same as the pron field). | Recommended field. |
- fluency | object | The fluency score. If the integrity score is 0, the fluency score is also 0. | Recommended field. |
- - overall | float | The overall fluency score (0–100). | Recommended field. |
- - pause | int | The number of pauses. | For reference. |
- - speed | int | Speech rate. The default is 0. It is also 0 for missed words. 0: Slow, 1: Normal, 2: Fast. | For reference. |
- rhythm | object | The rhythm score. | For reference. |
- - overall | float | The overall rhythm score (0–100). sense accounts for 50%, while stress and tone each account for 25%. | For reference. |
- - sense | float | The thought-group score (0–100). | Reserved for future use. |
- - stress | float | The stress score (0–100). | Reserved for future use. |
- - tone | float | The sentence intonation score (0–100). | Reserved for future use. |
-realtime_details | array | Real-time feedback information. | |
-- char | string | The word. | For reference. |
-- dp_type | int | 0: Read normally. 1: Missed or not read. 2: Read repeatedly. | For reference. |
- details | array | Detailed scores for words. | Recommended field. |
- - char | string | The word. | Recommended field. |
- - score | int | The pronunciation score of the word. | Recommended field. |
- - fake_pron | int | This field is output only when the word is not in the dictionary (out-of-vocabulary). To evaluate out-of-vocabulary words, you can pass their phonetic transcription using refText. | |
- - start | int | The start time of the word in the audio, in milliseconds (ms). | For reference. |
- - end | int | The end time of the word in the audio, in milliseconds (ms). | For reference. |
- - dur | int | The pronunciation duration of the word. | For reference. |
- - fluency | float | The fluency score (0–100). | Reserved for future use. |
- - stressref | int | The stress flag. | For reference. |
- - stressscore | int | Reread Status (0 or 1) | For reference. |
- - toneref | int | The rising intonation flag. | For reference. |
- - tonescore | int | The intonation score (0 or 1). | For reference. |
- - senseref | int | The thought-group pause flag. | For reference. |
- - sensescore | int | The thought-group pause score (0 or 1). | For reference. |
- - liaisonref | int | Continuous Read ID | For reference. |
- - liaisonscore | int | The liaison score (0 or 1). | For reference. |
- - dp_type | int | Normal reading (the dp_type field is not output), missed reading (1), or repeated reading (2). | Normal: This field is not output. Missed word: dp_type is 1. Repeated word: dp_type is 2. |
- - is_pause | int | The pause flag. | If there is a pause after a word, the JSON field for that word includes "is_pause": 1. This field does not appear under normal circumstances. |
- - accent | string | This field is not returned if accent is configured in the request parameters. Otherwise, this field is returned as "accent":"ea". | |
- - phone | array | The phoneme scores for the current word. | This field is present only when outputPhones is set. |
- - - char | string | The phoneme. If accent is set in the request parameters, this field contains standard British or American English phonemes. Otherwise, it contains internal mapped phonemes. | For reference. |
- - - score | float | The phoneme pronunciation score (0–100). | For reference. |
- - - phid | int | The position of this phoneme in the current word's phonetic transcription. | This field is output only when phdet is set. |
- - - pherr | int | The result of phoneme error detection. | This field is output only when phdet is set. |
- - - ph2alpha | string | The letter in the word that corresponds to the current phoneme. You do not need to calculate this from phid. | This field is output only when phdet is set. |
- statics | array | Internal parameter. | |
- - char | string | Phoneme. | Internal parameter. |
- - count | int | The number of times the phoneme appears. | Internal parameter. |
- - score | int | The average pronunciation score for the phoneme (0–100). | Internal parameter. |
- info | object | ||
- - snr | float | The signal-to-noise ratio (SNR). A higher value means clearer audio. The range is 0–40 dB. If this parameter affects the score, a corresponding tipId value is set. | For reference. |
- - clip | float | The audio volume is too high, causing clipping. The range is 0–1. If this parameter affects the score, a corresponding tipId value is set. | Internal parameter. |
- - volume | int | The recording volume. The range is 0–180 dB. | For reference. |
- - tipId | int | Audio quality. | Recommended field. |
For more information, see tipId message descriptions. | |||
- result_compare | object | Audio comparison result. | |
- - list | object | Result list. | |
- - - speed | object | Speech rate value. | |
- - - - std | array | Discrete points of the speech rate for the standard audio. The value is a float in the range of 0–1. | For reference. |
- - - - simu | array | Discrete points of the speech rate for the imitation audio. The value is a float in the range of 0–1. | For reference. |
- - - f0 | object | Intonation value. | |
- - - - std | array | Discrete points of the pitch for the standard audio. The value is a float in the range of 0–1. | For reference. |
- - - - simu | array | Discrete points of the pitch for the imitation audio. The value is a float in the range of 0–1. | For reference. |
- - - energy | object | Volume value. | |
- - - - std | char | Discrete points of the volume for the standard audio. The value is a float in the range of 0–1. | For reference. |
- - - - simu | char | Discrete points of the volume for the imitation audio. The value is a float in the range of 0–1. | For reference. |
- - correl_score | object | Correlation score. | |
- - - f0 | int | Intonation correlation score. | For reference. |
- - - speed | int | Speech rate correlation score. | For reference. |
- - - energy | int | Volume correlation score. | For reference. |
- - details | array | Information about the corresponding words in the sentence. | |
- - - char | string | The word. | Recommended field. |
- - - index | int | The index of the word, starting from 0. | Recommended field. |
- - - end | int | End of the audio. | For reference. |
- - - start | int | Start of the audio. | For reference. |
Suggestions for displaying evaluation results
Dimension | Description | Details |
overall | Overall score | An overall evaluation of the pronunciation for the entire sentence. |
pron | Pronunciation score | |
integrity | Integrity | Evaluates the completeness of the recorded content compared to the target sentence. It only checks if the sentence was read completely. |
accuracy | Accuracy | Evaluates the pronunciation of the content that was read. It is independent of integrity. |
fluency.overall | Fluency | Evaluates the fluency of the recording. It is independent of the content that was read. |
details[i].score | Word score | Evaluates the pronunciation of each word in the sentence. Check if the value is null before getting it. |
details[i].dp_type | Extra or missed words | Indicates if a word was repeated or missed. See the returned result description for the corresponding question type. |
details[i].is_pause | Pause field | Indicates a pause after a word is read. |
details[i].speed | Indicates speech rate | Indicates the speech rate. See the returned result description for the corresponding question type. |
details[i].fake_pron | Fake pronunciation | Indicates if the word is in the dictionary. This field is output only when the word is not in the dictionary (out-of-vocabulary). To evaluate out-of-vocabulary words, you can pass their phonetic transcription using refText. |
details[i].stressref | Stress flag | 1 indicates that stress is set for the word. 0 indicates that no stress is needed. The default is 0 for all words. |
details[i].stressscore | Reread Score | Returns 1 or 0. 1 means the actual stress matches the setting. 0 means it does not match. Check if the value is null before getting it. |
details[i].toneref | Intonation flag | 1 indicates rising intonation is set for the word. 0 indicates falling intonation is set. If not set, the default is 0 at the end. |
details[i].tonescore | Intonation score | Returns 1 or 0. 1 means the actual intonation matches the setting. 0 means it does not match. Check if the value is null before getting it. |
details[i]. senseref | Thought-group pause flag | 1 indicates a thought-group pause is set for the word. 0 indicates no pause is needed. The default is 1 at the end. |
details[i]. sensescore | Thought-group pause score | Returns 1 or 0. 1 means the actual pause matches the setting. 0 means it does not match. Check if the value is null before getting it. |
details[i]. liaisonref | Liaison flag | 1 indicates that a liaison is set for the word. 0 indicates that no liaison is needed. The default is 0 for all words. |
details[i]. liaisonscore | Liaison score | Returns 1 or 0. 1 means the actual liaison matches the setting. 0 means it does not match. Check if the value is null before getting it. |
Returned result on evaluation failure
If the result structure in the response is empty, the evaluation has failed.
You can retrieve the errId and error from the response. Your product can then use the corresponding error code for further analysis and processing.
Error codes
For more information about error codes, see Evaluation error codes.