Picture description
This document provides technical information about the English picture description speech evaluation for developers who want to integrate this feature.
Scenarios for this question type:
A picture is provided, and the user describes its content in English.
Parameter description
Field | Type | Optional | Description |
coreType | String | Required | en.pict.score indicates the English picture description question type. |
rank | int | Optional | The scoring system for the total score. |
precision | double | Optional | Sets the scoring precision. Supported values are 0.1, 0.5, and 1. The default value is 1. If you set the value to anything other than 0.1 or 0.5, the system uses 1. |
grade | int | Optional | Specifies the education level. Supported values are 1 and 2. The default value is 1.1: Indicates the junior high school level. A full score requires covering about 90% of the key points.2: Indicates the high school level. A high score requires covering about 80% of the key points because the topics are more difficult. |
lm | array | Required | The sample compositions for the picture. Each text subfield represents one sample composition. Note: The lm field is the lowercase form of LM. |
phones | object | Optional | Specifies the pronunciation of a word. If a word has multiple pronunciations, separate them with a pipe symbol (|). Use a space between each phoneme, and between phonemes and the syllable separator (-). Example: "phones":{"directory":"d ih - 'r eh k - tr ih|d ax - 'r eh k - t ax - r ih"}. |
points | array | Optional | Sets the key points for the answer. Each key point is a separate text object. |
attachAudioUrl | int | Optional | Specifies whether to return the audio URL in the evaluation response. |
rateScale | float | Optional | The scoring leniency. The value ranges from 0.8 to 1.5. The default value is 1.0. If the value is less than 0.8, the system uses 0.8. If the value is greater than 1.5, the system uses 1.5. This parameter acts as a multiplier. A higher value results in a higher score. |
Parameter example
{
"coreType": "en.pict.score",
"rank": 100,
"precision": 0.5,
"attachAudioUrl":1,
"lm": [
{
"text": "My aunt Judy was a single mother of fifty, working as a secretary. When doctors discovered she had a very serious heart disease, she didn't give in. She began studying medical reports and found an article in a magazine about a well known heart doctor, Dr. Michael Baker, of Houston, Texas. The article said Dr. Baker's fees were very high; Aunt Judy couldn't afford them. But could he tell her of someone whose fee she could pay? So Aunt Judy wrote to him. She simply listed her reasons for wanting to live: her three children, who would be independent in three or four more years, her little girl dream of travelling and seeing the world. There wasn't a word of self pity only warmth and humor and the joy of living. She mailed the letter, not really expecting an answer. A few days later, Aunt Judy rushed into my house, and read aloud a letter: your beautiful letter moved me very deeply. If you can come to Houston, there will be no charge for either the hospital or the operation. Michael Baker. That was seven years ago. Since then, Aunt Judy has been around the world. Her three children are happily married."
}
],
"points": [
{
"text": "My aunt Judy was a single mother of fifty, working as a secretary."
},
{
"text": "When doctors discovered she had a very serious heart disease, she didn't give in."
},
{
"text": "She began studying medical reports and found an article in a magazine about a well known heart doctor, Dr. Michael Baker, of Houston, Texas."
},
{
"text": "The article said Dr. Baker's fees were very high; Aunt Judy couldn't afford them."
},
{
"text": "But could he tell her of someone whose fee she could pay? So Aunt Judy wrote to him."
},
{
"text": "She simply listed her reasons for wanting to live: her three children, who would be independent in three or four more years, her little girl dream of travelling and seeing the world."
},
{
"text": "There wasn't a word of self pity only warmth and humor and the joy of living."
},
{
"text": "She mailed the letter, not really expecting an answer."
},
{
"text": "A few days later, Aunt Judy rushed into my house, and read aloud a letter: your beautiful letter moved me very deeply. If you can come to Houston, there will be no charge for either the hospital or the operation. Michael Baker."
},
{
"text": "That was seven years ago. Since then, Aunt Judy has been around the world. Her three children are happily married."
}
]
}Response example
{
"tokenId": "612f3d6c88bf5b0000040001",
"applicationId": "a148",
"audioUrl": "http:\/\/files.cloud.ssapi.cn:8080\/a148\/11ec0b00cf7d709f94a4a14872616788",
"connect": {
"param": {
"app": {
"timestamp": "1630485868",
"applicationId": "a148",
"sig": "342befbf5b7ccb03d4562fad7b21d7261908d5da"
},
"sdk": {
"os": "6.2(9200)1",
"product": "",
"source": 7,
"protocol": 1,
"os_version": "6.2(9200)",
"arch": "586",
"version": 16778752
}
},
"cmd": "connect"
},
"params": {
"app": {
"timestamp": "1630485868",
"userId": "ssound_text",
"sig": "342befbf5b7ccb03d4562fad7b21d7261908d5da",
"connect_id": "612f3d6c88bf5b0000060002",
"clientId": "",
"applicationId": "a148"
},
"audio": {
"sampleBytes": 2,
"audioType": "ogg",
"sampleRate": 16000,
"channel": 1
},
"request": {
"lm": [
{
"text": "My aunt Judy was a single mother of fifty, working as a secretary. When doctors discovered she had a very serious heart disease, she didn't give in. She began studying medical reports and found an article in a magazine about a well known heart doctor, Dr. Michael Baker, of Houston, Texas. The article said Dr. Baker's fees were very high; Aunt Judy couldn't afford them. But could he tell her of someone whose fee she could pay? So Aunt Judy wrote to him. She simply listed her reasons for wanting to live: her three children, who would be independent in three or four more years, her little girl dream of travelling and seeing the world. There wasn't a word of self pity only warmth and humor and the joy of living. She mailed the letter, not really expecting an answer. A few days later, Aunt Judy rushed into my house, and read aloud a letter: your beautiful letter moved me very deeply. If you can come to Houston, there will be no charge for either the hospital or the operation. Michael Baker. That was seven years ago. Since then, Aunt Judy has been around the world. Her three children are happily married."
}
],
"tokenId": "612f3d6c88bf5b0000040001",
"request_id": "612f3d6c88bf5b0000050001",
"precision": 0.5,
"attachAudioUrl": 1,
"coreType": "en.pict.score",
"rank": 100,
"points": [
{
"text": "My aunt Judy was a single mother of fifty, working as a secretary."
},
{
"text": "When doctors discovered she had a very serious heart disease, she didn't give in."
},
{
"text": "She began studying medical reports and found an article in a magazine about a well known heart doctor, Dr. Michael Baker, of Houston, Texas."
},
{
"text": "The article said Dr. Baker's fees were very high; Aunt Judy couldn't afford them."
},
{
"text": "But could he tell her of someone whose fee she could pay? So Aunt Judy wrote to him."
},
{
"text": "She simply listed her reasons for wanting to live: her three children, who would be independent in three or four more years, her little girl dream of travelling and seeing the world."
},
{
"text": "There wasn't a word of self pity only warmth and humor and the joy of living."
},
{
"text": "She mailed the letter, not really expecting an answer."
},
{
"text": "A few days later, Aunt Judy rushed into my house, and read aloud a letter: your beautiful letter moved me very deeply. If you can come to Houston, there will be no charge for either the hospital or the operation. Michael Baker."
},
{
"text": "That was seven years ago. Since then, Aunt Judy has been around the world. Her three children are happily married."
}
]
}
},
"recordId": "11ec0b00cf7d709f94a4a14872616788",
"eof": 1,
"result": {
"keypoints_hit": [
0.25,
0,
0,
0.166667,
0.142857,
0.1875,
0.222222,
0,
0.090909,
0.5
],
"overall": 11.5,
"wavetime": 7990,
"precision": 0.5,
"systime": 8098,
"res": "eng.retell.mfcc.0.1",
"version": "0.0.80.2021.8.27.04:29:39",
"pron": 85.5,
"info": {
"tipId": 0,
"clip": 0,
"snr": 14.624933,
"volume": 84
},
"fluency": 84,
"rank": 100,
"rhythm": 0,
"integrity": 8,
"pretime": 586,
"delaytime": 375
},
"dtLastResponse": "2021-09-01 16:44:34:784",
"cloud_platform": {
"origin_audio_length": 30457
}
}Response description
Key | Type | Description | Suggestion |
recordId | string | The unique ID of the audio file. | For reference |
tokenId | string | The unique ID of the request. | For reference |
applicationId | string | The provided authorization account, which is the AppKey. | Internal parameter |
audioUrl | string | The URL of the user's audio. By default, the URL is stored in the cloud for one month. Save the audio to your product server. | For reference |
dtLastResponse | string | The timestamp of the server response. | Internal parameter |
eof | Int | 0: The response is not complete. More results will follow.1: All results for this evaluation have been returned. | Internal parameter |
params | object | Echoes the evaluation request parameters from the client. | |
- app | object | ||
- - applicationId | string | The provided authorization account, which is the AppKey. | |
- - userId | string | The user ID. This is a custom string that helps with troubleshooting. | |
- - timestamp | string | The start time. | |
- - sig | string | The verification code. | |
- - clientId | string | ||
- request | object | The section for scoring request parameters. | |
- audio | object | The node for the audio parameters of the scoring request. | |
result | object | The evaluation result. | |
- res | string | The name of the resource used for this evaluation type. | Internal parameter |
- version | string | The engine version, including the release date and time. | Internal parameter |
- rank | int | The scoring system. | Internal parameter |
- precision | float | The scoring precision. | Internal parameter |
- pretime | Int | The time, in ms, that the server took to call the Start API. | Internal parameter |
- systime | int | The total time, in ms, for the entire evaluation process. | Internal parameter |
- delaytime | int | The time, in ms, from when the server finished processing the audio feed to when the result was obtained. | Internal parameter |
- wavetime | int | The audio duration, in ms. | For reference |
- overall | float | The overall score. | Recommended field |
- fluency | float | The fluency score. This score reflects the smoothness of speech and is not related to the correct answer. | Recommended field |
- pron | float | The pronunciation score. This score reflects pronunciation quality. It is not related to the correct answer and depends only on the accuracy of word pronunciation. | Recommended field |
- rhythm | float | The rhythm score. This parameter is not currently used. | Internal parameter |
- integrity | float | The integrity score. | Internal parameter |
- keypoints_hit | array | The hit status for each key point, in the same order as the points parameter. This field is not returned if the points field is not specified. | Recommended field |
- fake_words | object | This field is returned only for out-of-vocabulary (OOV) words. To evaluate OOV words, pass the phonetic transcription in the refText parameter. | |
- -char | string | ||
- info | object | ||
- - snr | float | The signal-to-noise ratio (SNR). A higher value indicates clearer audio. Range: 0 dB to 40 dB. If this parameter affects the score, a corresponding tipId is set. | For reference |
- - clip | float | Indicates audio clipping due to high volume. Range: 0 to 1. If this parameter affects the score, a corresponding tipId is set. | Internal parameter |
- - volume | int | The recording volume. Range: 0 dB to 180 dB. | For reference |
- - tipId | int | The audio quality ID. For more information, see tipId prompt information. | Recommended field |
Suggestions for displaying evaluation results
Dimension | Description | Details |
overall | Overall score | The evaluation score for the question type. |
pron | Pronunciation score | |
fluency | Fluency | |
keypoints_hit | Hit rate for each key point |
Response for a failed evaluation
If the result structure in the response is empty, the evaluation failed. Retrieve the errId and error from the response and handle the error based on the corresponding error code.
Error codes
For a list of error codes, see Evaluation error codes.