看图说话
该文档是开发者在产品功能集成阶段 使用的英文看图说话题型的语音评测技术文档
本题型适用场景:
题目预先给出图片,要求用户用英文完整口述图片内容。
参数说明:
字段 | 类型 | 可选 | 说明 |
coreType | String | 必选 | en.pict.score表示英文看图作文题型 |
rank | int | 可选 | 总分评分分制 |
precision | double | 可选 | 设置打分精度,只支持0.1、0.5或者1,默认值为1(如果设置的值不是0.1或0.5则按1处理) |
grade | int | 可选 | 指定学段;只支持1和2,默认值为1 1:表示初中学段(覆盖要点到90%左右才能得满分) 2:表示高中学段(因为高中话题有难度,只要覆盖80%左右就能得高分) |
lm | array | 必选 | 图片对应的范例作文 每个text子字段表示一个范例作文 (注:lm字段是LM的小写形式) |
phones | object | 可选 | 指定单词的发音。如果单词有多发音,要用”|”符号分隔开;每个音素之间及音素与音节分隔符”-“之间,必须使用空格隔开。 例:"phones":{"directory":"d ih - 'r eh k - tr ih|d ax - 'r eh k - t ax - r ih"}; |
points | array | 可选 | 设置答案要点; 每个要点一个text |
attachAudioUrl | int | 可选 | 评测结果中是否返回音频url |
rateScale | float | 可选 | 打分宽松度,设置的范围0.8~1.5,默认1.0;如果设置值小于0.8,按0.8计算;如果大于1.5则按1.5计算。这个参数可以看作是个乘数,值越高打分越高。 |
参数示例
{
"coreType": "en.pict.score",
"rank": 100,
"precision": 0.5,
"attachAudioUrl":1,
"lm": [
{
"text": "My aunt Judy was a single mother of fifty, working as a secretary. When doctors discovered she had a very serious heart disease, she didn't give in. She began studying medical reports and found an article in a magazine about a well known heart doctor, Dr. Michael Baker, of Houston, Texas. The article said Dr. Baker's fees were very high; Aunt Judy couldn't afford them. But could he tell her of someone whose fee she could pay? So Aunt Judy wrote to him. She simply listed her reasons for wanting to live: her three children, who would be independent in three or four more years, her little girl dream of travelling and seeing the world. There wasn't a word of self pity only warmth and humor and the joy of living. She mailed the letter, not really expecting an answer. A few days later, Aunt Judy rushed into my house, and read aloud a letter: your beautiful letter moved me very deeply. If you can come to Houston, there will be no charge for either the hospital or the operation. Michael Baker. That was seven years ago. Since then, Aunt Judy has been around the world. Her three children are happily married."
}
],
"points": [
{
"text": "My aunt Judy was a single mother of fifty, working as a secretary."
},
{
"text": "When doctors discovered she had a very serious heart disease, she didn't give in."
},
{
"text": "She began studying medical reports and found an article in a magazine about a well known heart doctor, Dr. Michael Baker, of Houston, Texas."
},
{
"text": "The article said Dr. Baker's fees were very high; Aunt Judy couldn't afford them."
},
{
"text": "But could he tell her of someone whose fee she could pay? So Aunt Judy wrote to him."
},
{
"text": "She simply listed her reasons for wanting to live: her three children, who would be independent in three or four more years, her little girl dream of travelling and seeing the world."
},
{
"text": "There wasn't a word of self pity only warmth and humor and the joy of living."
},
{
"text": "She mailed the letter, not really expecting an answer."
},
{
"text": "A few days later, Aunt Judy rushed into my house, and read aloud a letter: your beautiful letter moved me very deeply. If you can come to Houston, there will be no charge for either the hospital or the operation. Michael Baker."
},
{
"text": "That was seven years ago. Since then, Aunt Judy has been around the world. Her three children are happily married."
}
]
}返回结果示例
{
"tokenId": "612f3d6c88bf5b0000040001",
"applicationId": "a148",
"audioUrl": "http:\/\/files.cloud.ssapi.cn:8080\/a148\/11ec0b00cf7d709f94a4a14872616788",
"connect": {
"param": {
"app": {
"timestamp": "1630485868",
"applicationId": "a148",
"sig": "342befbf5b7ccb03d4562fad7b21d7261908d5da"
},
"sdk": {
"os": "6.2(9200)1",
"product": "",
"source": 7,
"protocol": 1,
"os_version": "6.2(9200)",
"arch": "586",
"version": 16778752
}
},
"cmd": "connect"
},
"params": {
"app": {
"timestamp": "1630485868",
"userId": "ssound_text",
"sig": "342befbf5b7ccb03d4562fad7b21d7261908d5da",
"connect_id": "612f3d6c88bf5b0000060002",
"clientId": "",
"applicationId": "a148"
},
"audio": {
"sampleBytes": 2,
"audioType": "ogg",
"sampleRate": 16000,
"channel": 1
},
"request": {
"lm": [
{
"text": "My aunt Judy was a single mother of fifty, working as a secretary. When doctors discovered she had a very serious heart disease, she didn't give in. She began studying medical reports and found an article in a magazine about a well known heart doctor, Dr. Michael Baker, of Houston, Texas. The article said Dr. Baker's fees were very high; Aunt Judy couldn't afford them. But could he tell her of someone whose fee she could pay? So Aunt Judy wrote to him. She simply listed her reasons for wanting to live: her three children, who would be independent in three or four more years, her little girl dream of travelling and seeing the world. There wasn't a word of self pity only warmth and humor and the joy of living. She mailed the letter, not really expecting an answer. A few days later, Aunt Judy rushed into my house, and read aloud a letter: your beautiful letter moved me very deeply. If you can come to Houston, there will be no charge for either the hospital or the operation. Michael Baker. That was seven years ago. Since then, Aunt Judy has been around the world. Her three children are happily married."
}
],
"tokenId": "612f3d6c88bf5b0000040001",
"request_id": "612f3d6c88bf5b0000050001",
"precision": 0.5,
"attachAudioUrl": 1,
"coreType": "en.pict.score",
"rank": 100,
"points": [
{
"text": "My aunt Judy was a single mother of fifty, working as a secretary."
},
{
"text": "When doctors discovered she had a very serious heart disease, she didn't give in."
},
{
"text": "She began studying medical reports and found an article in a magazine about a well known heart doctor, Dr. Michael Baker, of Houston, Texas."
},
{
"text": "The article said Dr. Baker's fees were very high; Aunt Judy couldn't afford them."
},
{
"text": "But could he tell her of someone whose fee she could pay? So Aunt Judy wrote to him."
},
{
"text": "She simply listed her reasons for wanting to live: her three children, who would be independent in three or four more years, her little girl dream of travelling and seeing the world."
},
{
"text": "There wasn't a word of self pity only warmth and humor and the joy of living."
},
{
"text": "She mailed the letter, not really expecting an answer."
},
{
"text": "A few days later, Aunt Judy rushed into my house, and read aloud a letter: your beautiful letter moved me very deeply. If you can come to Houston, there will be no charge for either the hospital or the operation. Michael Baker."
},
{
"text": "That was seven years ago. Since then, Aunt Judy has been around the world. Her three children are happily married."
}
]
}
},
"recordId": "11ec0b00cf7d709f94a4a14872616788",
"eof": 1,
"result": {
"keypoints_hit": [
0.25,
0,
0,
0.166667,
0.142857,
0.1875,
0.222222,
0,
0.090909,
0.5
],
"overall": 11.5,
"wavetime": 7990,
"precision": 0.5,
"systime": 8098,
"res": "eng.retell.mfcc.0.1",
"version": "0.0.80.2021.8.27.04:29:39",
"pron": 85.5,
"info": {
"tipId": 0,
"clip": 0,
"snr": 14.624933,
"volume": 84
},
"fluency": 84,
"rank": 100,
"rhythm": 0,
"integrity": 8,
"pretime": 586,
"delaytime": 375
},
"dtLastResponse": "2021-09-01 16:44:34:784",
"cloud_platform": {
"origin_audio_length": 30457
}
}返回结果说明
键 | 类型 | 说明 | 使用建议 |
recordId | string | 音频文件的唯一标识 | 可供参考 |
tokenId | string | 用户请求的唯一标识 | 可供参考 |
applicationId | string | 提供的授权账号,即Appkey | 内部参数 |
audioUrl | string | 用户音频地址,在云端默认只保存一个月,建议保存到产品端服务器 | 可供参考 |
dtLastResponse | string | 云端响应的时间点 | 内部参数 |
eof | Int | 0表示返回未结束,后续还有其它的返回结果 1:表示本次评测所有的返回结束 | 内部参数 |
params | object | 回显客户端传入的评测请求参数 | |
- app | object | ||
- - applicationId | string | 提供的授权账号,即Appkey | |
- - userId | string | 用户标识 ,自定义字符串,方便定位排查问题 | |
- - timestamp | string | start时间 | |
- - sig | string | 校验码 | |
- - clientId | string | ||
- request | object | 评分请求参数节 | |
- audio | object | 评分请求的音频参数节点数 | |
result | object | 评测结果 | |
- res | string | 评测本题型时,使用的资源名称 | 内部参数 |
- version | string | 引擎的版本,包含发布时间 | 内部参数 |
- rank | int | 评分分制 | 内部参数 |
- precision | float | 评分精度 | 内部参数 |
- pretime | Int | 云端调用Start接口本身耗时,单位ms | 内部参数 |
- systime | int | 评测整个过程的总耗时,单位ms | 内部参数 |
- delaytime | int | 云端从feed音频结束到获取结果的耗时,单位ms | 内部参数 |
- wavetime | int | 音频时长,单位ms | 可供参考 |
- overall | float | 总分 | 建议字段 |
- fluency | float | 流利度分(表述流畅情况,与正确答案无关) | 建议字段 |
- pron | float | 发音分(体现发音情况,与正确答案无关,只与单词发音准确程度有关) | 建议字段 |
- rhythm | float | 韵律分(暂未使用) | 内部参数 |
- integrity | float | 完整度分 | 内部参数 |
- keypoints_hit | array | 按points顺序分别对应每个要点的命中情况,无points字段时,该字段不输出; | 建议字段 |
- fake_words | object | 单词不在词典(集外词),才会输出此字段。如果需要对集外词进行评测,可以通过refText传音标的方式进行评测。 | |
- -char | string | ||
- info | object | ||
- - snr | float | 信噪比,值越高越清晰,范围(0~40dB) 此参数影响评分时,会设置相应的tipId值 | 可供参考 |
- - clip | float | 音频声音太高,出现截幅,范围(0\~1)。 此参数影响评分时,会设置相应的tipId值 | 内部参数 |
- - volume | int | 录音音量,范围(0~180dB) | 可供参考 |
- - tipId | int | 音频质量 ,详情请参考 tipId提示信息说明 | 建议字段 |
评测结果展现建议
维度 | 说明 | 详细 |
overall | 总分 | 题型的评测得分 |
pron | 发音分 | |
fluency | 流利度 | |
keypoints_hit | 各个要点的命令率 |
评测失败时的返回结果
如果返回的评测结果中result结构为空,说明评测失败。通过返回结果可以获取 errId 和 error,需要根据相应错误码进行后续判断和处理。
错误码
错误码参考:评测错误码