控制台项目,数据集,模型相关API操作文档。
1.创建项目
示例代码:
String accessKeyId = "your key";
String accessKeySecret = "your secret";
try {
// 创建DefaultAcsClient实例并初始化
DefaultProfile profile = DefaultProfile.getProfile("cn-hangzhou", accessKeyId,
accessKeySecret); // 您的AccessKey Secret
IAcsClient client = new DefaultAcsClient(profile);
// 创建API请求并设置参数
CommonRequest request = new CommonRequest();
request.setSysDomain("nlp-automl.cn-hangzhou.aliyuncs.com");
request.setSysVersion("2019-11-11");
request.setSysAction("CreateProject");
// Action需设置为CreateProject
request.setSysMethod(MethodType.POST);
request.putBodyParameter("ProjectType", ProjectType);
request.putBodyParameter("ProjectName", ProjectName);
request.putBodyParameter("ProjectDescription", ProjectDescription);
CommonResponse response = client.getCommonResponse(request);
logger.info("testBysdk is ok request:{} response:{}", JSON.toJSONString(request),
JSON.toJSONString(response));
System.out.println(response.getData());
} catch (Exception e) {
logger.error("testBysdk is error", e);
}
入参说明:
参数名称 | 类型 | 是否必填 | 备注 | 注意 |
ProjectType | string | 是 | 项目类型 | 'ProjectType'参照下表填入对应value,value为String类型 |
ProjectName | string | 是 | 项目名称 | ProjectName为String类型,不能以数字和符号开头,长度不能大于32个字符 |
ProjectDescription | string | 是 | 项目描述 | ProjectDescription为String类型,不能以数字和符号开头,长度不能大于256个字符 |
项目名称 | 对应参数value |
文本实体抽取 | ner |
文本分类 | textCategory |
文本关系抽取 | baseRelation |
短文本匹配 | textMatching |
情感分析 | sentimentAnalysis |
简历抽取 | resumeExtract |
商品评价解析 | reviewAnalysis |
返回数据示例:
{
"Code": 200,
"ProjectId": 1234,
"Message": "创建成功",
"RequestId": "xxxxxxxxxx",
"Success": true
}
返回的数据为JSON字符串需要反序列化后进行解析,ProjectId为创建的项目的ID
2.创建数据集
String accessKeyId = "your key";
String accessKeySecret = "your secret";
try {
// 创建DefaultAcsClient实例并初始化
DefaultProfile profile = DefaultProfile.getProfile("cn-hangzhou", accessKeyId,
accessKeySecret); // 您的AccessKey Secret
IAcsClient client = new DefaultAcsClient(profile);
// 创建API请求并设置参数
CommonRequest request = new CommonRequest();
request.setSysDomain("nlp-automl.cn-hangzhou.aliyuncs.com");
request.setSysVersion("2019-11-11");
request.setSysAction("CreateDataset");
// Action需设置为CreateDataset
request.setSysMethod(MethodType.POST);
request.putBodyParameter("ProjectId", ProjectId);
request.putBodyParameter("DatasetName", DatasetName);
CommonResponse response = client.getCommonResponse(request);
logger.info("testBysdk is ok request:{} response:{}", JSON.toJSONString(request),
JSON.toJSONString(response));
System.out.println(response.getData());
} catch (Exception e) {
logger.error("testBysdk is error", e);
}
入参说明:
参数名称 | 类型 | 是否必填 | 备注 | 注意 |
ProjectId | Long | 是 | 项目id | ProjectId为Long类型,要在某个项目下创建一个数据集,填写目标项目ID |
DatasetName | string | 是 | 数据集名称 | DatasetName为String类型,不能以数字和符号开头,长度不能大于32个字符 |
返回数据示例:
{
"Code": 200,
"DatasetId": 1234,
"Message": "创建成功",
"RequestId": "xxxxxx",
"Success": true
}
返回的数据为JSON字符串需要反序列化后进行解析,datasetId为创建的数据集的ID
3.数据集上传
String accessKeyId = "your key";
String accessKeySecret = "your secret";
try {
// 创建DefaultAcsClient实例并初始化
DefaultProfile profile = DefaultProfile.getProfile("cn-hangzhou", accessKeyId,
accessKeySecret); // 您的AccessKey Secret
IAcsClient client = new DefaultAcsClient(profile);
// 创建API请求并设置参数
CommonRequest request = new CommonRequest();
request.setSysDomain("nlp-automl.cn-hangzhou.aliyuncs.com");
request.setSysVersion("2019-11-11");
request.setSysAction("CreateDatasetRecord");
// Action需设置为CreateDatasetRecord
request.setSysMethod(MethodType.POST);
request.putBodyParameter("ProjectId", ProjectId);
request.putBodyParameter("DatasetId", DatasetId);
request.putBodyParameter("DatasetRecord", DatasetRecord);
CommonResponse response = client.getCommonResponse(request);
logger.info("testBysdk is ok request:{} response:{}", JSON.toJSONString(request),
JSON.toJSONString(response));
System.out.println(response.getData());
} catch (ServerException e) {
e.printStackTrace();
} catch (ClientException e) {
e.printStackTrace();
}
入参说明:
参数名称 | 类型 | 是否必填 | 备注 | 注意 |
ProjectId | Long | 是 | 项目id | |
DatasetId | Long | 是 | 数据集id | DatasetId为Long类型,要在某个数据集下添加一条数据,填写具体数据集id |
DatasetRecord | string | 是 | 标注文档内容 | DatasetRecord为String类型,需要传入JSON字符串,内容为标注数据,如下表格示例 |
DatasetId只可使用CreateDataset API接口创建的数据集,不可使用控制台创建的数据集ID。
项目类型 | DatasetRecord |
文本分类 | {"content":"这个牌子的红枣很不错","records":{"情感":"正"}} |
文本实体抽取 | {"content":"阿里巴巴集团达摩院的工作地主要有:北京、杭州、新加坡、美国等","records":{"公司名":[{"span":"阿里巴巴集团","offset":[0,6]}],"地点":[{"span":"北京","offset":[17,19]},{"span":"杭州","offset":[20,22]},{"span":"新加坡","offset":[23,26]},{"span":"美国","offset":[27,29]}]}} |
短文本匹配 | {"left": "快递太暴力了", "right": "快递太粗鲁了", "value": "0"} |
短文本匹配left与right为两个需要匹配的文本,value为标注是否匹配,0:不匹配,1:匹配
返回数据示例:
{
"Code": 200,
"DatasetRecordId": 1234,
"Message": "上传成功",
"RequestId": "1787856526166507",
"Success": true
}
4.数据集列表查询
String accessKeyId = "your key";
String accessKeySecret = "your secret";
try {
// 创建DefaultAcsClient实例并初始化
DefaultProfile profile = DefaultProfile.getProfile("cn-hangzhou", accessKeyId,
accessKeySecret); // 您的AccessKey Secret
IAcsClient client = new DefaultAcsClient(profile);
// 创建API请求并设置参数
CommonRequest request = new CommonRequest();
request.setSysDomain("nlp-automl.cn-hangzhou.aliyuncs.com");
request.setSysVersion("2019-11-11");
request.setSysAction("ListDataset");
// Action需设置为ListDataset
request.setSysMethod(MethodType.POST);
request.putBodyParameter("ProjectId", ProjectId);
request.putBodyParameter("PageNumber", PageNumber);
request.putBodyParameter("PageSize", PageSize);
CommonResponse response = client.getCommonResponse(request);
logger.info("testBysdk is ok request:{} response:{}", JSON.toJSONString(request),
JSON.toJSONString(response));
System.out.println(response.getData());
} catch (ServerException e) {
e.printStackTrace();
} catch (ClientException e) {
e.printStackTrace();
}
入参说明:
参数名称 | 类型 | 是否必填 | 备注 | 说明 |
ProjectId | Long | 是 | 项目ID | ProjectId为Long类型,为查询目标项目下的数据集列表 |
PageNumber | Integer | 是 | 页数 | PageNumber从1开始计算,必须大于0 |
PageSize | Integer | 是 | 分页大小 | PageSize必须大于0 |
返回数据示例:
{
"Code": 200,
"Data": {
"pageNumber": 1,
"pageSize": 1,
"rows": [
{
"annotatedDocNum": 7,
"id": 1234,
"name": "数据集名称",
"projectType": "textCategory",
"sourceType": "upload",
"status": "finished",
"totalDocNum": 7
}
],
"totalCount": 48
},
"Message": "",
"RequestId": "1787856526166507",
"Success": true
}
出参说明
参数名称 | 参数说明 | 备注 |
pageNumber | 当前页数 | 当前页数 |
pageSize | 分页大小 | 分页大小 |
rows | 返回数据集数据 | 返回数据集数据 |
totalCount | 总记录数 | 总记录数 |
rows参数说明
参数名称 | 参数说明 | 备注 |
id | 数据集ID | 数据集ID |
annotatedDocNum | 已标注文档数量 | 已标注文档数量 |
name | 数据集名称 | 数据集名称 |
projectType | 项目类型 | 项目类型 |
sourceType | 数据集来源 | upload 数据集上传 annotate 标注任务 |
status | 数据集状态 | init 初始化 parsing 解析中 parse_failed 解析失败 finished 完成 |
totalDocNum | 文档总数 | 文档总数 |
5.创建模型
示例代码:
String accessKeyId = "your key";
String accessKeySecret = "your secret";
try {
// 创建DefaultAcsClient实例并初始化
DefaultProfile profile = DefaultProfile.getProfile("cn-hangzhou", accessKeyId,
accessKeySecret); // 您的AccessKey Secret
IAcsClient client = new DefaultAcsClient(profile);
// 创建API请求并设置参数
CommonRequest request = new CommonRequest();
request.setSysDomain("nlp-automl.cn-hangzhou.aliyuncs.com");
request.setSysVersion("2019-11-11");
request.setSysAction("CreateModel");
// Action需设置为CreateModel
request.setSysMethod(MethodType.POST);
request.putBodyParameter("ModelType", "NLP-Resume-Chinese");
request.putBodyParameter("ProjectId", 3667);
request.putBodyParameter("ModelName", "apiTest");
request.putBodyParameter("DatasetIdList", "12267");
CommonResponse response = client.getCommonResponse(request);
logger.info("testBysdk is ok request:{} response:{}", JSON.toJSONString(request),
JSON.toJSONString(response));
System.out.println(response.getData());
} catch (Exception e) {
logger.error("testBysdk is error", e);
}
入参说明:
字段名称 | 类型 | 是否必填 | 字段含义 | 注意 |
ModelType | String | 是 | 模型类型 | 模型类型名称ModelType,不能以数字和符号开头,长度不能大于32个字符 |
ProjectId | Long | 是 | 项目ID | |
ModelName | String | 是 | 模型名称 | |
ModelId | Long | 否 | 模型ID,即自学习平台上可见的模型ID(增量版本时需要) | 新增模型版本时ModelId为必传项 |
DatasetIdList | String | 是 | 数据集ID列表,逗号分割如"1212,1211" | 上传数据集必须在同一ProjectId下,已上线状态,且已标注数据总数大于等于4条 |
TestDatasetIdList | String | 否 | 测试集ID列表,逗号分隔(文本分类下部分模型支持添加测试集数据) | |
IsIncrementalTrain | String | 否 | 是否增量训练,是:true,否:false,新增版本必填true | |
Nepochs | String | 否 | 训练数据集迭代次数 | |
Lr | String | 否 | 学习率,模型算法指标 |
ModelType参照下面对照表,传入对应的value
项目 | 模型类型名称对应传值 value | 不同项目下,传指定的modelType,后端会做校验,1.如果在Project下不存在该模型类型,则会提示未找到基础模型:BaseModelNotFoundError例如在文本实体抽取下传递文本分类的modelType等情况注:关键短语抽取DatasetIdList -1 |
文本实体抽取 | LSTM-CRF 中文实体抽取--> NLP-NER-Chinese | |
TinyStructBERT 中文实体抽取 --> NLP-NER-BERT | ||
英文实体抽取 --> NLP-NER-English | ||
文本分类 | 长文本分类融合模型 --> NLP-TextCategory-ensemble | |
CNN分类模型 --> NLP-TextCategory-CNN2 | ||
FastText分类模型 --> NLP-TextCategory-FT | ||
Self-Attention分类模型 --> NLP-TextCategory-SA | ||
短文本分类融合模型 --> NLP-TextCategory-STC | ||
Bert小样本分类 --> classification_bert | ||
关键短语抽取 | 关键短语抽取 --> NLP-Keyphrase-textrank | |
文本关系抽取 | 关系抽取PCNN --> NLP-RE-PCNN | |
关系抽取Bert --> NLP-RelationClassficiation-BERT | ||
关系抽取BertNoise --> NLP-RE-BERT-NOISE | ||
短文本匹配 | structbert_matching | |
情感分析 | 中文CNN情感分析 --> NLP-TextCategory-CNN2 | |
商品评价解析 | classification_cnn | |
简历抽取 | 简历抽取自训练模型(中文)--> NLP-Resume-Chinese | |
无标注数据的平台预置简历抽取模型(中文)--> NLP-Resume-Pretrain-Chinese | ||
无标注数据的平台预置简历抽取模型(英文)--> NLP-Resume-Pretrain-English |
返回结果示例:
{
"RequestId": "xxxxxxxxx",
"Message": "",
"Data": {
"modelName": "api",
"modelId": 1841,
"modelStatus": "初始化",
"modelVersion": "v1",
"updateTime": "2020-09-24 18:36:34",
"modelType": "NLP-NER-Chinese",
"trainingCount": 18,
"projectId": 1922
},
"Code": 200,
"Success": true
}
6.发布、上线、下线模型
同一个版本模型不能连续重复相同操作,比如已经发布的模型再次发布。重复操作会有错误信息提示10003。
代码示例:
String accessKeyId = "your key";
String accessKeySecret = "your secret";
try {
// 创建DefaultAcsClient实例并初始化
DefaultProfile profile = DefaultProfile.getProfile("cn-hangzhou", accessKeyId,
accessKeySecret); // 您的AccessKey Secret
IAcsClient client = new DefaultAcsClient(profile);
// 创建API请求并设置参数
CommonRequest request = new CommonRequest();
request.setSysDomain("nlp-automl.cn-hangzhou.aliyuncs.com");
request.setSysVersion("2019-11-11");
request.setSysAction("DeployModel");
// Action需设置为DeployModel
request.setSysMethod(MethodType.POST);
request.putBodyParameter("ModelId", 5590);
request.putBodyParameter("ProjectId", 3667);
request.putBodyParameter("ModelVersion", "v1");
//"1":模型发布 "2":模型下线 "3":模型下线
request.putBodyParameter("OptType", "1");
CommonResponse response = client.getCommonResponse(request);
System.out.println(response.getData());
logger.info("testBysdk is ok request:{} response:{}", JSON.toJSONString(request),
JSON.toJSONString(response));
} catch (Exception e) {
logger.error("testBysdk is error", e);
}
入参说明:
参数名称 | 类型 | 是否必填 | 备注 |
ModelId | Long | 是 | 模型ID |
OptType | String | 是 | "1":模型发布 "2":模型下线 "3":模型上线 |
ProjectId | Long | 是 | 项目ID |
ModelVersion | String | 是 | 模型迭代版本,如:v1 |
返回结果示例:
{
"RequestId": "xxxxxxxxx",
"Message": "",
"Data": {
"modelName": "api",
"modelId": 1841,
"modelStatus": "发布中",
"modelVersion": "v1",
"precision": "50.00",
"recall": "50.00",
"updateTime": "2020-09-24 18:51:45",
"modelType": "NLP-NER-Chinese",
"f1": "50.00",
"trainingCount": 18,
"projectId": 1922
},
"Code": 200,
"Success": true
}
7.查询指定模型版本信息
代码示例:
String accessKeyId = "your key";
String accessKeySecret = "your secret";
try {
// 创建DefaultAcsClient实例并初始化
DefaultProfile profile = DefaultProfile.getProfile("cn-hangzhou", accessKeyId,
accessKeySecret); // 您的AccessKey Secret
IAcsClient client = new DefaultAcsClient(profile);
// 创建API请求并设置参数
CommonRequest request = new CommonRequest();
request.setSysDomain("nlp-automl.cn-hangzhou.aliyuncs.com");
request.setSysVersion("2019-11-11");
request.setSysAction("GetModel");
// Action需设置为GetModel
request.setSysMethod(MethodType.POST);
request.putBodyParameter("ModelId", 4972);
request.putBodyParameter("ProjectId", 4559);
request.putBodyParameter("ModelVersion", "v1");
CommonResponse response = client.getCommonResponse(request);
System.out.println(response.getData());
logger.info("testBysdk is ok request:{} response:{}", JSON.toJSONString(request),
JSON.toJSONString(response));
} catch (Exception e) {
logger.error("testBysdk is error", e);
}
入参说明:
参数名称 | 类型 | 是否必填 | 备注 |
ModelId | Long | 是 | 模型ID |
ProjectId | Long | 是 | 项目ID |
ModelVersion | String | 是 | 指定查询的版本,如:v1 |
返回结果示例:
{
"RequestId": "xxxxxxxxx",
"Message": "",
"Data": {
"modelName": "api",
"modelId": 1841,
"modelStatus": "发布完成",
"modelVersion": "v1",
"precision": "50.00",
"recall": "50.00",
"updateTime": "2020-09-24 18:53:08",
"modelType": "NLP-NER-Chinese",
"f1": "50.00",
"trainingCount": 18,
"projectId": 1922
},
"Code": 200,
"Success": true
}
8.模型分页查询
代码示例:
String accessKeyId = "your key";
String accessKeySecret = "your secret";
try {
// 创建DefaultAcsClient实例并初始化
DefaultProfile profile = DefaultProfile.getProfile("cn-hangzhou", accessKeyId,
accessKeySecret); // 您的AccessKey Secret
IAcsClient client = new DefaultAcsClient(profile);
// 创建API请求并设置参数
CommonRequest request = new CommonRequest();
request.setSysDomain("nlp-automl.cn-hangzhou.aliyuncs.com");
request.setSysVersion("2019-11-11");
request.setSysAction("ListModels");
request.setSysMethod(MethodType.POST);
request.putBodyParameter("ProjectId", 3667);
request.putBodyParameter("PageNumber", 1);
request.putBodyParameter("PageSize", 5);
CommonResponse response = client.getCommonResponse(request);
logger.info("testBysdk is ok request:{} response:{}", JSON.toJSONString(request),
JSON.toJSONString(response));
} catch (Exception e) {
logger.error("testBysdk is error", e);
}
参数说明:
参数 | 类型 | 是否必填 | 备注 | 注意 |
ProjectId | Long | 是 | 项目ID | ProjectId为Long类型,为查询目标项目下的模型列表 |
PageSize | Integer | 是 | 每页展示模型数量 | PageSize必须大于0 |
PageNumber | Integer | 是 | 当前页数 | PageNumber从1开始计算,必须大于0 |
以模型ID分页返回每个模型下的所有迭代模型信息
返回结果示例:
{
"RequestId": "xxxxxxxxx",
"Message": "",
"Data": {
"pageNumber": 1,
"pageSize": 5,
"rows": [
{
"modelName": "test",
"models": [
{
"modelStatus": "训练结束",
"modelVersion": "v3",
"precision": "0.60",
"recall": "0.64",
"f1": "0.64",
"trainingCount": 65
},
{
"modelStatus": "训练结束",
"modelVersion": "v4",
"precision": "0.66",
"recall": "0.49",
"f1": "0.57",
"trainingCount": 65
},
{
"modelStatus": "训练结束",
"modelVersion": "v5",
"precision": "0.62",
"recall": "0.49",
"f1": "0.55",
"trainingCount": 65
},
{
"modelStatus": "训练结束",
"modelVersion": "v6",
"precision": "0.99",
"recall": "0.99",
"f1": "0.99",
"trainingCount": 65
},
{
"modelStatus": "训练结束",
"modelVersion": "v7",
"precision": "0.99",
"recall": "0.99",
"f1": "0.99",
"trainingCount": 65
},
{
"modelStatus": "发布完成",
"modelVersion": "v1",
"precision": "0.62",
"recall": "0.62",
"f1": "0.62",
"trainingCount": 22
},
{
"modelStatus": "训练结束",
"modelVersion": "v2",
"precision": "0.37",
"recall": "0.37",
"f1": "0.37",
"trainingCount": 22
}
],
"modelId": 1773,
"modelType": "NLP-TextCategory-ensemble",
"projectId": 1923
}
],
"totalCount": 11
},
"Code": 200,
"Success": true
}
9.删除模型
代码示例:
String accessKeyId = "your key";
String accessKeySecret = "your secret";
try {
// 创建DefaultAcsClient实例并初始化
DefaultProfile profile = DefaultProfile.getProfile("cn-hangzhou", accessKeyId,
accessKeySecret); // 您的AccessKey Secret
IAcsClient client = new DefaultAcsClient(profile);
// 创建API请求并设置参数
CommonRequest request = new CommonRequest();
request.setSysDomain("nlp-automl.cn-hangzhou.aliyuncs.com");
request.setSysVersion("2019-11-11");
request.setSysAction("DeleteModel");
request.setSysMethod(MethodType.POST);
request.putBodyParameter("ProjectId", projectId);
request.putBodyParameter("ModelId", modelId);
CommonResponse response = client.getCommonResponse(request);
logger.info("testBysdk is ok request:{} response:{}", JSON.toJSONString(request),
JSON.toJSONString(response));
} catch (Exception e) {
logger.error("testBysdk is error", e);
}
参数说明:
参数 | 类型 | 是否必填 | 备注 | 注意 |
ProjectId | Long | 是 | 项目ID | |
ModelId | Long | 是 | 模型组ID | ModelId 为模型组ID |
返回结果示例:
{
"RequestId": "28E59BD8-4591-425C-8752-93DCF1BEA93F",
"Message": "",
"Data": true,
"Code": 200,
"Success": true
}