更新时间:2019-11-04 15:59
GraphCompute 支持将MaxCompute 表批量导入,但是这个过程中涉及到读取用户MaxCompute表的操作,所以需要用户提前授权GraphCompute公共账号有读取用户MaxCompute表的权限,具体的操作步骤如下所示:
-- 1. 切换到用户MaxCompute表所在的project
use project ***;
-- 2. 给MaxGraph公共账号进行授权
add user ALIYUN$big_graph@aliyun-inner.com;
GRANT Describe,Select ON TABLE table_name TO USER ALIYUN$big_graph@aliyun-inner.com;
使用SDK进行图数据的导入,需要先完成Client创建和Schema构建。
在此基础上, 本文介绍如何初始化mapping信息(maxCompute表的一行记录中的哪个字段写到graphCompute 点/边的哪个属性上)及bulk load的一些配置信息;
我们假设maxCompute上存在如下数据,可作为图模型的源数据。
maxCompute表person结构如下,可以作为GraphCompute中图模型的点person的源数据
字段名 | 字段类型 |
---|---|
id | bigint |
name | string |
age | int |
maxCompute表knows结构如下,可以作为GraphCompute中图模型的点knows的源数据
字段名 | 字段类型 |
---|---|
id | bigint |
srcid | bigint |
dstid | bigint |
weight | double |
用户需要在sdk中做好图属性和MaxCompute表项的对应关系。代码示例如下:
private String odpsEndpoint = "***";
private String yourOdpsAccessId = "***";
private String yourOdpsAccessKey = "**";
private String yourBizId = "**";
private String yourOdpsProject = "**";
// odps点表的table name
private String yourOdpsVertexTable = "**";
// odps边表的table name
private String yourOdpsEdgeTable = "**";
// odps中点表的colume
private String odpsPersonIdField = "**";
private String odpsPersonNameField = "**";
private String odpsPersonAgeField = "**";
// maxgraph 中点的属性
private String personIdProp = "**";
private String personNameProp = "**";
private String personAgeProp = "**";
// odps 中边表的colume
private String odpsKnowsIdField = "**";
private String odpsKnowsWeightField = "**";
// maxgaph 中边的属性
private String knowsIdProp = "**";
private String knowsWeightProp = "**";
// maxgraph 中边的起点主键和终点主键
private String knowsSrcPersonIdField = "**";
private String knowsDstPersonIdField = "**";
ResultSet resultSet = client.submit("graph.bulkloadVertexFromOdps('person')" +
".endpoint('" + odpsEndpoint + "')" +
".accessId('" + yourOdpsAccessId + "')" +
".accessKey('" + yourOdpsAccessKey + "')" +
".bizOwnerId('" + yourBizId + "')" +
".project('" + yourOdpsProject + "')" +
".table('" + yourOdpsVertexTable + "')" +
//".partition('" + yourOdpsPartition + "')" +
//".maxInvalidDataCount(10)" + // Default invalid data count is 0
".mappingColumn('" + odpsPersonIdField + "', '" + personIdProp + "')" +
// mappingColumn 定义了将odps表的一行记录的colume与maxgraph中点的属性的映射关系
".mappingColumn('" + odpsPersonNameField + "', '" + personNameProp + "')" +
".mappingColumn('" + odpsPersonAgeField + "', '" + personAgeProp + "')" +
".signature('" + getCredentialsManager().getPassword() + "')");
ResultSet resultSet = client.submit("graph.bulkloadEdgeFromOdps('knows')" +
".endpoint('" + odpsEndpoint + "')" +
".accessId('" + yourOdpsAccessId + "')" +
".accessKey('" + yourOdpsAccessKey + "')" +
".bizOwnerId('" + yourBizId + "')" +
".project('" + yourOdpsProject + "')" +
".table('" + yourOdpsEdgeTable + "')" +
//".partition('" + yourOdpsPartition + "')" +
//".maxInvalidDataCount(10)" + // Default invalid data count is 0
".mappingColumn('" + odpsKnowsIdField + "', '" + knowsIdProp + "')" +
".mappingColumn('" + odpsKnowsWeightField + "', '" + knowsWeightProp + "')" +
".srcVertex('person')" +
".mappingSrcPrimaryKey('" + knowsSrcPersonIdField + "','" + personIdProp + "')" +
".dstVertex('person')" +
".mappingDstPrimaryKey('" + knowsDstPersonIdField + "','" + personIdProp + "')" +
".signature('" + getCredentialsManager().getPassword() + "')");
在文档使用中是否遇到以下问题
更多建议
匿名提交