DataWorks提供了丰富的OpenAPI,您可以根据需要使用DataWorks的OpenAPI等开放能力实现各种业务场景。本文以元数据表管理为例,为您介绍如何串连元数据的OpenAPI来达成查询列表、查询表详情等操作。
前置概念
在进行本实践之前,建议您先参考以下链接,了解DataWorks的OpenAPI的基本能力和概念:
一、查询表列表
本实践介绍如何使用OpenAPI查询MaxCompute项目/模式下的表列表,并支持分页查询。主要流程如下:
后端:使用MetaServiceProxy查询表详情
在MetaServiceProxy中编写一个ListTables方法,处理前端参数并发送请求到OpenAPI ListTables中,以获取元数据表的列表信息。
支持查询的参数:父层级元数据实体ID、名称(模糊匹配)、注释(模糊匹配)、类型以及排序参数和分页的参数。
说明参考元数据实体相关概念说明,此处的父层级元数据实体ID可能存在两种格式。
父层级元数据实体ID为MaxCompute项目ID,格式为:
maxcompute-project:${主账号ID}::{projectName}
父层级元数据实体ID为MaxCompute模式ID,格式为:
maxcompute-schema:${主账号ID}::{projectName}:{schemaName}
查询结果中包含内容:数据表总数、分页信息、每个数据表的ID、父层级元数据实体ID、表名称、注释、数据库名称、模式名称、类型、分区字段列表、创建时间与修改时间。
获取主账号ID:无论是RAM子账号还是主账号都可在页面右上角的账号处查看主账号ID。
登录主账号或RAM子账号到DataWorks控制台。
鼠标悬浮右上方的头像按钮上即可查看主账号ID。
主账号:账号ID即为需要获取的主账号ID。
RAM账号:可直接查看到主账号ID。
示例代码:
/** * @author dataworks demo */ @Service public class MetaServiceProxy { @Autowired private DataWorksOpenApiClient dataWorksOpenApiClient; /** * DataWorks OpenAPI : ListTables * * @param listTablesDto */ public ListTablesResponseBodyPagingInfo listTables(ListTablesDto listTablesDto) { try { Client client = dataWorksOpenApiClient.createClient(); ListTablesRequest listTablesRequest = new ListTablesRequest(); // 父层级实体ID,元数据相关实体概念说明。 listTablesRequest.setParentMetaEntityId(listTablesDto.getParentMetaEntityId()); // 表名称,支持模糊匹配 listTablesRequest.setName(listTablesDto.getName()); // 表注释,支持模糊匹配 listTablesRequest.setComment(listTablesDto.getComment()); // 表类型,默认返回所有类型 listTablesRequest.setTableTypes(listTablesDto.getTableTypes()); // 排序方式,CreateTime (默认) / ModifyTime / Name / TableType listTablesRequest.setSortBy(listTablesDto.getSortBy()); // 排序方向,支持 Asc (默认) / Desc listTablesRequest.setOrder(listTablesDto.getOrder()); // 分页页码,默认为1 listTablesRequest.setPageNumber(listTablesDto.getPageNumber()); // 分页大小,默认为10,最大100 listTablesRequest.setPageSize(listTablesDto.getPageSize()); ListTablesResponse response = client.listTables(listTablesRequest); // 获取符合要求的表总数 System.out.println(response.getBody().getPagingInfo().getTotalCount()); for (Table table : response.getBody().getPagingInfo().getTables()) { // 表ID System.out.println(table.getId()); // 表的父层级实体ID System.out.println(table.getParentMetaEntityId()); // 表名称 System.out.println(table.getName()); // 表注释 System.out.println(table.getComment()); // 表所属数据库/maxcompute项目名称 System.out.println(table.getId().split(":")[3]); // 表所属schema名称 System.out.println(table.getId().split(":")[4]); // 表类型 System.out.println(table.getTableType()); // 表创建时间 System.out.println(table.getCreateTime()); // 表修改时间 System.out.println(table.getModifyTime()); // 表的分区字段列表 System.out.println(table.getPartitionKeys()); } return response.getBody().getPagingInfo(); } catch (TeaException error) { // 此处仅做打印展示,请谨慎对待异常处理,在工程项目中切勿直接忽略异常。 // 错误 message System.out.println(error.getMessage()); // 诊断地址 System.out.println(error.getData().get("Recommend")); com.aliyun.teautil.Common.assertAsString(error.message); } catch (Exception _error) { TeaException error = new TeaException(_error.getMessage(), _error); // 此处仅做打印展示,请谨慎对待异常处理,在工程项目中切勿直接忽略异常。 // 错误 message System.out.println(error.getMessage()); // 诊断地址 System.out.println(error.getData().get("Recommend")); com.aliyun.teautil.Common.assertAsString(error.message); } return null; } }
在MetaRestController中添加一个
listTables
方法作为入口供前端访问。/** * 演示如何通过DataWorks OpenAPI构建自定义元数据平台 * * @author dataworks demo */ @RestController @RequestMapping("/meta") public class MetaRestController { @Autowired private MetaServiceProxy metaService; /** * 分页查询表数据 * * @param listTablesDto * @return {@link ListTablesResponseBodyPagingInfo} */ @CrossOrigin(origins = "http://localhost:8080") @GetMapping("/listTables") public ListTablesResponseBodyPagingInfo listTables(ListTablesDto listTablesDto) { System.out.println("listTablesDto = " + listTablesDto); return metaService.listTables(listTablesDto); } }
前端:展示表元数据信息
import React from "react";
import cn from "classnames";
import {
Table,
Form,
Field,
Input,
Button,
Pagination,
Dialog,
Card,
Grid,
Select,
} from "@alifd/next";
import type { TableListInput, TableEntity } from "../services/meta";
import * as services from "../services";
import DetailContent from "./detailContent";
import LineageContent from "./lineageContent";
import classes from "../styles/app.module.css";
import moment from "moment";
export interface Props {}
const { Column } = Table;
const { Item } = Form;
const { Header, Content } = Card;
const { Row, Col } = Grid;
const { Option } = Select;
const App: React.FunctionComponent<Props> = () => {
const field = Field.useField();
const [datasource, setDatasource] = React.useState<TableEntity[]>([]);
const [loading, setLoading] = React.useState<boolean>(false);
const [total, setTotal] = React.useState<number>(0);
const [current, setCurrent] = React.useState<number>(1);
const onSubmit = React.useCallback(
(pageNumber: number = 1) => {
field.validate(async (errors, values) => {
if (errors) {
return;
}
setLoading(true);
try {
const response = await services.meta.getTableList({
pageNumber,
...values,
} as TableListInput);
setDatasource(response.tables);
setCurrent(pageNumber);
setTotal(response.totalCount);
} catch (e) {
throw e;
} finally {
setLoading(false);
}
});
},
[field]
);
const onViewDetail = React.useCallback((record: TableEntity) => {
Dialog.show({
title: "数据表详情",
content: <DetailContent item={record} />,
shouldUpdatePosition: true,
footerActions: ["ok"],
});
}, []);
const onViewLineage = React.useCallback((record: TableEntity) => {
Dialog.show({
title: "表血缘关系",
content: <LineageContent item={record} />,
shouldUpdatePosition: true,
footerActions: ["ok"],
});
}, []);
React.useEffect(() => {
field.setValue("dataSourceType", "odps");
}, []);
return (
<div className={cn(classes.appWrapper)}>
<Card free hasBorder={false}>
<Header title="元数据表管理场景Demo(MaxCompute)" />
<Content style={{ marginTop: 24 }}>
<Form field={field} colon fullWidth>
<Row gutter="1">
<Col>
<Item
label="MaxCompute项目/模式ID"
name="parentMetaEntityId"
required
help="ID"
>
<Input />
</Item>
</Col>
</Row>
<Row gutter="4">
<Col>
<Item label="名称" name="name">
<Input placeholder="输入表名称,模糊匹配" />
</Item>
</Col>
<Col>
<Item label="注释" name="comment">
<Input placeholder="输入表注释,模糊匹配" />
</Item>
</Col>
<Col>
<Item label="类型" name="tableTypes">
<Select mode="multiple" style={{ width: "100%" }}>
<Option value="TABLE">表</Option>
<Option value="VIEW">视图</Option>
<Option value="MATERIALIZED_VIEW">物化视图</Option>
</Select>
</Item>
</Col>
<Col>
<Item label="排序方式" name="sortBy">
<Select defaultValue="CreateTime" style={{ width: "100%" }}>
<Option value="CreateTime">创建时间</Option>
<Option value="ModifyTime">修改时间</Option>
<Option value="Name">名称</Option>
<Option value="TableType">类型</Option>
</Select>
</Item>
</Col>
<Col>
<Item label="排序方向" name="order">
<Select defaultValue="Asc" style={{ width: "100%" }}>
<Option value="Asc">升序</Option>
<Option value="Desc">降序</Option>
</Select>
</Item>
</Col>
</Row>
<div
className={cn(
classes.searchPanelButtonWrapper,
classes.buttonGroup
)}
>
<Button type="primary" onClick={() => onSubmit()}>
查询
</Button>
</div>
</Form>
<div>
<Table
dataSource={datasource}
loading={loading}
className={cn(classes.tableWrapper)}
emptyContent={
<div className={cn(classes.noDataWrapper)}>暂无数据</div>
}
>
<Column
title="项目名称"
dataIndex="id"
cell={(value) => {
const parts = value.split(":");
return parts.length > 1 ? parts[parts.length - 3] : "";
}}
/>
<Column
title="schema"
dataIndex="id"
cell={(value) => {
const parts = value.split(":");
return parts.length > 1 ? parts[parts.length - 2] : "";
}}
/>
<Column title="表名称" dataIndex="name" />
<Column title="表注释" dataIndex="comment" />
<Column title="表类型" dataIndex="tableType" />
<Column
title="是否分区表"
dataIndex="partitionKeys"
cell={(value) => {
return value != null && value.length > 0 ? "是" : "否";
}}
/>
<Column
title="创建时间"
dataIndex="createTime"
cell={(value) => {
return moment(value).format("YYYY-MM-DD HH:mm:ss");
}}
/>
<Column
title="修改时间"
dataIndex="modifyTime"
cell={(value) => {
return moment(value).format("YYYY-MM-DD HH:mm:ss");
}}
/>
<Column
title="操作"
width={150}
cell={(value, index, record) => (
<div className={cn(classes.buttonGroup)}>
<Button
type="primary"
onClick={() => onViewDetail(record)}
text
>
查看详情
</Button>
<Button
type="primary"
onClick={() => onViewLineage(record)}
text
>
查看表血缘
</Button>
</div>
)}
/>
</Table>
<Pagination
current={current}
total={total}
onChange={onSubmit}
showJump={false}
className={cn(classes.tablePaginationWrapper)}
/>
</div>
</Content>
</Card>
</div>
);
};
export default App;
完成上述代码开发后,您可在通用操作:本地部署运行工程代码。您可以输入MaxCompute项目空间ID及其他参数进行查询,获得项目空间下的所有表的列表,支持分页查询。
二、查找表详情
以下实践将结合元数据中GetTable、ListColumns与ListPartitions三个OpenAPI来实现查询表详情,通过UpdateColumnBusinessMetadataAPI,更新字段的业务说明,实践操作流程如下。
后端:使用MetaServiceProxy查询表详情
在MetaServiceProxy中创建
getTable
、listColumns
、listPartitions
方法来调用OpenAPI服务获取表的基础信息、表的字段信息与表的分区信息,创建updateColumnBusinessMetadata
方法来更新字段的业务说明/** * @author dataworks demo */ @Service public class MetaServiceProxy { @Autowired private DataWorksOpenApiClient dataWorksOpenApiClient; /** * DataWorks OpenAPI : getTableDto * * @param getTableDto */ public Table getTable(GetTableDto getTableDto) { try { Client client = dataWorksOpenApiClient.createClient(); GetTableRequest getTableRequest = new GetTableRequest(); // 数据表ID getTableRequest.setId(getTableDto.getId()); // 是否返回业务元数据 getTableRequest.setIncludeBusinessMetadata(getTableDto.getIncludeBusinessMetadata()); GetTableResponse response = client.getTable(getTableRequest); // 对应的数据表 Table table = response.getBody().getTable(); // 表ID System.out.println(table.getId()); // 表的父层级实体ID System.out.println(table.getParentMetaEntityId()); // 表名称 System.out.println(table.getName()); // 表注释 System.out.println(table.getComment()); // 表所属数据库 / maxcompute项目名称 System.out.println(table.getId().split(":")[3]); // 表所属schema名称 System.out.println(table.getId().split(":")[4]); // 表类型 System.out.println(table.getTableType()); // 表创建时间 System.out.println(table.getCreateTime()); // 表修改时间 System.out.println(table.getModifyTime()); // 是否分区表 System.out.println(table.getPartitionKeys() != null && !table.getPartitionKeys().isEmpty()); // 表的技术元数据 TableTechnicalMetadata technicalMetadata = table.getTechnicalMetadata(); // 表的负责人(名称) System.out.println(technicalMetadata.getOwner()); // 是否压缩表 System.out.println(technicalMetadata.getCompressed()); // 输入格式(HMS, DLF等类型支持) System.out.println(technicalMetadata.getInputFormat()); // 输出格式(HMS, DLF等类型支持) System.out.println(technicalMetadata.getOutputFormat()); // 表序列化使用的类(HMS, DLF等类型支持) System.out.println(technicalMetadata.getSerializationLibrary()); // 表存储位置(HMS, DLF等类型支持) System.out.println(technicalMetadata.getLocation()); // 其他参数 Map<String, String> parameters = technicalMetadata.getParameters(); // 上次DDL表更时间(毫秒级时间戳),仅MaxCompute类型支持 System.out.println(parameters.get("lastDDLTime")); // 生命周期(单位:天),仅MaxCompute类型支持 System.out.println(parameters.get("lifecycle")); // 存储量(单位:字节),非实时,仅MaxCompute类型支持 System.out.println(parameters.get("dataSize")); // 表的业务元数据 TableBusinessMetadata businessMetadata = table.getBusinessMetadata(); if (businessMetadata != null) { // 使用说明 System.out.println(businessMetadata.getReadme()); // 自定义标签信息 List<TableBusinessMetadataTags> tags = businessMetadata.getTags(); if (tags != null && !tags.isEmpty()) { for (TableBusinessMetadataTags tag : tags) { System.out.println(tag.getKey() + ":" + tag.getValue()); } } // 上游产出任务 List<TableBusinessMetadataUpstreamTasks> upstreamTasks = businessMetadata.getUpstreamTasks(); if (upstreamTasks != null && !upstreamTasks.isEmpty()) { for (TableBusinessMetadataUpstreamTasks upstreamTask : upstreamTasks) { // 任务ID 与任务名称,可通过 GetTask API 获取任务详情信息 System.out.println(upstreamTask.getId() + ":" + upstreamTask.getName()); } } // 所属类目信息 List<List<TableBusinessMetadataCategories>> categories = businessMetadata.getCategories(); if (categories != null && !categories.isEmpty()) { // 多级类目的遍历 for (List<TableBusinessMetadataCategories> category : categories) { // 输出单个多级类目 System.out.println( category.stream() .map(TableBusinessMetadataCategories::getName) .collect(Collectors.joining("->")) ); } } // 扩展信息,仅MaxCompute类型支持 TableBusinessMetadataExtension extension = businessMetadata.getExtension(); if (extension != null) { // 所属项目空间ID System.out.println(extension.getProjectId()); // 所属环境类型,Prod 生产 / Dev 开发 System.out.println(extension.getEnvType()); // 收藏次数 System.out.println(extension.getFavorCount()); // 读取次数 System.out.println(extension.getReadCount()); // 浏览次数 System.out.println(extension.getViewCount()); } } return table; } catch (TeaException error) { // 此处仅做打印展示,请谨慎对待异常处理,在工程项目中切勿直接忽略异常。 // 错误 message System.out.println(error.getMessage()); // 诊断地址 System.out.println(error.getData().get("Recommend")); com.aliyun.teautil.Common.assertAsString(error.message); } catch (Exception _error) { TeaException error = new TeaException(_error.getMessage(), _error); // 此处仅做打印展示,请谨慎对待异常处理,在工程项目中切勿直接忽略异常。 // 错误 message System.out.println(error.getMessage()); // 诊断地址 System.out.println(error.getData().get("Recommend")); com.aliyun.teautil.Common.assertAsString(error.message); } return null; } /** * DataWorks OpenAPI : ListColumns * * @param listColumnsDto */ public ListColumnsResponseBodyPagingInfo listColumns(ListColumnsDto listColumnsDto) { try { Client client = dataWorksOpenApiClient.createClient(); ListColumnsRequest listColumnsRequest = new ListColumnsRequest(); // 数据表ID,可通过 ListTables 接口获取 listColumnsRequest.setTableId(listColumnsDto.getTableId()); // 字段名称,支持模糊匹配 listColumnsRequest.setName(listColumnsDto.getName()); // 字段注释,支持分词匹配 listColumnsRequest.setComment(listColumnsDto.getComment()); // 排序方式,支持 Name / Position (默认) listColumnsRequest.setSortBy(listColumnsDto.getSortBy()); // 排序方向,支持 Asc (默认) / Desc listColumnsRequest.setOrder(listColumnsDto.getOrder()); // 分页页码,默认为1 listColumnsRequest.setPageNumber(listColumnsDto.getPageNumber()); // 分页大小,默认为10,最大100 listColumnsRequest.setPageSize(listColumnsDto.getPageSize()); ListColumnsResponse response = client.listColumns(listColumnsRequest); // 获取符合要求的字段总数 System.out.println(response.getBody().getRequestId()); System.out.println(response.getBody().getPagingInfo().getTotalCount()); for (Column column : response.getBody().getPagingInfo().getColumns()) { // 字段ID System.out.println(column.getId()); // 字段名称 System.out.println(column.getName()); // 字段注释 System.out.println(column.getComment()); // 字段类型 System.out.println(column.getType()); // 字段位置 System.out.println(column.getPosition()); // 是否分区字段 System.out.println(column.getPartitionKey()); // 是否主键,仅MaxCompute类型支持 System.out.println(column.getPrimaryKey()); // 是否外键,仅MaxCompute类型支持 System.out.println(column.getForeignKey()); // 字段业务说明,目前MaxCompute, HMS(EMR集群), DLF类型支持 if (column.getBusinessMetadata() != null) { System.out.println(column.getBusinessMetadata().getDescription()); } } return response.getBody().getPagingInfo(); } catch (TeaException error) { // 此处仅做打印展示,请谨慎对待异常处理,在工程项目中切勿直接忽略异常。 // 错误 message System.out.println(error.getMessage()); // 诊断地址 System.out.println(error.getData().get("Recommend")); com.aliyun.teautil.Common.assertAsString(error.message); } catch (Exception _error) { TeaException error = new TeaException(_error.getMessage(), _error); // 此处仅做打印展示,请谨慎对待异常处理,在工程项目中切勿直接忽略异常。 // 错误 message System.out.println(error.getMessage()); // 诊断地址 System.out.println(error.getData().get("Recommend")); com.aliyun.teautil.Common.assertAsString(error.message); } return null; } /** * DataWorks OpenAPI : UpdateColumnBusinessMetadata * * @param updateColumnBusinessMetadataDto */ public boolean updateColumnBusinessMetadata(UpdateColumnBusinessMetadataDto updateColumnBusinessMetadataDto) { try { // 当前支持MaxCompute, DLF, HMS(EMR集群)类型 Client client = dataWorksOpenApiClient.createClient(); UpdateColumnBusinessMetadataRequest updateColumnBusinessMetadataRequest = new UpdateColumnBusinessMetadataRequest(); // 字段ID updateColumnBusinessMetadataRequest.setId(updateColumnBusinessMetadataDto.getId()); // 字段业务说明 updateColumnBusinessMetadataRequest.setDescription(updateColumnBusinessMetadataDto.getDescription()); UpdateColumnBusinessMetadataResponse response = client.updateColumnBusinessMetadata(updateColumnBusinessMetadataRequest); System.out.println(response.getBody().getRequestId()); // 是否更新成功 return response.getBody().getSuccess(); } catch (TeaException error) { // 此处仅做打印展示,请谨慎对待异常处理,在工程项目中切勿直接忽略异常。 // 错误 message System.out.println(error.getMessage()); // 诊断地址 System.out.println(error.getData().get("Recommend")); com.aliyun.teautil.Common.assertAsString(error.message); } catch (Exception _error) { TeaException error = new TeaException(_error.getMessage(), _error); // 此处仅做打印展示,请谨慎对待异常处理,在工程项目中切勿直接忽略异常。 // 错误 message System.out.println(error.getMessage()); // 诊断地址 System.out.println(error.getData().get("Recommend")); com.aliyun.teautil.Common.assertAsString(error.message); } return false; } /** * DataWorks OpenAPI : ListPartitions * * @param listPartitionsDto */ public ListPartitionsResponseBodyPagingInfo listPartitions(ListPartitionsDto listPartitionsDto) { try { // 当前支持 MaxCompute类型和HMS(EMR集群)类型 Client client = dataWorksOpenApiClient.createClient(); ListPartitionsRequest listPartitionsRequest = new ListPartitionsRequest(); // 数据表ID,可通过 ListTables 接口获取 listPartitionsRequest.setTableId(listPartitionsDto.getTableId()); // 分区名称,支持模糊匹配 listPartitionsRequest.setName(listPartitionsDto.getName()); // 排序方式,支持 Name(HMS方式默认,Maxcompute类型支持) / CreateTime (MaxCompute类型默认) / ModifyTime (MaxCompute类型支持)/ RecordCount(MaxCompute类型支持)/ DataSize(MaxCompute类型支持) listPartitionsRequest.setSortBy(listPartitionsDto.getSortBy()); // 排序方向,支持 Asc (默认) / Desc listPartitionsRequest.setOrder(listPartitionsDto.getOrder()); // 分页页码,默认为1 listPartitionsRequest.setPageNumber(listPartitionsDto.getPageNumber()); // 分页大小,默认为10,最大100 listPartitionsRequest.setPageSize(listPartitionsDto.getPageSize()); ListPartitionsResponse response = client.listPartitions(listPartitionsRequest); // 获取符合要求的分区总数 System.out.println(response.getBody().getPagingInfo().getTotalCount()); for (Partition partition : response.getBody().getPagingInfo().getPartitionList()) { // 所属数据表ID System.out.println(partition.getTableId()); // 分区名称 System.out.println(partition.getName()); // 创建时间(毫秒级时间戳) System.out.println(partition.getCreateTime()); // 修改时间(毫秒级时间戳) System.out.println(partition.getModifyTime()); // 分区记录数,仅MaxCompute类型支持 System.out.println(partition.getRecordCount()); // 分区存储量(单位:字节),仅MaxCompute类型支持 System.out.println(partition.getDataSize()); } return response.getBody().getPagingInfo(); } catch (TeaException error) { // 此处仅做打印展示,请谨慎对待异常处理,在工程项目中切勿直接忽略异常。 // 错误 message System.out.println(error.getMessage()); // 诊断地址 System.out.println(error.getData().get("Recommend")); com.aliyun.teautil.Common.assertAsString(error.message); } catch (Exception _error) { TeaException error = new TeaException(_error.getMessage(), _error); // 此处仅做打印展示,请谨慎对待异常处理,在工程项目中切勿直接忽略异常。 // 错误 message System.out.println(error.getMessage()); // 诊断地址 System.out.println(error.getData().get("Recommend")); com.aliyun.teautil.Common.assertAsString(error.message); } return null; } }
在MetaRestController中提供四个方法分别为
getTable
、listColumns
、listPartitions
与updateColumnBusinessMetadata
供前端进行调用。/** * 演示如何通过DataWorks OpenAPI构建自定义元数据平台 * * @author dataworks demo */ @RestController @RequestMapping("/meta") public class MetaRestController { @Autowired private MetaServiceProxy metaService; /** * 获取表详情 * * @param getTableDto * @return {@link Table} */ @CrossOrigin(origins = "http://localhost:8080") @GetMapping("/getTable") public Table getTable(GetTableDto getTableDto) { System.out.println("getTableDto = " + getTableDto); return metaService.getTable(getTableDto); } /** * 分页查询字段数据 * * @param listColumnsDto * @return {@link ListColumnsResponseBodyPagingInfo} */ @CrossOrigin(origins = "http://localhost:8080") @GetMapping("/listColumns") public ListColumnsResponseBodyPagingInfo listColumns(ListColumnsDto listColumnsDto) { System.out.println("listColumnsDto = " + listColumnsDto); return metaService.listColumns(listColumnsDto); } /** * 更新字段业务说明 * * @param updateColumnBusinessMetadataDto * @return {@link Boolean} */ @CrossOrigin(origins = "http://localhost:8080") @PostMapping("/updateColumnBusinessMetadata") public Boolean updateColumnBusinessMetadata(@RequestBody UpdateColumnBusinessMetadataDto updateColumnBusinessMetadataDto) { System.out.println("updateColumnBusinessMetadataDto = " + updateColumnBusinessMetadataDto); return metaService.updateColumnBusinessMetadata(updateColumnBusinessMetadataDto); } /** * 分页查询分区数据 * * @param listPartitionsDto * @return {@link ListPartitionsResponseBodyPagingInfo} */ @CrossOrigin(origins = "http://localhost:8080") @GetMapping("/listPartitions") public ListPartitionsResponseBodyPagingInfo listPartitions(ListPartitionsDto listPartitionsDto) { System.out.println("listPartitionsDto = " + listPartitionsDto); return metaService.listPartitions(listPartitionsDto); } }
前端:展示表基本信息、表字段信息以及表分区的信息。
import React from "react";
import moment from "moment";
import {
Form,
Grid,
Table,
Pagination,
Tag,
Field,
Input,
Button,
Select,
Dialog,
} from "@alifd/next";
import cn from "classnames";
import * as services from "../services";
import {
type ListColumnsInput,
type TableColumn,
type TableEntity,
type TablePartition,
} from "../services/meta";
import classes from "../styles/detailContent.module.css";
export interface Props {
item: TableEntity;
}
const formItemLayout = {
labelCol: {
fixedSpan: 6,
},
wrapperCol: {
span: 18,
},
labelTextAlign: "left" as const,
colon: true,
className: cn(classes.formItemWrapper),
};
const { Row, Col } = Grid;
const { Item } = Form;
const { Column } = Table;
const { Option } = Select;
const DetailContent: React.FunctionComponent<Props> = (props) => {
let columnsPageSize = 10;
let partitionsPageSize = 5;
const listColumnsField = Field.useField();
const listPartitionsField = Field.useField();
const labelAlign = "top";
const [detail, setDetail] = React.useState<Partial<TableEntity>>({});
const [columns, setColumns] = React.useState<Partial<TableColumn[]>>([]);
const [partitions, setPartitions] = React.useState<Partial<TablePartition[]>>(
[]
);
const [columnsTotal, setColumnsTotal] = React.useState<number>(0);
const [partitionTotal, setPartitionTotal] = React.useState<number>(0);
const [editingKey, setEditingKey] = React.useState<string>("");
const [editingDescription, setEditingDescription] =
React.useState<string>("");
const [updateColumnDialogVisible, setUpdateColumnDialogVisible] =
React.useState<boolean>(false);
const onUpdateColumnDialogOpen = () => {
setUpdateColumnDialogVisible(true);
};
const onUpdateColumnDialogOk = () => {
handleColumnDescriptionSave();
onUpdateColumnDialogClose();
};
const onUpdateColumnDialogClose = () => {
setUpdateColumnDialogVisible(false);
};
const handleEditColumnDescription = async (record: TableColumn) => {
setEditingKey(record.id);
setEditingDescription(record.businessMetadata?.description);
onUpdateColumnDialogOpen();
};
const handleColumnDesrciptionChange = async (e: string) => {
setEditingDescription(e);
};
const handleColumnDescriptionSave = async () => {
try {
const res = await services.meta.updateColumnDescription({
id: editingKey,
description: editingDescription,
});
if (!res) {
console.log("请求更新失败");
}
} catch (e) {
console.error("更新失败", e);
} finally {
listColumns();
setEditingKey("");
}
};
const getTableDetail = React.useCallback(async () => {
const response = await services.meta.getTableDetail({
id: props.item.id,
includeBusinessMetadata: true,
});
setDetail(response);
}, [props.item.id]);
const listColumns = React.useCallback(
async (pageNumber: number = 1) => {
listColumnsField.setValue("tableId", props.item.id);
listColumnsField.setValue("pageSize", columnsPageSize);
listColumnsField.validate(async (errors, values) => {
if (errors) {
return;
}
try {
const response = await services.meta.getMetaTableColumns({
pageNumber,
...values,
} as ListColumnsInput);
setColumns(response.columns);
setColumnsTotal(response.totalCount);
} catch (e) {
throw e;
} finally {
}
});
},
[listColumnsField]
);
const listPartitions = React.useCallback(
async (pageNumber: number = 1) => {
listPartitionsField.setValue("tableId", props.item.id);
listPartitionsField.setValue("pageSize", partitionsPageSize);
listPartitionsField.validate(async (errors, values) => {
if (errors) {
return;
}
try {
const response = await services.meta.getTablePartition({
pageNumber,
...values,
} as ListColumnsInput);
setPartitions(response.partitionList);
setPartitionTotal(response.totalCount);
} catch (e) {
throw e;
} finally {
}
});
},
[listPartitionsField]
);
React.useEffect(() => {
if (props.item.id) {
getTableDetail();
listColumns();
listPartitions();
}
}, [props.item.id]);
return (
<div className={cn(classes.detailContentWrapper)}>
<Dialog
v2
title="更新字段业务元数据"
visible={updateColumnDialogVisible}
onOk={onUpdateColumnDialogOk}
onClose={onUpdateColumnDialogClose}
>
<Row>
<Col>
<Item
{...formItemLayout}
labelAlign={labelAlign}
label="字段业务说明"
>
<Input
value={editingDescription}
onChange={(e) => handleColumnDesrciptionChange(e.toString())}
></Input>
</Item>
</Col>
</Row>
</Dialog>
<Form labelTextAlign="left">
<Row>
<Col>
<Item {...formItemLayout} label="表ID">
<span className={cn(classes.formContentWrapper)}>
{detail.id}
</span>
</Item>
</Col>
</Row>
<Row>
<Col>
<Item {...formItemLayout} label="MaxCompute项目">
<span className={cn(classes.formContentWrapper)}>
{detail?.id?.split(":").slice(-3, -2)[0]}
</span>
</Item>
</Col>
<Col>
<Item {...formItemLayout} label="schema名称">
<span className={cn(classes.formContentWrapper)}>
{detail?.id?.split(":").slice(-2, -1)[0]}
</span>
</Item>
</Col>
</Row>
<Row>
<Col>
<Item {...formItemLayout} label="表名称">
<span className={cn(classes.formContentWrapper)}>
{detail.name}
</span>
</Item>
</Col>
<Col>
<Item {...formItemLayout} label="注释">
<span className={cn(classes.formContentWrapper)}>
{detail.comment}
</span>
</Item>
</Col>
</Row>
<Row>
<Col>
<Item {...formItemLayout} label="表类型">
<span className={cn(classes.formContentWrapper)}>
{detail.tableType}
</span>
</Item>
</Col>
<Col>
<Item {...formItemLayout} label="是否压缩表">
<span className={cn(classes.formContentWrapper)}>
{detail.technicalMetadata?.compressed ? "是" : "否"}
</span>
</Item>
</Col>
</Row>
<Row>
<Col>
<Item {...formItemLayout} label="所属工作空间ID">
<span className={cn(classes.formContentWrapper)}>
{detail.businessMetadata?.extension?.projectId}
</span>
</Item>
</Col>
<Col>
<Item {...formItemLayout} label="环境">
<span className={cn(classes.formContentWrapper)}>
{detail.businessMetadata?.extension?.envType === "Dev"
? "开发"
: "生产"}
</span>
</Item>
</Col>
</Row>
<Row>
<Col>
<Item {...formItemLayout} label="表生命周期">
<span className={cn(classes.formContentWrapper)}>
{detail.technicalMetadata?.parameters["lifecycle"]
? `${detail.technicalMetadata?.parameters["lifecycle"]} 天`
: "永久"}
</span>
</Item>
</Col>
<Col>
<Item {...formItemLayout} label="存储空间">
<span
className={cn(classes.formContentWrapper)}
>{`${detail.technicalMetadata?.parameters["dataSize"]} Bytes`}</span>
</Item>
</Col>
</Row>
<Row>
<Col>
<Item {...formItemLayout} label="DDL上次变更时间">
<span className={cn(classes.formContentWrapper)}>
{moment(
Number(detail.technicalMetadata?.parameters["lastDDLTime"])
).format("YYYY-MM-DD HH:mm:ss")}
</span>
</Item>
</Col>
<Col>
<Item {...formItemLayout} label="是否为分区表">
<span className={cn(classes.formContentWrapper)}>
{detail.partitionKeys != null && detail.partitionKeys.length > 0
? "是"
: "否"}
</span>
</Item>
</Col>
</Row>
<Row>
<Col>
<Item {...formItemLayout} label="标签">
<div className={cn(classes.formContentWrapper)}>
{detail.businessMetadata?.tags?.map((tag, index) => (
<span key={index} className={cn(classes.tag)}>
<Tag type="primary" size="small">
{tag.key + (tag.value ? ":" + tag.value : "")}
</Tag>
</span>
))}
</div>
</Item>
</Col>
</Row>
<Row>
<Col>
<Item {...formItemLayout} label="类目">
<span className={cn(classes.formContentWrapper)}>
{detail.businessMetadata?.categories?.map((category, index) => (
<span key={index} className={cn(classes.tag)}>
<Tag type="normal" color="gray" size="small">
{category.map((obj) => obj.name).join("->")}
</Tag>
</span>
))}
</span>
</Item>
</Col>
</Row>
<Row>
<Col>
<Item {...formItemLayout} label="上游产出任务">
<span className={cn(classes.formContentWrapper)}>
{detail.businessMetadata?.upstreamTasks?.map((task, index) => (
<span key={index} className={cn(classes.tag)}>
<Tag type="primary" size="small">
{task.name + " (" + task.id + ")"}
</Tag>
</span>
))}
</span>
</Item>
</Col>
</Row>
<Row>
<Col>
<Item {...formItemLayout} label="浏览次数">
<span className={cn(classes.formContentWrapper)}>
<Tag type="primary" size="small">
{detail.businessMetadata?.extension?.viewCount}
</Tag>
</span>
</Item>
</Col>
<Col>
<Item {...formItemLayout} label="读取次数">
<span className={cn(classes.formContentWrapper)}>
<Tag type="primary" size="small">
{detail.businessMetadata?.extension?.readCount}
</Tag>
</span>
</Item>
</Col>
</Row>
<Row>
<Col>
<Item {...formItemLayout} label="收藏次数">
<span className={cn(classes.formContentWrapper)}>
<Tag type="primary" size="small">
{detail.businessMetadata?.extension?.favorCount}
</Tag>
</span>
</Item>
</Col>
</Row>
<Row>
<Col>
<Item {...formItemLayout} label="创建时间">
<span className={cn(classes.formContentWrapper)}>
{moment(detail.createTime).format("YYYY-MM-DD HH:mm:ss")}
</span>
</Item>
</Col>
<Col>
<Item {...formItemLayout} label="修改时间">
<span className={cn(classes.formContentWrapper)}>
{moment(detail.modifyTime).format("YYYY-MM-DD HH:mm:ss")}
</span>
</Item>
</Col>
</Row>
<br />
<Item label="字段详情" colon>
<Form
field={listColumnsField}
colon
fullWidth
style={{ marginTop: 10, marginBottom: 10 }}
>
<Row gutter="4">
<Col>
<Item
{...formItemLayout}
labelAlign={labelAlign}
label="名称"
name="name"
>
<Input placeholder="输入字段名称,模糊匹配" />
</Item>
</Col>
<Col>
<Item
{...formItemLayout}
labelAlign={labelAlign}
label="注释"
name="comment"
>
<Input placeholder="输入字段注释,模糊匹配" />
</Item>
</Col>
<Col>
<Item
{...formItemLayout}
labelAlign={labelAlign}
label="排序方式"
name="sortBy"
>
<Select defaultValue="Position" style={{ width: "100%" }}>
<Option value="Position">位置</Option>
<Option value="Name">名称</Option>
</Select>
</Item>
</Col>
<Col>
<Item
{...formItemLayout}
labelAlign={labelAlign}
label="排序方向"
name="order"
>
<Select defaultValue="Asc" style={{ width: "100%" }}>
<Option value="Asc">升序</Option>
<Option value="Desc">降序</Option>
</Select>
</Item>
</Col>
<Col>
<div
className={cn(
classes.searchPanelButtonWrapper,
classes.buttonGroup
)}
style={{ marginTop: "20px", textAlign: "right" }}
>
<Button type="primary" onClick={() => listColumns()}>
查询
</Button>
</div>
</Col>
</Row>
</Form>
<Table dataSource={columns}>
<Column title="名称" dataIndex="name" />
<Column title="类型" dataIndex="type" />
<Column title="注释" dataIndex="comment" />
<Column
title="业务说明"
dataIndex="businessMetadata"
cell={(value, index, record) => (
<div
style={{
display: "flex",
justifyContent: "space-between",
alignItems: "center",
}}
>
<span>{value?.description}</span>
<Button
onClick={() => handleEditColumnDescription(record)}
size="small"
type="primary"
text
style={{ marginLeft: 8 }}
>
编辑
</Button>
</div>
)}
/>
<Column
title="是否为主键"
dataIndex="primaryKey"
cell={(value) => (value ? "是" : "")}
/>
<Column
title="是否为外键"
dataIndex="foreignKey"
cell={(value) => (value ? "是" : "")}
/>
<Column
title="是否为分区字段"
dataIndex="partitionKey"
cell={(value) => (value ? "是" : "")}
/>
<Column title="位置" dataIndex="position" />
</Table>
<Pagination
total={columnsTotal}
pageSize={columnsPageSize}
onChange={listColumns}
showJump={false}
className={cn(classes.tablePaginationWrapper)}
/>
</Item>
<br />
<Item label="分区详情" style={{ marginBottom: 32 }} colon>
<Form
field={listPartitionsField}
colon
fullWidth
style={{ marginTop: 10, marginBottom: 10 }}
>
<Row gutter="4">
<Col>
<Item
{...formItemLayout}
labelAlign={labelAlign}
label="名称"
name="name"
>
<Input placeholder="输入分区名称,模糊匹配" />
</Item>
</Col>
<Col>
<Item
{...formItemLayout}
labelAlign={labelAlign}
label="排序方式"
name="sortBy"
>
<Select defaultValue="CreateTime" style={{ width: "100%" }}>
<Option value="CreateTime">创建时间</Option>
<Option value="ModifyTime">修改时间</Option>
<Option value="Name">分区名称</Option>
<Option value="RecordCount">分区记录数</Option>
<Option value="DataSize">分区大小</Option>
</Select>
</Item>
</Col>
<Col>
<Item
{...formItemLayout}
labelAlign={labelAlign}
label="排序方向"
name="order"
>
<Select defaultValue="Asc" style={{ width: "100%" }}>
<Option value="Asc">升序</Option>
<Option value="Desc">降序</Option>
</Select>
</Item>
</Col>
<Col>
<div
className={cn(
classes.searchPanelButtonWrapper,
classes.buttonGroup
)}
style={{ marginTop: "20px", textAlign: "right" }}
>
<Button type="primary" onClick={() => listPartitions()}>
查询
</Button>
</div>
</Col>
</Row>
</Form>
<Table dataSource={partitions} emptyContent={<span>非分区表</span>}>
<Column title="名称" dataIndex="name" />
<Column
title="创建时间"
dataIndex="createTime"
cell={(value) => moment(value).format("YYYY-MM-DD HH:mm:ss")}
/>
<Column
title="修改时间"
dataIndex="modifyTime"
cell={(value) => moment(value).format("YYYY-MM-DD HH:mm:ss")}
/>
<Column
title="分区大小"
dataIndex="dataSize"
cell={(value) => `${value} bytes`}
/>
<Column title="分区记录数" dataIndex="recordCount" />
</Table>
<Pagination
total={partitionTotal}
pageSize={partitionsPageSize}
onChange={listPartitions}
showJump={false}
className={cn(classes.tablePaginationWrapper)}
/>
</Item>
</Form>
</div>
);
};
export default DetailContent;
完成上述代码开发后,您可在本地部署并运行工程代码。部署并运行的操作请参见通用操作:本地部署运行。
三、查找表血缘
当一张表口径发生变更时,您可以通过DataWorks OpenAPI进行上下游任务的血缘分析,查找表对应的节点。
使用ListLineages来查询数据表的上下游血缘。
后端:使用MetaServiceProxy查询表血缘
在MetaServiceProxy中创建listTableLineages方法来调用OpenAPI服务获取表的上下游血缘实体和关系信息。
/** * @author dataworks demo */ @Service public class MetaServiceProxy { @Autowired private DataWorksOpenApiClient dataWorksOpenApiClient; /** * DataWorks OpenAPI : ListLineages * * @param listTableLineagesDto */ public ListLineagesResponseBodyPagingInfo listTableLineages(ListTableLineagesDto listTableLineagesDto) { try { Client client = dataWorksOpenApiClient.createClient(); ListLineagesRequest listLineagesRequest = new ListLineagesRequest(); // 血缘查询方向,是否为上游 boolean needUpstream = "up".equalsIgnoreCase(listTableLineagesDto.getDirection()); // 数据表ID,可通过 ListTables 接口获取 // 分区名称,支持模糊匹配 if (needUpstream) { listLineagesRequest.setDstEntityId(listTableLineagesDto.getTableId()); listLineagesRequest.setSrcEntityName(listTableLineagesDto.getName()); } else { listLineagesRequest.setSrcEntityId(listTableLineagesDto.getTableId()); listLineagesRequest.setDstEntityName(listTableLineagesDto.getName()); } // 排序方式,支持 Name listLineagesRequest.setSortBy(listTableLineagesDto.getSortBy()); // 排序方向,支持 Asc (默认) / Desc listLineagesRequest.setOrder(listTableLineagesDto.getOrder()); // 分页页码,默认为1 listLineagesRequest.setPageNumber(listTableLineagesDto.getPageNumber()); // 分页大小,默认为10,最大100 listLineagesRequest.setPageSize(listTableLineagesDto.getPageSize()); // 是否需要返回具体的血缘关系信息 listLineagesRequest.setNeedAttachRelationship(true); ListLineagesResponse response = client.listLineages(listLineagesRequest); // 获取符合要求的血缘实体总数 System.out.println(response.getBody().getPagingInfo().getTotalCount()); if (response.getBody().getPagingInfo().getLineages() == null) { response.getBody().getPagingInfo().setLineages(Collections.emptyList()); } if (response.getBody().getPagingInfo().getLineages() != null) { for (ListLineagesResponseBodyPagingInfoLineages lineage : response.getBody().getPagingInfo().getLineages()) { LineageEntity entity; // 获取对应血缘实体 if (needUpstream) { entity = lineage.getSrcEntity(); } else { entity = lineage.getDstEntity(); } // 所属实体ID System.out.println(entity.getId()); // 所属实体名称 System.out.println(entity.getName()); // 所属实体的属性信息 System.out.println(entity.getAttributes()); // 实体间血缘关系列表 List<LineageRelationship> relationships = lineage.getRelationships(); if (relationships != null) { relationships.forEach(relationship -> { // 血缘关系ID System.out.println(relationship.getId()); // 创建时间 System.out.println(relationship.getCreateTime()); // 血缘关系对应的任务 LineageTask task = relationship.getTask(); if (task != null) { // 任务ID,对于DataWorks任务,可以通过GetTask获取任务详情信息 System.out.println(task.getId()); // 任务类型 System.out.println(task.getType()); // 任务属性信息 Map<String, String> attributes = task.getAttributes(); if (attributes != null) { // 对于DataWorks任务,可以从中获取任务和实例的属性信息 // 项目空间ID System.out.println(attributes.get("projectId")); // 任务ID String taskId = attributes.containsKey("taskId") ? attributes.get("taskId") : attributes.get("nodeId"); System.out.println(taskId); // 实例ID String taskInstanceId = attributes.containsKey("taskInstanceId") ? attributes.get("taskInstanceId") : attributes.get("jobId"); System.out.println(taskInstanceId); } } }); } } return response.getBody().getPagingInfo(); } } catch (TeaException error) { // 此处仅做打印展示,请谨慎对待异常处理,在工程项目中切勿直接忽略异常。 // 错误 message System.out.println(error.getMessage()); // 诊断地址 System.out.println(error.getData().get("Recommend")); com.aliyun.teautil.Common.assertAsString(error.message); } catch (Exception _error) { TeaException error = new TeaException(_error.getMessage(), _error); // 此处仅做打印展示,请谨慎对待异常处理,在工程项目中切勿直接忽略异常。 // 错误 message System.out.println(error.getMessage()); // 诊断地址 System.out.println(error.getData().get("Recommend")); com.aliyun.teautil.Common.assertAsString(error.message); } return null; } }
在MetaRestController中提供listTableLineages方法供前端进行调用。
/** * 演示如何通过DataWorks OpenAPI构建自定义元数据平台 * * @author dataworks demo */ @RestController @RequestMapping("/meta") public class MetaRestController { @Autowired private MetaServiceProxy metaService; /** * 查询表的血缘关系 * * @param listTableLineagesDto * @return */ @CrossOrigin(origins = "http://localhost:8080") @GetMapping("/listTableLineages") public ListLineagesResponseBodyPagingInfo listTableLineages(ListTableLineagesDto listTableLineagesDto) { System.out.println("listTableLineagesDto = " + listTableLineagesDto); return metaService.listTableLineages(listTableLineagesDto); } }
前端:展示表血缘关系
import React from 'react';
import Graphin, { Behaviors } from '@antv/graphin';
import type { IUserNode, IUserEdge, GraphinData, GraphEvent } from '@antv/graphin';
import cn from 'classnames';
import * as services from '../services';
import type { TableEntity, ListTableLineageOutput, LineageEntity } from '../services/meta';
import classes from '../styles/lineageContent.module.css';
import '@antv/graphin/dist/index.css';
export interface Props {
item: TableEntity;
}
function transNode(entity: LineageEntity | TableEntity, direction?: string): IUserNode {
return {
id: entity.id,
label: entity.name,
data: {
...entity,
direction,
},
style: {
label: {
value: entity.name,
}
},
}
}
function transEdge(source: LineageEntity | TableEntity, target: LineageEntity | TableEntity): IUserEdge {
return {
source: source.id,
target: target.id,
style: {
keyshape: {
lineDash: [8, 4],
lineWidth: 2,
},
animate: {
type: 'line-dash',
repeat: true,
},
}
};
}
function parse(
source: LineageEntity | TableEntity,
data: ListTableLineageOutput,
direction: string,
): [IUserNode[], IUserEdge[]] {
const nodes: IUserNode[] = [];
const edges: IUserEdge[] = [];
data.lineages.forEach((lineageRelationship) => {
if (direction === 'down') {
nodes.push(transNode(lineageRelationship.dstEntity, direction));
edges.push(transEdge(source, lineageRelationship.dstEntity));
} else {
nodes.push(transNode(lineageRelationship.srcEntity, direction));
edges.push(transEdge(lineageRelationship.srcEntity, source));
}
});
return [nodes, edges];
}
function mergeNodes(prev: IUserNode[], next: IUserNode[]) {
const result: IUserNode[] = prev.slice();
next.forEach((item) => {
const hasValue = prev.findIndex(i => i.id === item.id) >= 0;
!hasValue && result.push(item);
});
return result;
}
function mergeEdges(source: IUserEdge[], target: IUserEdge[]) {
const result: IUserEdge[] = source.slice();
target.forEach((item) => {
const hasValue = source.findIndex(i => i.target === item.target && i.source === item.source) >= 0;
!hasValue && result.push(item);
});
return result;
}
const { ActivateRelations, DragNode, ZoomCanvas } = Behaviors;
const LineageContent: React.FunctionComponent<Props> = (props) => {
const ref = React.useRef<Graphin>();
const [data, setData] = React.useState<GraphinData>({ nodes: [], edges: [] });
const getTableLineage = async (
collection: GraphinData,
target: TableEntity | LineageEntity,
direction: string,
) => {
if (!direction) {
return collection;
}
const response = await services.meta.getTableLineage({
direction,
tableId: target.id,
});
const [nodes, edges] = parse(target, response, direction);
collection.nodes = mergeNodes(collection.nodes!, nodes);
collection.edges = mergeEdges(collection.edges!, edges);
return collection;
};
const init = async () => {
let nextData = Object.assign({}, data, { nodes: [transNode(props.item)] });
nextData = await getTableLineage(nextData, props.item, 'up');
nextData = await getTableLineage(nextData, props.item, 'down');
setData(nextData);
ref.current!.graph.fitCenter();
};
React.useEffect(() => {
ref.current?.graph && init();
}, [
ref.current?.graph,
]);
React.useEffect(() => {
const graph = ref.current?.graph;
const event = async (event: GraphEvent) => {
const source = event.item?.get('model').data;
let nextData = Object.assign({}, data);
nextData = await getTableLineage(nextData, source, source.direction);
setData(nextData);
};
graph?.on('node:click', event);
return () => {
graph?.off('node:click', event);
};
}, [
data,
ref.current?.graph,
]);
return (
<div className={cn(classes.lineageContentWrapper)}>
<Graphin
data={data}
ref={ref as React.LegacyRef<Graphin>}
layout={{
type: 'dagre',
rankdir: 'LR',
align: 'DL',
nodesep: 10,
ranksep: 40,
}}
>
<ActivateRelations />
<DragNode/>
<ZoomCanvas enableOptimize />
</Graphin>
</div>
);
};
export default LineageContent;
完成上述代码开发后,您可在本地部署并运行工程代码。部署并运行的操作请参见通用操作:本地部署运行。
四、查找表对应的任务
当一张表口径发生变更时,您可以通过DataWorks OpenAPI、OpenData、消息订阅的方式进行下游任务的血缘分析,查找表对应的节点。具体操作如下。
消息目前支持表变更、任务变更等。企业版用户可以对接表变更的消息,当接收到表变更的时候,您可以查看表的血缘关系。
血缘任务
通过ListLineages,查看表的血缘关系,从血缘关系的Task 中获取DataWorks 任务ID (以及任务实例ID)。
根据获取的任务ID,使用GetTask获取任务详情。
产出任务
通用操作:本地部署运行
准备依赖环境。
您需准备好以下依赖环境:java8及以上、maven构建工具、node环境、pnpm工具。您可以执行以下命令来确定是否具备上述环境:
npm -v // 如果已安装成功,执行命令将展示版本号,否则会报没有命令错误 java -version // 如果已安装成功,执行命令将展示版本号,否则会报没有命令错误 pnpm -v // 如果已安装成功,执行命令将展示版本号,否则会报没有命令错误
下载工程代码并执行以下命令。
工程代码下载链接:meta-api-demo.zip。
pnpm i
在范例工程中的backend/src/main/resources路径下找到application.properties文件,修改文件中的核心参数。
api.access-key-id与api.access-key-secret需修改为您阿里云账号的AccessKey ID 和 AccessKey Secret。
说明您可以参考访问密钥管理(AccessKey)获取AccessKey等阿里云账号的相关信息。
api.region-id、api.endpoint需修改为待调用OpenAPI的地域信息。
其他参数可根据实际情况修改,修改后的填写示例如下。
在工程根目录下执行以下命令,运行示例实践代码。
npm run dev