Data lineage

Overview

Data lineage displays the data dependencies among tables, views, and columns. You can use the data catalog to trace metric origins, identify the impact of changes, and view the original SQL statements that form dependency relationships. After an intermediate table or column is deleted, you can still trace historical lineage that was collected before the deletion.

For example, when the net revenue in a business report changes, you can trace upstream to find the sales amounts and refund amounts involved in the calculation. Before you adjust the calculation logic, you can expand downstream to find the summary tables and views that need to be reviewed.

View table-level lineage

This example uses the order net revenue table dwd_order_profit as the central object. This table reads order details and refund summaries, and then provides net revenue data to daily business summaries, customer value summaries, and other downstream analysis objects.

1. Log on to the AnalyticDB for MySQL console, select a region, and go to the target cluster.

2. In the target cluster, choose Data Management > Data Catalog. Select a database and the target table, and then click the Lineage tab.

3. Starting from the direct upstream and downstream objects of the current object, click the plus icon on both sides of a node to expand the dependencies layer by layer.

4. Use the zoom, fit-to-canvas, and Zoom In buttons in the upper-right corner of the canvas to view long processing chains.

Expand upstream along order details to view orders, order items, customers, and products. Continue expanding along the refund summary to find refund records. Expand downstream to view daily business summaries and customer value summaries, which reveal channel business metrics, business overview views, and customer segmentation views.

An orange border identifies the current object, and a purple icon identifies a view. The levels in the graph are calculated relative to the current object. You can expand the analysis scope from direct dependencies to a complete business branch by expanding layers.

Trace column origins

Table-level relationships help you locate related objects. To explain the data source of a metric, you need to further check the column level. The following example uses the net revenue column net_revenue:

1. In the dwd_order_profit node, click Expand Columns. If the table has many columns, use pagination to find net_revenue.

2. Select net_revenue and click the plus icon on the left side of the column to view upstream columns. Related columns and expanded connections are highlighted in blue.

3. Click the plus icon on the right side of the column to view downstream columns and extend the source investigation to impact analysis.

The two inputs of net revenue are:

• dwd_order_detail.sales_amount: sales amount.

• dwd_refund_summary.refund_amount: refund amount.

When a metric is abnormal, you can check the sales amount summary and refund amount separately, and then use the SQL evidence on the connections to confirm the actual calculation method.

Assess downstream impact

Before you adjust refund rules or net revenue calculation logic, check the downstream of the target column. In this example, net_revenue flows to the same-named columns in two summary tables. Both analysis chains should be included in the review scope.

Continue expanding along table-level relationships to find the analysis objects after these summary tables. When confirming the impact of a change, use the corresponding SQL to check calculation, join, and filter conditions.

Trace historical lineage after intermediate nodes are deleted

As long as the related lineage was collected before deletion, you can still trace historical chains to find earlier data sources and later consumers after intermediate tables are cleaned up or columns are dropped.

ods_sales
  → dwd_sales_base
  → tmp_revenue_bridge (deleted)
  → dws_daily_revenue
  → ads_revenue_report

dwd_sales_base.net_revenue
  → dwd_column_bridge.net_revenue (column deleted, table still exists)
  → dws_column_revenue.net_revenue
  → downstream report columns

View SQL evidence

Click the information icon on a connection to open the Lineage Evidence panel, where you can view the query ID, SQL type, lineage type, and occurrence time.

On the connection from the refund amount to net revenue, the panel shows the actual INSERT_SELECT statement. Scroll down to the raw SQL to see that the detail sales amounts are aggregated by order, joined with the refund summary through a LEFT JOIN, and the net revenue is calculated by using the following expression:

CAST(
  d.sales_amount - COALESCE(r.refund_amount, 0)
  AS DECIMAL(18, 2)
) AS net_revenue

COALESCE treats amounts without refund records as 0. Combined with the upstream column relationships, you can verify the example logic that net revenue equals sales amount minus successful refund amounts.

The confidence level indicates the certainty of the lineage inference and does not mean that the business data has passed quality validation.

Focus the analysis scope

On the Lineage tab, select a direction and a time range to focus on a specific chain.

The time range is based on the occurrence time of lineage events, not business fields such as order dates. The page initially displays table-level relationships. If the column relationship count is 0, expand the target column first and then view the source or destination.