Data source table details

Area

Description

① Basic information

Shows the metadata name, environment, tags, and description. You can also perform the following operations:

For Elasticsearch data source tables, you can search metadata, view asset details, switch between production and development objects, modify tags, add favorites, report quality issues, request data source permissions, and export fields. Field export is supported only for table objects.

• If the metadata is deleted from the source system, its name is grayed out. You can click the icon next to the name to go to the Metadata Checklist page and delete the metadata that is no longer used.

• Search Metadata: Enter keywords such as a table name in the upper-right search box to find data source tables. You can also switch to view other asset details.

• View Asset Details: If the current object is listed in the Asset Directory, you can click this button to go to the directory details page and view the listing information.

• View Production/Development Object: Click this button in the upper-right corner to switch to the object's details page in the production or development environment.

• Tag: Displays the tags that are configured for the current asset. To modify tags, click Edit.

  • Each tag value can be up to 128 characters long.

  • You can configure a maximum of 20 tags for each asset object.

• Favorite: Add or remove the asset from favorites. The 30 most recent favorites appear in Asset Checklist > My Footprint. View all favorites in the Personal Data Center. For more information, see View and manage my favorited assets.

• Request Permission: Click to go to the permission request page for the current data table. For more information, see Request, renew, and return table permission.

• Feedback quality issues: This option is unavailable for deleted metadata. Report quality issues encountered during asset usage (such as unstable data output or inaccurate data) to the quality owner for timely resolution. For configuration details, see Add and manage issue checklist.

  Requires the Data Quality module. You can report quality issues only for production-environment data source tables and view the processing flow and results in the Data Quality issue checklist.

• Generate SELECT Statement: Generate a query statement for the current table, with an option to add escape characters. Copy and run the statement on the ad hoc query or analysis page.

• View DDL Statement: Click More in the upper-right corner and select View DDL Statement to view the DDL statement for the current metadata in its data source. You can also select a Data Source Type and click Generate DDL Statement to create a table with the same structure in the specified data source type. Select Automatically Add Escape Characters to include escape characters for the selected source type, reducing issues with system keyword escaping.

• Request Data Source Permission: Click More in the upper-right corner and select Request Data Source Permission to go to the permission request page for the current data source. For more information, see Request, renew, and return data source permission.

• Export Fields: Click More in the upper-right corner and select Export Fields to export the metadata field information in CSV format.

② Detail information

Displays the attribute and field information of tables and views.

• Detail: Displays the business and technical attributes of the metadata.

  • Business attributes: Includes source system information.

  • Technical attributes: Includes the source data source, number of table rows, highest sensitivity level (requires the Data Security feature), whether the table is a lakehouse table, lakehouse table format, table storage mode, storage class (internal or foreign table), storage size, and location information.

    • Table storage mode: If the data table is a lakehouse table and its format is Hudi, this information can be collected from the table's data source. This information cannot be collected for StarRocks, SelectDB, or Doris data sources.

    • Location: The storage address of the current table. For example: hdfs://node01.exp5:8020/user/hive/warehouse.

      Note

      • The technical attributes of a view only include the source data source information.

      • Storage class information is not available for MySQL or Microsoft SQL Server data sources.

      • For Oracle tables, the storage size is an estimate based on the number of rows and the average space used per row in bytes. It does not include statistics for indexes, unused space, or fragmentation. Therefore, the estimated value is lower than the actual space occupied.

      • If the data source table is a partitioned table, the row count is for reference only. The system does not currently support collecting the row count of the main table in a partitioned table.

      • For Greenplum data sources, you can view location information if the storage class is a foreign table.

      • For Amazon Redshift data sources, you can only view the source data source and storage class information.

      • For Elasticsearch data sources, if the object is a table (index), you can only view the document count and storage size.

• Field Information: Displays the field information for tables and views. This includes the field name, data type, description, whether the field can be null, default value, and data classification or sensitivity level (requires the Data Security module). For Elasticsearch data sources, if the object is a table (index), you can view the field name and data type.

  Search and filter: You can search for fields by name or description. You can also filter for specific fields by data classification or sensitivity level (requires Data Security).

• Partition Info: View the partition information of the data table. Multi-level partitions are displayed as a combination of all partition levels, separated by forward slashes (/). For example: ds=20221001/pt1=a/pt2=b.

  Note

  • The partition record count and storage size are for reference only.

  • For Elasticsearch data sources, you cannot view partition information.

③ Lineage & impact

• Lineage: Displays the data lineage between tables and fields.

  • Table-level lineage sources include integrated sync tasks, custom lineage compute tasks, SQL compute tasks that the system can automatically parse, and logical table tasks. For more information, see Table-level lineage.

  • Field lineage sources include custom lineage compute tasks, and SQL compute tasks and logical table tasks that the system can automatically parse. For more information, see Field lineage.

    Note

    The system parses table and field lineage in the developer environment when a task is submitted. It parses lineage in the production environment when the task is published. A single task submission or publication can parse a maximum of 100,000 lineage relationships. If this limit is exceeded, the lineage is not recorded and cannot be displayed in the Asset Directory.

• Impact: Impact is divided into data table impact and sync impact.

  • Data table impact: Displays the direct child tables that reference the current table and the indirect descendant tables that reference the direct child tables. You can export the data or display only the direct child tables.

    • Export Data: Export the data to an Excel file for business analysts to browse.

    • Display Only Direct Descendant Tables: Select this option to display only the downstream tables that are directly affected by the current table. Deselect this option to display all affected downstream tables, up to 15 levels. The direct downstream tables of the current table are considered the first level.

  • Sync impact: Displays the sync tasks where the current table and its descendant tables are used as source tables. You can export the data or display only the impact of the current table.

    • Export Data: Export the data to an Excel file for business analysts to browse.

    • Display Only Current Table Impact: Select this option to display only the sync tasks where the current table is used as an input table. Deselect this option to display the sync tasks where the current table and all its affected downstream tables are used as input tables, up to 15 levels. The direct downstream tables of the current table are considered the first level.

④ Quality overview

Enable the Data Quality feature to view the rule verification overview and quality monitoring rules for the current table. Click View Report Details or View Rule Details to go to the Data Quality module for more details. The quality overview is available only for data tables that support quality monitoring. For supported data sources, see Supported data sources in Dataphin.

You can create quality rules and view the quality overview only for data source tables in the production environment.

⑤ Data exploration

With the Data Quality feature enabled, you can run data explorations on supported data source tables to understand the data overview and assess availability and potential threats. To enable automatic exploration, configure the settings in Administration > Metadata Center > Exploration and Analysis. For details about exploration tasks, see Create a data exploration task. For supported data sources, see Exploration partitions and ranges supported by different data sources.

⑥ Data preview

With query permission on the current table, data preview returns results for fields where you have SELECT permission. Up to 50 entries are shown. To request query permission, see Request, renew, and return table permission.

You can search or filter by field, view single-row details, auto-adjust column widths, and transpose rows and columns. Click the sort icon next to a field to sort by No Sort, Ascending, or Descending. Double-click a field value to copy it.

⑦ Asset information

Displays the basic and usage information of the data source table.

• Basic Information: Includes the environment, table type, last collection time, and collection task of the data source table.

  • Last Collection Time: The time when the last successful run of the collection task for the current data source table was completed. The format is YYYY-MM-DD hh:mm:ss.

  • Collection Task: The collection task information for the current data source table.

• Usage Information: Includes the number of favorites and page views for the data source table.

  • Number Of Favorites: The number of users who have added the current data source table to their favorites. This value is updated in real time.

  • Page Views: The number of page views (PV) for the current data source table. The count increases by one each time the page is refreshed. This value is updated in real time.

Ordinal number

Description

① Quick actions

• Search: Search for expanded nodes on the current page by table name. Click the left or right arrow next to the search box to switch between the matched nodes. The system locates the first matched node by default.

• Display Only Direct Lineage: Click Filter and select Display Only Direct Lineage to display the upstream and downstream tables that have a direct lineage relationship with the current table.

  Note

  • Direct lineage refers to upstream and downstream assets that are read from or written to using statements such as SELECT and INSERT.

  • Associated lineage refers to related assets that are referenced using methods such as JOIN, GROUP BY, and WHERE conditions.

  • Table-level lineage displays the corresponding identifier if it contains any fields with direct lineage.

• Hide Data Source Table: Click Filter and select Hide Data Source Table to hide table objects from the data source and their downstream lineage.

• Hide Deleted Tables: Click Filter and select Hide Deleted Tables. Lineage is parsed when tasks are submitted and published. In some cases, if a data table is not deleted by a Dataphin scheduling task (for example, it is deleted by an ad hoc query or directly in the DPI engine), the lineage is not updated and is still displayed in the lineage graph. If you select this option, deleted table objects are not displayed.

② Legend

Table-level lineage supports the following data tables: Physical Table, Logical Dimension Table, Logical Fact Table, Logical Summary Table, Logical Tag Table, View, Materialized View, Logical View, Meta Table, Mirror Table, and Datasource Table.

③ Lineage graph display

Displays the full lineage graph. Expand multiple levels of upstream or downstream nodes and search by table name. If a circular dependency exists, you cannot expand further and must view downstream lineage from the start node.

• Central node: Displays the name of the current asset. The node is marked with Current Table in the upper-left corner.

• Fields in the table: Hover over a node and click the drop-down arrow on the right to perform a fuzzy search by field name. Click a field name to go to the asset details page and view the lineage relationship.

④ Object details

Hover over a table to view its details.

For a data source table, the details include its Name, Object Type, Data Source Type, Data Source, and Lineage Source. You can also perform the following operations: View Lineage, View DDL, and Request Permission.

• If the metadata is deleted from the source system, you can click the icon next to the name to go to the Metadata Checklist page and delete the metadata that is no longer used.

• View Lineage: Click to view the table-level lineage of the table.

• View DDL Statement: You can click this button to view the DDL statement for a physical table, physical view, or mirror table.

• Request Permission: This operation is not supported if the metadata is deleted. For metadata with a Normal status, you can click this button to go to the table's permission request page.

Ordinal number

Description

① Quick actions

• Search: Search for expanded nodes on the current page by field name. Click the left or right arrow next to the search box to switch between the matched nodes. The system locates the first matched node by default.

• Display Only Direct Lineage: Select this option to display only the upstream and downstream fields that have a direct lineage relationship with the current field.

  Note

  • Direct lineage refers to upstream and downstream assets that are read from or written to using statements such as SELECT and INSERT. In a field lineage graph, if the calculation logic of a field is processed using a where condition in a filter component or a built-in function in a field calculation component, the lineage is displayed as direct lineage.

  • Associated lineage refers to related assets that are referenced using methods such as JOIN, GROUP BY, and WHERE conditions.

• Hide Deleted Fields: Lineage is parsed when tasks are submitted and published. In some cases, if a data table is not deleted by a Dataphin scheduling task (for example, it is deleted by an ad hoc query or directly in the DPI engine), the lineage is not updated and is still displayed in the lineage graph. If you select this option, deleted table objects are not displayed.

② Legend

Field lineage supports the following fields: Compute Source Table Field and Data Source Table Field.

③ Lineage graph display

Displays the full lineage graph. Expand multiple levels of upstream or downstream nodes and search by field name. If a circular dependency exists, you cannot expand further and must view downstream lineage from the start node.

Central node: Displays the current field and its table name. The node is marked with Current Field in the upper-left corner. You can perform a fuzzy search by field keyword to switch the view to the lineage graph of a different field.

④ Object details

Hover over a field to view its details. The details include the Name, Object Type, Data Source Type, Table, Owner, Data Type, Description, and Lineage Source. You can also perform the View Lineage operation.

• Table: If a collection task is configured for the data source table, click the table name to go to the details page of the data source to which the table belongs.

• Request Permission: For metadata with a Normal status, you can click this button to go to the table's permission request page.

• View Lineage: Click to view the lineage information of the field as the central field.