Logical table and field details

Area

Description

① Overview

Displays information such as the table type, environment, name, tags, and description. You can also perform the following operations:

• Search for other asset objects: Quickly search for and switch to the details of other assets.

• View Asset Details: If the current object is published to the Asset Directory, you can go to the directory details page to view the published information.

• View Production/Development Object: Quickly switch to the object details in the corresponding production or developer environment.

• Tag: Displays the tag values that are configured for the current asset. To modify tags, click Edit:

  • Each tag value can be up to 128 characters in length.

  • You can configure a maximum of 20 tag values for each asset object.

  • Super administrators can modify asset tags for all table types. The owner of a table can modify the asset tags for that table. A subject area architect can modify the asset tags for logical tables within the subject areas that they manage.

• Favorite: Click to add the asset to or remove it from your favorites. After you add an asset to your favorites, you can view the 30 most recently added assets on the My Footprint tab of the Asset Checklist page for quick access. You can also view all your favorite assets in the Personal Data Center. For more information, see View and manage my favorite assets.

• Go Analysis: Click to go to the Notebook page and automatically create a Notebook. For more information, see Create a Notebook.

• Request Permission: Click to go to the permission request page for the current table. For more information, see Request, renew, and return table permissions.

• Report Quality Issue: Use this feature to report quality issues to the person in charge of data quality. Examples of quality issues include unstable data output and inaccurate data content. This notifies the relevant personnel to make corrections and helps improve asset availability and health. For more information about the configuration, see Add and manage an issue checklist.

  You must activate the Data Quality module to use the report quality issue feature. You can then view the processing flow and results of the reported issues in the issue checklist of the Data Quality module.

• Generate SELECT Statement: In the upper-right corner, click More and select Generate SELECT Statement. This generates a query statement for the current table. You can select whether to add escape characters. You can copy the query statement to the ad hoc query or analysis page to query data.

• View DDL Statement: In the upper-right corner, click More and select View DDL Statement. DML operations are not supported for logical tables. This section displays the DDL statement for the physical table that has the same schema as the logical table. You can use this statement to quickly create a target table in a data source for data synchronization. You can also select a Data Source Type and click Generate DDL Statement. The system generates a DDL statement to create a table that has the same structure as the current table in the specified data source. If you select Automatically Add Escape Characters, the generated DDL statement automatically includes the appropriate escape characters for the selected source type. This reduces issues such as incorrect escaping of system keywords.

  

• Edit Table: In the upper-right corner, click More and select Edit Table. You are redirected to the Table Management page in the Developer module. On this page, you can edit the DDL information of the table. For more information, see Create and manage an offline physical table.

• Export Fields: In the upper-right corner, click More and select Export Fields to export the field information from the table. The file is in CSV format. This allows other developers or business analysts to quickly analyze and use the data.

• View Transfer Records: In the upper-right corner, click More and select View Transfer Records. This displays the 100 most recent owner transfer records for the data table.

• View Permission List: In the upper-right corner, click More and select View Permission List to view the permission information of the data table.

• Refresh Metadata: In the upper-right corner, click More and select Refresh Metadata. If a data table was not created in Dataphin, or if a query for a new table returns no results due to a delay in metadata retrieval, you can click Refresh Metadata. This retrieves the latest metadata and refreshes the metadata of the specified data table in Dataphin. 

② Details

• Displays table, field, and partition information.

  • Detail: Displays the properties of the table. This includes the data subject area, project, highest sensitivity level (requires the data security feature), whether it is a partitioned table, whether it is a manually created table on the analysis platform, whether it is a lakehouse table, the lakehouse table format, table storage mode, storage format, storage size, lifecycle, and location.

    • Project: The project to which the table belongs. Click the project name to go to the asset details page of the project.

    • Highest Sensitivity Level: The highest sensitivity level among the fields in the table. This helps you quickly understand the confidentiality of the table data. Data Classification levels from low to high are L1, L2, L3, and L4 (from public to top secret), in addition to custom data classifications.

    • Table Storage Mode: If the data table is a lakehouse table and its format is Hudi, this information can be collected from the table's source compute engine.

    • Storage Size: The actual storage size of the table. This is updated on a T+1 basis.

    • Lifecycle: The retention period of the table in days.

  Note

  When you create a logical table, the system may automatically generate multiple materialized tables. The system determines whether the table is a lakehouse table, the lakehouse table format, and the table storage mode based on the first materialized table that is generated.

  • Field Information: Includes the field name (primary keys are marked with a key icon), description, data type, associated standard (requires the Data Standard module), sample data (displayed only when the data sampling feature is enabled), data classification (requires the data security feature), data sensitivity level (requires the data security feature), and popularity information. You can also search for fields, filter fields, and view field lineage.

    • View Lineage: In the Actions column, click the lineage icon to view the field lineage centered on the specified field.

    • Search and Filter: You can search for fields by name or description. You can also filter fields by data classification or data sensitivity level (requires the data security feature).

  • Partition Info: View the partition information of the data table.

    • Field: Select the field and the date range that you want to view. The data for the last 30 partitions from the selected end date is displayed.

    • Field Partition Details: A dark color indicates that the field has records in the corresponding partition. A light color indicates that the field has no records in the partition.

  • Associated Assets: The dimension information associated with the current table. Click a table name to go to the details page of that table.

③ Lineage & Impact

• Lineage displays the data lineage between tables and fields.

  • Table-level lineage sources include sync tasks, SQL compute tasks, and logical table tasks from which the system automatically parses lineage, custom lineage that is manually configured for compute tasks. For more information, see Table-level lineage.

  • Field lineage sources include sync tasks (field-level lineage can be parsed for only some data sources), SQL compute tasks, and logical table tasks from which the system automatically parses lineage, custom lineage that is manually configured for compute tasks. For more information, see Field lineage.

    Note

    • For lineage that is automatically parsed by the system and lineage that is manually configured in compute tasks, the system parses the table and field lineage in the developer environment when a task is submitted. When the task is published, the system parses the table and field lineage in the production environment. A maximum of 100,000 lineage relationships can be parsed for a single task submission or publication. If this limit is exceeded, the lineage is not recorded and cannot be displayed in the asset checklist.

    • Deleting a logical table also deletes its associated lineage.

• Impact refers to the effects on data tables and integration synchronization.

  • Data table impact: Displays the direct descendant tables that reference the current table and the indirect descendant tables that reference the direct descendant tables. You can export the data or display only direct descendant tables.

    • Export Data: Export the data to an Excel file for business analysts to review.

    • Display Only Direct Descendant Tables: If you select this option, only the descendant tables that are directly affected by the current table are displayed. If you clear this option, all affected descendant tables are displayed, up to a maximum of 15 levels. The direct descendants of the current table are considered the first level.

  • Sync task impact: Displays the sync tasks that use the current table and its descendant tables as source tables. You can export the data or display only the impact of the current table.

    • Export Data: Export the data to an Excel file for business analysts to review.

    • Display Only Current Table Impact: If you select this option, only the sync tasks that use the current table as an input table are displayed. If you clear this option, the sync tasks that use the current table and all its affected descendant tables as input tables are displayed, up to a maximum of 15 levels. The direct descendants of the current table are considered the first level.

④ Quality Overview

If you have activated the Data Quality feature, this section displays an overview of rule verification and a list of quality monitoring rules for the current data table. Click View Report Details or View Rule Details to go to the corresponding page in the Data Quality module for more details.

⑤ Data Exploration

If you have activated the Data Quality feature, you can configure data exploration tasks for the data table to quickly understand the data profile and assess its availability and potential threats in advance. To enable automatic exploration, go to Administration > Metadata Center > Exploration and Analysis and enable the corresponding configuration. For more information, see Create a data exploration task.

⑥ Data Preview

If sample data exists for the data table, the sample data is displayed by default. You can also manually trigger a query to retrieve the latest data. If no sample data exists, a data preview query is automatically triggered.

• Sample Data: This is displayed when data sampling is enabled and the data preview switch in the usage configuration is turned on. Only sample data for fields for which you have column-level permissions and that do not require data masking is queried. The system stores and sorts the sample data for each field independently, but does not guarantee the existence or correctness of row records.

• Data Preview: If you have the permissions to query the current table, you can use the data preview feature. This feature queries only the results for fields for which you have permissions for SELECT statements, including field-level and row-level permissions. You can preview the first 50 data records. For more information about how to request permissions, see Request, renew, and return table permissions.

You can search or filter the data by field, view the details of a single row, and use the automatic column width adjustment and row-column transformation features. You can also click the sort icon next to a field to select No Sort, Ascending, or Descending. Double-click a field value to copy it.

⑦ Output Information

Output tasks include data write tasks for the object, tasks with auto-parsed or custom-configured lineage that use the current table as the output table, and tasks where the node output name is `Project name.Table name`.

The list of output tasks is updated in near real-time. The output details are updated on a T+1 basis.

• ① View Output Details: You can view the output details of only auto triggered tasks. For more information, see Output details.

• ② Go To O&M: Click the Go To O&M button to go to the task list page in the Operation Center. The current task is automatically filtered. You can view more information on this page.

⑧ Usage Instructions

You can add usage instructions for the data table to provide reference information for data viewers and consumers. Click Add Usage Instructions, and then enter a title and content for the instructions.

⑨ Asset Information

Displays detailed information about the physical table, such as Basic Information, Change Information, and Usage Information.

• Basic Information: Includes the environment, table type, creation time, creator, owner, and output tasks.

  • Owner: The owner of the current table. You can transfer ownership of the table to another user. In the Change Owner dialog box, you can select whether to also transfer ownership of the table in the developer and production environments. Select a Recipient and click OK to complete the transfer. We recommend that you notify the recipient after the transfer. You can view transfer information on the transfer record page. For more information, see View transfer records.

    Note

    • Super administrators can change the owner of all table types. The current owner of a table can change the owner of that table.

    • A subject area architect can change the owner of logical tables within the subject areas that they manage.

    • In special cases, if a logical aggregate table does not have an initial owner, a super administrator or subject area administrator can change the owner.

  • Output Task: You can view the output tasks of the current table. These include data write tasks for the object, tasks with parsed or configured lineage that use the current table as the output table, and tasks where the node output name is `Project name.Table name`. Click the name of an output task to go to the O&M details page of the data table.

    Note

    You can view the output details of only auto triggered tasks.

• Change Information: Includes data changes, recent access, and DDL changes.

  • Data Updated At: The time of the last table content change (corresponding to DML operations) that Dataphin collects from the compute engine. This is updated in near real-time.

  • Last Accessed At: The time of the last SELECT operation (corresponding to DQL operations) that was parsed from SQL by Dataphin. Accesses triggered by external systems are not recorded. This is updated in real time.

  • Last DDL Time: The time of the last table schema evolution (corresponding to DDL operations) that was parsed from SQL by Dataphin. Changes triggered by external systems are not recorded. This is updated in real time.

• Usage Information: Includes the number of favorites, page views, and access count.

  • Number Of Favorites: The number of users who have added the current table to their favorites. This is updated in real time.

  • Page Views: The number of page views (PV) for the current data table. The count increases by one each time the page is refreshed. This is updated in real time.

  • Access Count: Based on SQL parsing, each time the table is selected in a Dataphin task (corresponding to DQL operations), the count increases by one. This information is updated on a T+1 basis and shows the total access count over the last 30 days.