When automatic parsing is enabled, you must confirm any changes to a node's scheduling dependencies during the publishing process. This ensures production data integrity and gives you control over node changes. When you submit a node, the system parses its code to identify inputs and outputs based on table lineage. You can use these code parsing results to verify that the node's upstream and downstream dependencies are correct.
Background information
When you submit a node, the system compares its current scheduling dependencies with those in the development or production environment. A node's current dependencies combine the automatic parsing results and any manual changes made in the Schedule > Dependencies section. If a discrepancy is found, a prompt indicates which inputs or outputs have been added or removed compared to the previous version. You can choose whether to use the new parsing results and submit the node with the updated dependencies. If you proceed, the system automatically adds the latest results to the Schedule > Dependencies section.
If automatic parsing is disabled, you can define node scheduling dependencies manually based on your business requirements. You can then compare node versions to track changes to these dependencies.
Limitations
The comparison feature does not currently support cross-cycle dependency analysis. If your node has a cross-cycle dependency, verify that the dependency is configured as expected before you submit the node.
Confirm new parsing results
When you submit a node, if the system detects a difference between the parsed scheduling dependencies and the configuration in the production environment, you must verify that the new dependencies meet your business needs. This helps prevent data issues arising from incorrect dependency changes.
-
If the current node has many downstream nodes, the impact of a change can be significant. Ensure you understand the business context and proceed with caution.
-
The following examples focus on the impact of dependency changes on tasks in a production schedule.
Example: Impact of upstream dependency changes
When you submit the current node, the system compares its scheduling configuration with that of the same node in the production environment. Suppose it finds that input name A (an output from an upstream node) is missing. You must then verify the node's scheduling dependencies. For example, if your node's code depends on data from table A but the node that produces table A is no longer an upstream dependency, your node might run before table A is ready. This can lead to incorrect data output. In the Compare Code Parsing Results dialog box, a yellow banner warns, "Automatic parsing is enabled for this node. Please confirm the code parsing results." The dialog box has two tabs: Upstream outputs (6 differences) and Current outputs (3 differences). The Upstream outputs tab shows a comparison across three columns: Last Saved, Development, and Production. ods_log_info_d1 is added, and ods_log_info_d is removed. The changes are consistent across all three environments, with the production environment comparison highlighted by an orange border.
Example: Impact of node output changes
When you commit the current node, its scheduling configuration is compared to the configuration of the same node in the production environment. If an output name is missing, you must confirm whether the table that the node produces has changed. If this output has downstream dependencies, verify whether this change will have an unpredictable impact on the downstream nodes. In the Code Parsing Result Comparison dialog box, switch to the Output Names of This Node tab to view the differences between the development and production environments. For example, there are three differences from the development environment: workshop0423.dw_user_info_all_d1 is added, while workshop0423.dw_user_info_all_d and workshop0423.dw层 are deleted. There are two differences from the production environment: workshop0423.dw_user_info_all_d1 is added, and workshop0423.dw_user_info_all_d is deleted. The Outputs of This Node panel on the right shows that the deleted output name workshop0423.dw_user_info_all_d is a dependency for the downstream node rpt_user_inf_o_d (Node ID: 700003444850) based on automatic parsing. This indicates that the output change will affect downstream dependencies. After you confirm, click Use New Parsing Result to apply the change. If a node has downstream dependencies, deleting its output may cause the downstream node to become an orphan node, which can result in scheduling failures or data retrieval issues. For more information, see the documentation about the effects of deleting or changing node outputs. When you change a node's output name, an Important dialog box appears. This dialog box lists the nodes in the development and production environments that depend on the original output name and warns that this change may cause downstream nodes to become orphan nodes that cannot be scheduled properly. Click Confirm to modify the output name and continue to commit the node.
When you choose to apply the new parsing results, you are deleting an output with downstream dependencies. You must assess the potential impact and proceed with caution.