Configure same-cycle dependencies-DataWorks(DataWorks)-阿里云帮助中心

A same-cycle dependency means that the current node in its current cycle depends on whether the upstream node has successfully produced data in the same cycle—that is, whether the task executed successfully. The current node runs only after the upstream node’s instance for the current cycle completes successfully. Typically, if your task needs to use table data produced by an upstream task in the same cycle, configure the current task to depend on that upstream task’s same-cycle instance. DataWorks offers multiple ways to configure same-cycle dependencies and includes a dependency preview feature so you can review and correct any incorrect dependencies right away, ensuring your task scheduling behaves as expected. This topic covers key considerations, configuration principles, and methods for setting up same-cycle dependencies.

Important notes

To ensure smooth scheduling dependencies, first understand node scheduling configuration.
In DataWorks DAG graphs, same-cycle dependencies appear as solid lines.
In some complex scenarios, if same-cycle dependencies do not meet your needs, configure cross-cycle dependencies instead. For example, by default, a daily task depends on all hourly instances from the same day. You can use self-dependency in the hourly task so the daily task depends only on a specific hourly cycle. For more information about complex dependency scenarios, see Scheduling configuration principles and examples for complex dependency scenarios.

Configuration principles

To improve development efficiency, use the automatic dependency parsing feature to quickly set up node dependencies. When using automatic parsing during development, follow these principles:

Node creation: Use the same name for the node and its output table.
Code development: Avoid writing data from multiple nodes into the same table.
Dependency configuration: Configure the node’s output table as the node’s output.

Access point and configuration details

Open the editing page for a Data Development node. In the navigation pane on the right, click Scheduling Settings. In the Scheduling Dependency section, configure the node’s dependencies.

When configuring same-cycle dependencies, define which upstream tasks the current task depends on in the Parent Nodes section. In the Output Name of Current Node section, define how other tasks can reference this node’s outputs to establish dependencies.

Note

By default, DataWorks uses table lineage from your code—based on tables queried and written—to quickly set up dependencies. When you submit the node, the system automatically checks whether the current dependencies align with the data lineage. You can choose whether to enable automatic code parsing before submitting a task, based on your business needs.
The platform supports multiple dependency configuration methods: automatic lineage parsing, drawing connectors in the workflow panel, and manual addition. Choose the method that fits your needs.
If your node needs data produced by the upstream node yesterday, or if an hourly or minute-level task depends on its previous hourly or minute-level instance, configure a cross-cycle dependency.

The dependency configuration panel has two core sections: Upstream Nodes (inputs for this node) and Node Outputs.

Upstream Nodes

This section defines the upstream nodes for the current node. After configuration, the current node runs only after its upstream nodes complete successfully. Enter the output names of the upstream nodes here as inputs for the current node. Keep the following in mind when configuring upstream nodes:

All nodes must have upstream dependencies configured. Base your configuration on table lineage whenever possible. If no table lineage exists, choose to depend on the workspace root node or a zero load node based on business requirements.
Ensure that upstream nodes are already submitted. If you receive an error saying the upstream node output does not exist during submission, confirm that the required upstream nodes have been submitted.

Method 1: Automatic dependency parsing

DataWorks parses output table names in the format projectName.tableName from your code and recommends upstream nodes based on those tables.

Click Parse Inputs and Outputs from Code to auto-generate the parsed results in the dependency list below.

Method 2: Manual addition

Click the Add Dependency button. After selecting a dependency method, search for the upstream dependency by Name, Output Name, or scheduling task ID.

Note

When using automatic dependency parsing, ensure the upstream task has already been submitted and published to the production environment and has actually produced the table data. Recommended nodes must be submitted to the scheduling system one day in advance. Only after data is produced the next day can the automatic recommendation feature detect them. Therefore, automatically recommended nodes have a T+1 delay.

Node Outputs

Node outputs act as the connection point for other nodes to depend on this node. Other nodes find this node by its output name and configure it as their upstream node through scheduling dependency settings. After a downstream node sets a dependency on this node and completes submission, the downstream node’s name appears under this node’s outputs. DataWorks does not allow manual editing of downstream nodes in the node output interface. Configure outputs as follows:

Note

If your workspace contains nodes with identical names, their outputs may also share the same name, causing submission failures. Deleting a node output that already has downstream dependencies can cause serious issues. For details, see: Impact of deleting or changing node outputs.

Method 1: Default generation

DataWorks automatically generates one node output. Click Modify to change the output table name.

Note

Output names are globally unique and cannot be modified or deleted. When you draw a connector in the workflow panel to set a dependency, DataWorks automatically generates an output table name and output name as inputs for the downstream node.

Method 2: Manual addition

In the node output configuration section, click Add Output to manually create a new output and specify its output name and output table name.

Note

Output names follow the format workspace_name.custom_configuration and must be globally unique.

Configuration principle

Scheduling dependency configuration treats the output of an upstream node as the input of a downstream node, creating a dependency chain. Set scheduling dependencies based on the table lineage of tables your node queries and writes to. After configuration, the downstream node starts only after the upstream node finishes running successfully, ensuring correct data availability at runtime.

You can configure dependencies in three ways. The underlying principle remains the same regardless of method.

Configuration method	Description
Set node dependencies by drawing connectors in the workflow panel	DataWorks automatically adds the upstream node’s default output as an input to the downstream node, establishing the dependency.
Set node dependencies using the automatic parsing feature	DataWorks automatically parses table lineage from node code and configures scheduling dependencies based on that lineage.
Manually add upstream node dependencies in the Scheduling Configuration interface	Use this method when automatically parsed dependencies do not match your actual requirements, allowing you to adjust the scheduling dependency manually.

Configuration methods

Set node dependencies by drawing connectors in the workflow panel

When you draw a connector between nodes in the workflow panel, DataWorks automatically adds the upstream node’s default output as an input to the downstream node, creating the dependency.

Note

The platform automatically passes the upstream node’s output parameters to the downstream node.
If you delete a dependency connector in the workflow panel, the dependency is also removed from the node’s scheduling configuration.

Manually add upstream node dependencies in the Scheduling Configuration interface

In the Scheduling Configuration interface, filter and select upstream dependencies by name, output name, or ID based on dependency type (same-cycle, cross-cycle, cross-cycle self-dependency, or cross-cycle child node dependency).

Enter a keyword in the Dependent Object field to search, select the target node from the results, and click Add to complete the configuration.

Set node dependencies using the automatic parsing feature

DataWorks supports rapid dependency setup based on table lineage in your node code. During scheduling configuration, click Parsing input and output from code to automatically parse your latest code and recommend upstream nodes to depend on.

Next steps: Verify that dependencies meet expectations

After you complete the configuration, you can perform the following operations to verify that the job scheduling behaves as expected:

Submission check: Confirm during node submission that dependency changes match your expectations.
Recurring task dependency verification: After publishing the node, go to Operation Center to confirm that production scheduling dependencies match your expectations. Recurring tasks reflect the latest state of the task in the production environment. Instance dependencies for recurring instances depend on how the instances were generated.

Appendix

Appendix A: Impact of deleting or changing node outputs

If changes to a node’s output table—or direct manual edits to the node output—occur, note the following:

Deleting a node output does not directly affect the table data the node produces.
If the node output already has downstream dependencies, changing or deleting it may severely impact downstream tasks.
- Output table deletion: If an automatically parsed node output changes because the node’s output table changed, downstream nodes might become Scenario 2: Orphaned nodes, which are not scheduled, or suffer data pollution due to missing data dependencies.
- Output table change: If the node’s output table must move to another node.
If a node output has downstream dependencies and you plan to delete its output name, coordinate with the owners of downstream tasks beforehand. Notify them that you will delete a specific output so they can update their dependency configurations promptly and avoid orphaned tasks.