basic operations for auto triggered nodes,operations,deploy an auto triggered node,undeploy an auto triggered node-DataWorks(DataWorks)-阿里云帮助中心

Usage notes

Only the Operation Center in the production environment supports automatic scheduling to generate auto triggered node instances; the development environment does not.
Use the commit and deploy process to add, modify, or undeploy node code, scheduling configuration, resources, and functions in the production environment.
All changes to a node must be deployed to take effect in the production environment. A deployment may fail, be blocked, or result in an unexpected version. Therefore, check the node on the Operation Center > Auto Triggered Node page after deployment. For more information, see Manage auto triggered nodes.

O&M operations

This topic describes how to perform O&M operations on auto triggered nodes.

O&M operation	Description
View auto triggered nodes	On the Cycle Task page, you can view detailed information about nodes and perform operations from the list view or the DAG.
Run auto triggered nodes	Running modes: Nodes deployed to the production environment run automatically according to their schedule. You can view the results of automatically scheduled runs on the Operation Center > Scheduled Instance page. You can also run tests or backfill data for a historical or future period in the production environment and view the results in data backfill instances and test instances. Node run conditions and troubleshooting: Understand the conditions for a node to run and how to troubleshoot failures. Emergency operations: If a node fails to run, you can perform emergency operations, such as removing its dependencies on ancestor nodes that have not yet run or forcing a rerun of descendant nodes if data contamination occurs.
Manage auto triggered nodes	Deploy and undeploy nodes Deploy an auto triggered node: A node must be deployed to the production scheduling system to appear in the Cycle Task list. Undeploy an auto triggered node: If a node or an entire business process is no longer needed, you can undeploy the node or batch-undeploy all nodes in the process. Note Undeployed nodes are moved to the recycle bin, from where you can restore them. Node management Pause a scheduled run: If a node and its descendant nodes do not need to run automatically for a period, you can freeze the node. Node monitoring and alerting: In the auto triggered node list, you can configure monitoring rules for the run status of nodes, the quality of output table data, and the resource groups used for execution. Manage node priority: You can use the baseline feature to manage node priority. High-priority nodes are allocated scheduling resources first. Modify the node's resource group: You can modify a node's run properties, including its scheduling resource group and Data Integration resource group. Modify the node owner: You can perform this operation in batches.
View O&M records for auto triggered nodes	You can view the operation logs, node versions, and operation history for nodes and instances.

Viewing auto-triggered nodes

The auto triggered node list displays all nodes scheduled to run automatically in the production environment. You can verify that the node code, scheduling parameters, dependencies, and data lineage are correct. You can also view detailed information and perform operations from the list view or the DAG. For more information, see Manage auto triggered nodes.

Note

Only nodes deployed to the production environment appear in the auto triggered node list in Operation Center.
A node is also not scheduled if it depends on one of its own descendant nodes, as this creates a dependency loop.

Running auto-triggered nodes

This section describes DataWorks node run modes and how to diagnose issues based on execution status.

Running modes

Every night, DataWorks automatically generates the next day's auto triggered node instances from the current auto triggered nodes. You can also manually trigger an auto triggered node to generate data backfill instances and test instances.

Instance type	Use case	Generation method	Trigger method
auto triggered node instance	Run periodic, automated ETL operations.	DataWorks automatically generates the next day's instances based on a nightly snapshot of the auto triggered node. Note The Operation Center in the development environment does not automatically schedule or generate auto triggered node instances.	DataWorks automatically runs the instance.
data backfill instance	Run ETL operations to backfill data for a past or future period for the current node and its descendant nodes.	A data backfill instance is generated when you manually trigger a data backfill for an auto triggered node.	The data backfill instance runs immediately after it is generated.
Run a test and view the test instance	Test an auto triggered node to verify that its execution result is correct. Note A test instance runs the actual code logic.	A test instance is generated when you manually trigger a test run for an auto triggered node.	The test instance runs immediately after it is generated.

Node run conditions

A scheduled node runs only when all of its ancestor nodes have completed successfully, its scheduled time has arrived, sufficient scheduling resources are available, and the node is not frozen. For more information, see Prerequisites for a scheduled node to run. 运行流程图

Troubleshooting node run issues

Symptom: The node does not run even after its scheduled time has arrived. Possible causes include its ancestor nodes not running successfully, insufficient scheduling resources, or the node being frozen.

Troubleshooting: First, use the upstream analysis feature in the DAG panel to identify the blocking ancestor nodes. Then, use the intelligent diagnosis feature to diagnose why the key instance has not run or to identify other issues. For nodes with complex dependencies, this feature helps you locate the problem and improves O&M efficiency. For more information, see Why has my node not run after its scheduled time?, Waiting for resources, and Freeze and unfreeze a node.

Emergency operations

If an ancestor node has not run and is blocking the current node, you can right-click the instance that has not run and select Emergency Operations > Delete Dependencies to run the current node immediately.

Note
Before you perform this operation, check the node's code and data lineage to confirm that it will not adversely affect your data.
If a widespread data quality issue occurs, you can right-click an instance and select Emergency Operations > Force Rerun > Force Rerun Descendant Nodes. For more information, see Appendix: Force rerun descendant nodes.
In some extreme cases, such as a server power outage or a primary/secondary switchover, DataWorks may not be able to completely terminate related MaxCompute tasks. In this situation, go to the corresponding MaxCompute project to terminate the job.

Managing auto-triggered nodes

Deploying auto-triggered nodes

A node must be deployed to the production scheduling system before it appears in the auto triggered node list in Operation Center. For more information, see Deploy nodes (old-version DataStudio) and Deploy nodes or workflows (new-version DataStudio).

Undeploying auto-triggered nodes

If a node or a business process is no longer needed, you can undeploy it. After a node is undeployed, it no longer appears on the Cycle Task page. For more information, see Undeploy nodes (old-version DataStudio) and Undeploy nodes (new-version DataStudio).

Pausing scheduled runs

Important

Do not modify the projectname_root node. This node is the root node of the workspace. Auto triggered node instances depend on this node. If you freeze this node, auto triggered node instances cannot run.

Operation	Use case	Description
Freeze an auto triggered node	If a node and its descendant nodes do not need to be automatically scheduled for a period, you can freeze the node.	How it works: In an automatic scheduling scenario, a node runs only after all its ancestor nodes have run successfully. If an ancestor node is frozen, it blocks all its descendant nodes. Therefore, if a node and all its descendants do not need to run, you can freeze the root node of the corresponding business process. Effect: When an auto triggered node is frozen, its generated instances are also frozen, which in turn blocks all dependent descendant instances.
Freeze an instance	If a specific instance does not need to run, you can freeze it.	The freeze operation affects only the selected instance. It does not affect other instances from the same day or any subsequent instances generated by the auto triggered node.
Set an auto triggered node to dry run	If a node does not need to run for a period but should not block its descendant nodes, you can set its scheduling type to Empty Run Scheduling.	An auto triggered node set to dry run generates dry run instances. These instances have empty run logs, no execution duration, and do not process any data. Note Setting the scheduling type to Paused in DataStudio has the same effect as freezing an auto triggered node in Operation Center. However, changes made in DataStudio take effect only in the development environment. To apply them to the production environment, you must deploy the node. For more information, see Deploy nodes (old-version DataStudio) and Deploy nodes or workflows (new-version DataStudio).

Note

For more information about the effects of freezing and unfreezing auto triggered nodes and their instances, see Freeze and unfreeze a node.

Managing node priority

You can use the baseline management feature to manage the priority of a node. The scheduler allocates resources to high-priority nodes first. For more information, see Manage baselines.

Node monitoring and alerting

You can configure monitoring rules for a node from the auto triggered node list:

Monitor the status of auto triggered node instances during scheduled runs. For more information, see Create and manage custom alert rules.
Monitor the quality of table data generated by auto triggered node instances, data backfill instances, and test instances. For more information, see Data Quality overview.
Monitor the resource group used by the node by setting alerts for metrics such as instance count and resource utilization. For more information, see Create and manage custom alert rules.

Modifying a node's resource group

You can modify the resource groups used by a node, including the scheduling resource group and the resource group for Data Integration. For more information, see Reference: Switch resource groups.

Note

To redefine other properties of an auto triggered node, go to DataStudio, find the node, and open its configuration panel. For more information, see Configure basic properties. To modify multiple nodes at once, go to the Batch Operations page.

Modifying the node owner

You must first enable Allow Sub-accounts to Change Node Owner. Once enabled, the workspace administrator can change owners in the following ways:

In DataStudio, you can change the owner for a single node or for multiple nodes in a batch:
- Single node: Open the Node Scheduling Configuration > General panel.
- Multiple nodes: Use the Batch Operations page. For more information, see Batch Operations.
Changes to the owner in the development environment must be deployed to the production environment to take effect.
In the production environment:
- Single node: In the Actions column for the node, click More > Change Owner.
- Multiple nodes: Select the nodes whose owner you want to change, and then click Change Owner at the bottom of the page.
  
  Note
  Changes to the owner in the production environment are synchronized to the development environment.

Viewing O&M records

Entry point	Description
Operation Log for a node or instance	Click a node or instance and go to the Operation Log tab to view its change history.
Node Version	If the deployment process lacks detailed records, go to DataStudio and compare the node version with the one in the production environment to see detailed changes. For more information, see Deploy nodes (old-version DataStudio) and Deploy nodes or workflows (new-version DataStudio).
Operation History	You can view the operation records for nodes, instances, and baselines on the Operation History page. For more information, see View O&M records in Operation Center.

FAQ

For frequently asked questions about O&M for auto triggered nodes, see FAQ about auto triggered nodes.