Manage integration and compute tasks-Dataphin(Dataphin)-阿里云帮助中心

Integration and Compute Tasks page

On the Dataphin homepage, choose Develop > Task O&M from the top navigation bar.
In the left-side navigation pane, choose Task O&M > Periodic Tasks.
In the top navigation bar, select the production or development environment.
On the Periodic Tasks page, click the Integration and Compute Tasks tab.

Operations for integration and compute tasks

When you submit an integration or compute task to the O&M center for scheduling, it appears in the Periodic Tasks > Integration and Compute Tasks list. The list displays the task object, scheduling cycle, directory, priority, owner, related baselines, project, HTTP path, scheduling resource group, last updated time, tags, and supported operations for each task.

Task object: A script-based cycle task submitted to the O&M center. This field displays the script name, script ID, and the task's scheduling method. For more information, see Scheduling methods.
Scheduling cycle: The time the task is scheduled to run in the specified time zone.
Directory: The task's directory. You can see only the tasks in directories for which you have permissions.
Priority: The priority of the task. If the baseline feature is enabled, the task's priority is set to the highest priority of its associated baselines, which overrides any previously configured priority.
Related baselines: Lists the baselines that guarantee the task as an end node and those that include it as an upstream node in their guarantee scope.

Note
This field is not displayed if the baseline feature is not enabled.
Project: The project to which the task belongs, displayed in the format: Project_English_Name (Project_Chinese_Name).
HTTP path: For Databricks SQL tasks, this field displays the HTTP path for the selected production or development environment.

Note
This field is displayed only for Databricks SQL tasks. For other task types, it shows -.
Scheduling resource group: The name of the scheduling resource group that the task instance uses at runtime.

If the custom resource group specified for the task is unavailable, the project's default resource group is used. If the project's default resource group is also unavailable, the tenant's default resource group is used. The fallback order is: custom resource group > project default resource group > tenant default resource group.

Note
When you change a project's default resource group, the update may not be immediately reflected here, but the task runs using the new resource group.

Tenant default resource group: A default resource group that does not belong to any project. Each tenant has one. This resource group is used for a task if no custom resource group is specified and its project also lacks a default resource group, and this applies only to tasks that use dedicated resources, excluding tasks such as SQL and Virtual tasks.

The following operations are available for integration and compute tasks.

Actions	Description
DAG	Click to view the DAG for the task. For more information, see Operations for nodes in a task DAG.
View Periodic Instances	View the periodic instances generated by the task. You can also perform O&M and manage these instances.
Edit Development Node	Opens the task's edit page in the Dev project. Note This operation is available only in Dev-Prod mode.
Edit Node	Opens the task's edit page. Note This operation is available only in Basic mode.
View Production Node	Opens the Prod project to view the task configuration. Note This operation is not supported for tasks that have not been published to the production environment in either Basic mode or Dev-Prod mode.
View Node Code	View the code of the integration or compute task node.
View Backfill Instances	View and manage the instances generated by a data backfill operation.
Backfill Data	The data backfill feature reruns cycle tasks over a specified range of historical business dates. After a cycle task is developed and published, it runs at scheduled intervals. Use the data backfill feature to run a cycle task for a specific time period or reprocess data from a past period. For more information about performing a data backfill on an integration or compute task node, see Appendix: Data backfill for cycle tasks.
Download Upstream/Downstream Nodes	Downloads a list of the current node's upstream and downstream nodes. The list includes all columns, including those not displayed in the UI. Click Download Upstream and Downstream Nodes. In the Download Upstream and Downstream Nodes dialog box, select the levels for the upstream and downstream nodes. You can select from Level 1 to Level 10 or Unlimited levels. The default for both is Level 1. Click OK to download an Excel file named `{task_name}_Upstream_and_Downstream_Nodes_{timestamp}.xlsx`.
Modify Owner	Changes the task's owner. Note This operation is available in Basic mode and the production environment of Dev-Prod mode. It is not supported in the development environment.
Modify Priority	Changes the priority of the task. Among tasks ready to run concurrently, higher-priority tasks execute first. Note This operation is available in Basic mode and the production environment of Dev-Prod mode. It is not supported in the development environment. If the baseline feature is enabled, you can only set the task priority to Lowest, Low, or Medium. Higher priorities must be configured through baselines. You cannot directly modify the priority of a baseline task. The priority is determined by the baseline itself and must be adjusted in the baseline configuration. If your compute engine is MaxCompute, Dataphin task priorities map to MaxCompute job priorities as follows: Lowest (9), Low (7), Medium (5), High (3), and Highest (1). For more information about MaxCompute job priorities, see MaxCompute Job Priority. The priority of a Spark SQL task takes effect only if different priority queues are configured in HDFS for the Hadoop compute source.
Pause	Pauses the scheduling of the current task node. This is useful for temporarily stopping a task and its downstream dependencies, for example, when adjusting logic without affecting downstream data.
Resume	Resumes scheduling for a paused node, returning it to a normal state.
Configure Monitoring and Alerting	Configures monitoring rules for task execution. For more information, see Offline task monitoring overview. Note This operation applies only to Prod and Basic projects.
Modify HTTP path	Modifies the production environment HTTP path for a task. You can choose from any HTTP path configured for the cluster corresponding to the production project. Note This operation is supported only in the production environment.
Modify Scheduling Resource Group	Modifies the scheduling resource group used by instances generated from the task. Note If you select tasks from multiple projects, the target scheduling resource group list shows only resource groups that are shared and authorized across all selected projects. We recommend filtering by a single project before performing this bulk operation. This change affects only new instances, not those already generated. To change the resource group for existing instances, you must modify them from the instance list.

Operations for nodes in a task DAG

The DAG provides a clear visualization of the upstream and downstream dependencies between task nodes. You can also perform O&M and manage these nodes. Select a node to perform O&M operations. By default, the DAG displays the main node (the selected node) and its first-level upstream and downstream nodes.

Dataphin supports cross-project node management. To perform O&M operations on a script node in another project, you must have view and operation permissions for that project.

Operations in the DAG view

Expand Parent Nodes and Expand Child Nodes: Expand different levels of dependent nodes for the main node in the DAG.
Operations for a DAG node

Hover over a node in the DAG to view its name, type, scheduling cycle, owner, and description. The operations available for a node in the DAG are the same as those in the task list. For more information, see Operations for integration and compute tasks.

Batch operations for integration and compute tasks

The following batch operations are supported for cycle integration and compute tasks:

Actions	Description
Pause	Click Modify Scheduling Type and select Pause Scheduling to pause all selected tasks. After a task is paused, it still generates instances, but these instances and their downstream dependencies will not run.
Resume	Click Modify Scheduling Type and select Normal Scheduling to resume scheduling for the selected tasks.
Modify Information by Uploading File	Click Modify Information and select Modify Information by Uploading File to update the owner , priority , and scheduling resource group for multiple tasks by uploading an Excel file. The first column of the file must contain the task ID. You can click Download Template to download and populate the template file. Note Only the first 5,000 records in the file are processed. You can specify the owner by their account or ID. If both are provided, the owner ID takes precedence. Modifying task information in bulk is an asynchronous operation. You can view the results in Batch Operation Records.
Modify Owner	Click Modify Information and select Modify Owner to change the owner for the selected tasks in bulk. Note This operation is available in Basic mode and the production environment of Dev-Prod mode. It is not supported in the development environment.
Modify Priority	Click Modify Information and select Modify Priority to change the priority for the selected tasks in bulk. Note This operation is available in Basic mode and the production environment of Dev-Prod mode. It is not supported in the development environment. If the baseline feature is enabled, you can only set the task priority to Lowest, Low, or Medium. Higher priorities must be configured through baselines. You cannot directly modify the priority of a baseline task. The priority is determined by the baseline itself and must be adjusted in the baseline configuration. If your compute engine is MaxCompute, Dataphin task priorities map to MaxCompute job priorities as follows: Lowest (9), Low (7), Medium (5), High (3), and Highest (1). For more information about MaxCompute job priorities, see MaxCompute Job Priority. The priority of a Spark SQL task takes effect only if different priority queues are configured in HDFS for the Hadoop compute source.
Modify HTTP path	Click Modify Information and select Modify HTTP path to modify the production environment HTTP path for multiple Databricks SQL tasks. If the selected tasks belong to different Databricks clusters, you must specify an HTTP path for each cluster from the list of all available paths configured for that cluster. Note This operation is available only when you select Databricks SQL tasks in the production environment.
Modify Scheduling Resource Group	Click Modify Information and select Modify Scheduling Resource Group to change the scheduling resource group used by instances generated from the selected tasks in bulk. Note If you select tasks from multiple projects, the target scheduling resource group list shows only resource groups that are shared and authorized across all selected projects. We recommend filtering by a single project before performing this bulk operation. This change affects only new instances, not those already generated. To change the resource group for existing instances, you must modify them from the instance list.
Backfill Data	Click Backfill Data at the bottom of the page. By default, this action uses the Massive Mode - Specify Node ID method to backfill data for the current task and its dependencies. You can customize the basic information and other configurations for the data backfill, but the scope is fixed. For more information about the parameters, see Backfill data for current and upstream/downstream tasks.