How to use the intelligent diagnostics feature-DataWorks(DataWorks)-阿里云帮助中心

Use the intelligent diagnosis feature to analyze tasks end-to-end. When a task does not run as expected, you can use this feature to quickly pinpoint the problem.

Background

The intelligent diagnosis feature analyzes tasks based on the following dimensions:

View running details: A scheduled task runs only when its upstream dependencies are successful, it reaches its scheduled time, sufficient scheduling resources are available, and it is not already running. For more information, see Node run conditions.
View basic information: You can view the key timestamps for the current task instance.
Affected baselines: You can view a list of baselines that monitor the current task and check their operational status. For more information about intelligent baselines, see Intelligent baseline overview.
Historical instance running status: You can view the task's execution history for the last 15 days in charts and lists.

Limitations

The Perform Diagnostics feature is available only in DataWorks Professional Edition and higher. You can currently try this feature for free, but we recommend that you upgrade to the Professional Edition to access more product capabilities. For more information about upgrading your edition, see Features of DataWorks editions.
The intelligent diagnosis feature is supported in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Zhangjiakou), China (Shenzhen), China (Hong Kong), Japan (Tokyo), Singapore, Malaysia (Kuala Lumpur), Indonesia (Jakarta), Germany (Frankfurt), US (Silicon Valley), US (Virginia), and UAE (Dubai).

Accessing intelligent diagnosis

Log on to the DataWorks console. In the target region, click Data Development and O&M > Operation Center in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Operation Center.
After you open Operation Center, you can access the Intelligent Diagnostics page in one of two ways.
- Method 1: Access intelligent diagnosis for an instance.
  - In the left-side navigation pane, choose Auto Triggered Task O&M > Scheduled Instance. On the Instance Perspective tab, click Run Diagnostics in the Actions column of the target instance to go to the intelligent diagnosis page.
  - In the left-side navigation pane, choose Auto Triggered Task O&M > Scheduled Instance. On the Instance Perspective tab, click DAG in the Actions column of the target instance. In DAG view, right-click the DAG node of the target instance and select Run Diagnostics from the context menu to go to the intelligent diagnosis page.
- Method 2: In the left-side navigation pane, choose O&M Assistant > Intelligent Diagnostics to go to the intelligent diagnosis page.
  
  Note
  Intelligent diagnosis supports locating specific instances only by instance ID.

View running details

Based on the prerequisites for a task to run, DataWorks sequentially checks the running status of upstream tasks, the scheduled time of the current task, the usage of scheduling resources, and the execution status of the current task.

Upstream Nodes

On the Upstream Nodes page, intelligent diagnosis checks the running status of upstream tasks. If an upstream task has not run successfully, the current task is blocked. You can click Run Diagnostics in the Actions column of the upstream task to identify the cause of failure.

Note
If an upstream task is in the Not Run state and the upstream dependency hierarchy is deep, we recommend that you first use the Upstream analysis feature in the DAG panel to quickly locate the key upstream task that blocks the current task, and then use Intelligent Diagnostics to diagnose why the key task has not run. This improves O&M efficiency.
Timing Check

Timing Check checks whether the current task has reached its scheduled time. This check is triggered only after the upstream dependency check passes.

Note
When you define the scheduling properties of a task in the DataStudio module, you must set the expected run time of the task in the scheduling environment. However, the actual run time may be later than the scheduled time due to issues such as upstream task failures.

Resources

The Resources page of intelligent diagnosis displays the resource usage and a list of tasks that occupy the resources while the current task is waiting. If the check fails, the resources are insufficient and the task waits until other tasks release resources. You can arrange the scheduled time of tasks based on resource usage trends to stagger peak hours.

Note

If the scheduled time of a task is around 00:00 and the wait time is only a few minutes, this is usually normal during the peak scheduling period. A large number of tasks are triggered around 00:00 every day, and this delay occurs during the task triggering phase. We recommend that you stagger the scheduled time to a time point after 00:00.

Feature	Description
Scheduling resource information	Displays the name of the scheduling resource group used by the current task, the number of tasks currently running on the resource group, and the number of tasks waiting to run on the resource group. Note We recommend that you use serverless resource groups to ease resource shortages. If you use the shared scheduling resource group, the period from 00:00 to 09:00 every day is the peak hour for DataWorks tasks. Shared scheduling resources may be insufficient, and tasks may need to wait for available resources.
Diagnosis Results	Displays the execution status of the current task.
Resource Usage Trends	If you use the shared scheduling resource group, this section displays the resource utilization of the current scheduling resource group across time periods, and the time that the current task spent waiting for resources.

Execution

Execution displays the execution log, the execution details of associated data quality rules, and code details of the current task. For a task that failed, intelligent diagnosis provides diagnostic suggestions based on the log information to help you quickly identify the cause of the error.

Feature	Description
Log	Displays the detailed execution process of the task. In the Execution section of Operation Details, you can click the EMR Web UI URL printed in the Log to go to the corresponding EMR component web page, and click the Intelligent Diagnostics button in the lower-right corner to go to the intelligent diagnosis page and analyze the error information.
Intelligent Diagnostics	Uses a large language model (LLM) to analyze task error logs. The following models are supported: Qwen, DeepSeek, and DW Knowledge Base. You can use Qwen or DeepSeek to analyze error logs, parse error logs and generate error analyses and resolution suggestions, and view recommended solutions in the DW Knowledge Base. Note After the LLM completes the error log analysis, you can quickly perform actions such as Modify Code, Rerun Instance, Set Instance to Successful, Modify Instance Scheduling Resource Group, Modify Instance Data Integration Resource Group, Submit Ticket, and Apply for Table Permissions.
DQC	If data quality rules are associated with the task, the rules are triggered when the task runs. You can view the detailed execution results of the data quality rules here.
Code details	Displays the code details of the current task.

View basic information

On the General page, you can view the key timestamps and basic information for the current task run. For more information about the properties, see Configure scheduling properties.

View affected baselines

On the Influenced Baseline page, you can view the list of baselines that monitor the current task and their running status. For more information about intelligent baselines, see Overview.

View historical instances

On the Historical instance page, you can view the following information:

Trends of metrics across dimensions for the current task: Visual charts display the trends of Running time, Start run time, Waiting Time for Resources, and Completed At for the current task over the last 15 days.
Historical instance running status of the current task: A list displays the running details of historical instances for the current task, including the start time, end time, running duration, and resource waiting time. You can click Run Diagnostics in the Actions column to go to the diagnosis details page of the corresponding instance.