Nodes that are waiting for resources-DataWorks(DataWorks)-阿里云帮助中心

This topic describes common issues and solutions for instances that are waiting for resources.

Why are instances waiting for resources?
Why does a task keep waiting for gateway scheduling resources?
Why does a Data Integration task keep showing a 'wait' status?

Why are instances waiting for resources?

Symptom
When the scheduler dispatches a task to the execution engine, the instance might be queued for the following reasons:
- Waiting for a resource group for scheduling.
- Waiting for a resource group for Data Integration.
- Waiting for compute resources from the engine.
Cause
The available resources in a resource group are limited. If a long-running task occupies resources and does not release them, it blocks other tasks. For more information about the task dispatching mechanism, see Overview of DataWorks resource groups.
Solution
Go to Operation Center > Intelligent Diagnostics, enter an instance ID to perform an end-to-end analysis of the task. This lets you check the task's execution and queue status and identify which other tasks are consuming resources. For more details, see Intelligent Diagnosis.

For an instance node that is Waiting for Resources, right-click the node and select Run Diagnostics > ResourcesRunning StatusWaiting for Resources to view other running tasks. Alternatively, in the node details panel at the bottom right, click the Diagnose link next to the task status to open Instance Diagnose. When filtering tasks, you can set the execution status to waiting for resource in the top filter bar to quickly locate relevant instances.

Why tasks wait for gateway scheduling resources

Symptom
The task log indicates that the task is waiting for gateway scheduling resources.
Cause
The number of concurrent tasks running in the current resource group has reached its limit.
Solution
Wait for running tasks to complete and release resources, or scale up your exclusive resource group for scheduling.
- Shared scheduling resource group
  - Check with Intelligent Diagnosis
    Go to Operation Center > Intelligent Diagnostics, enter an instance ID to perform an end-to-end analysis of the task. This lets you check the task's execution and queue status and identify which other tasks are consuming resources. For more details, see Intelligent Diagnosis.
    
    For an instance node that is Waiting for Resources, right-click it and select Run Diagnostics > ResourcesRunning StatusWaiting for Resources to view other running tasks.
  - Check on the Overview page
    On the Dashboard of Operation Center, you can view the usage of the shared resource group for scheduling.
    Note Peak hours for DataWorks tasks are typically from 00:00 to 09:00 every day. During this period, the shared resource group for scheduling may experience high contention, causing tasks to be queued. You can run your scheduled tasks during off-peak hours or add an exclusive resource group for scheduling in the DataWorks console to mitigate resource shortages.
- If you use an exclusive resource group for scheduling, you can view the list of currently executing tasks and resource usage in the DataWorks console or Operation Center.
  - Check with Intelligent Diagnosis
    Go to Operation Center > Intelligent Diagnostics, enter an instance ID to perform an end-to-end analysis of the task. This lets you check the task's execution and queue status and identify which other tasks are consuming resources. For more details, see Intelligent Diagnosis.
    
    For an instance node that is Waiting for Resources, right-click it and select Run Diagnostics > ResourcesRunning StatusWaiting for Resources to view other running tasks.
  - Check in the DataWorks console
    In the navigation pane on the left of the DataWorks console, click Resource GroupsResource Groups. On this page, check the Utilization rateUtilization rate column. Click the usage percentage to view Resource Groups and Resource Usage.
  - Check in Resource
    In the navigation pane on the left of Operation Center, click Resource O&M to view resource usage and running tasks.
    
    Note The maximum number of instances that can run concurrently varies based on the specifications of the exclusive resource group for scheduling. For more information, see Subscription billing for exclusive resource groups for scheduling.

Why Data Integration tasks show 'wait' status

Symptom
The log of a Data Integration task shows a 'wait' status.
Cause
The number of tasks running on the current resource group for Data Integration has exceeded the maximum concurrency supported by the machine. The task is waiting for Data Integration execution resources.
Note If resource usage is high but no tasks are actually running, or if new tasks cannot run even though the number of running tasks has not reached the resource group's limit, first click the application link to join the "Alibaba Cloud Big Data & AI Platform" exchange group, and then scan the QR code below to join the DataWorks product DingTalk exchange group for pre-sales and after-sales consultation. You can consult the intelligent robot directly or contact on-duty staff during business hours.
Solution
- Check the value of the concurrent parameter in the task configuration. If the concurrency is set too high, adjust it based on the machine's maximum supported concurrency and the number of tasks running simultaneously.
- If the concurrency setting for a single task exceeds the machine's maximum limit, stop all of its concurrent tasks, modify the concurrency value, and then run it again.
  In the Operation Center navigation pane, go to the Dashboard and open the Data Integration tab to view the Running Details of Synchronization Task. The page includes filters at the top, including Submission time, Task status, Task name, Source data source type, Resource group for Data Integration, Destination data source type, and Has public network traffic. The task list includes columns such as Instance ID, Task name, Task status, Submission time, Start time, End time, Duration, Source data source, Destination data source, Resource group for Data Integration, Running concurrency, Data synchronized, and Synchronization progress. Use this information to investigate why the task is waiting.
  Note
  
  Configure task concurrency based on the maximum concurrency supported by the machine, the number of tasks that run at the same time, and the sum of concurrency values across all running tasks.
  
  A Data Integration task occupies one scheduling resource. If a task runs for an extended period without completing, it may block other tasks from running.
- Wait for running tasks to complete and release resources, or scale up your exclusive resource group for Data Integration.
  Note If a Data Integration task is waiting for a resource group, see Speed up or throttle offline synchronization tasks for solutions.
  - Check with Intelligent Diagnosis
    Go to Operation Center > Intelligent Diagnostics, enter an instance ID to perform an end-to-end analysis of the task. This lets you check the task's execution and queue status and identify which other tasks are consuming resources. For more details, see Intelligent Diagnosis.
    
    For an instance node that is Waiting for Resources, right-click it and select Run Diagnostics > ResourcesRunning StatusWaiting for Resources to view other running tasks.
  - Check in the DataWorks console
    In the navigation pane on the left of the DataWorks console, click Resource GroupsResource Groups. On this page, check the Utilization rateUtilization rate column. Click the usage percentage to view Resource Groups and Resource Usage.
  - Check in Resource
    In the navigation pane on the left of Operation Center, click Resource O&M to view resource usage and running tasks.
    Note The maximum number of instances that can run concurrently varies based on the specifications of the exclusive resource group for Data Integration. For more information, see Subscription billing for exclusive resource groups for Data Integration.