Resource Group Orchestration

更新时间:
复制 MD 格式

The Resource Group Orchestration feature in DataWorks helps you change the scheduling resource groups for multiple nodes within a workflow. When your workspace has multiple scheduling resource groups, you can use this feature to quickly reassign them to nodes based on your business needs and optimize resource usage. This topic describes how to use Resource Group Orchestration.

Prerequisites

  • Resource Group Orchestration is workflow-based. You must create a workflow first. For more information, see Create an auto triggered workflow.

  • Resource Group Orchestration is used to allocate scheduling resource groups within your workspace. You must have at least one scheduling resource group available. To purchase and use a serverless resource group, see Use a serverless resource group.

Background information

The Resource Group Orchestration feature allows you to change the scheduling resource groups for multiple nodes in a workflow during development. Different resource group specifications support different maximum task concurrencies. You can assign target nodes to different resource groups based on your business requirements. For more information about resource group specifications and their supported task concurrency limits, see Billing of serverless resource groups.

Note

To change the scheduling resource groups for multiple nodes in the production environment, see Batch operations for auto triggered nodes.

Limits

  • DataWorks supports Resource Group Orchestration only for scheduling resource groups used by auto triggered nodes.

  • Zero-load nodes do not consume scheduling resources at runtime. Therefore, you cannot change their scheduling resource group.

  • This feature is available only in DataWorks Standard Edition and later versions.

Procedure

  1. Go to the DataStudio page.

    Log on to the DataWorks console. In the target region, click Data Development and O&M > Data Development in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Development.

  2. On the DataStudio page, hover over the target workflow and click the 资源组编排 icon to the right of the workflow name to open the Resource Group Orchestration page for that workflow.

    Note

    Resource Group Orchestration is workflow-based. You must first locate the target workflow.

  3. Note
    • You can move nodes from different source resource groups to the same target resource group in a single operation.

    • To move nodes to different target resource groups, you must perform the changes in separate operations.

    On the Resource Governance page, select the target nodes from different resource groups, such as Public Scheduling Resource Group and test_shanyun. Click Switch Resource Group. In the resource group selection tree, select the target resource group, such as xc_test, and then click OK. After the switch, the nodes are reassigned to the target resource group, and the source groups show "No data".

    1. Optional: Filter nodes by criteria such as Node Name, Node Type, or Engine Type to find specific nodes.

    2. Confirm the node information and select the target nodes whose resource groups you want to change in bulk.

    3. Click Switch Resource Groups and select the target resource group.

    4. Click Determine to complete the switch.

    Other operations:

    • Configuration: The configuration feature is supported for serverless resource groups (recommended) and earlier-version exclusive scheduling resource groups. Click Configuration next to a resource group to go to the Overview page of DataWorks. In the navigation pane on the left, click Resource Groups to open the Resource Groups page. On the Exclusive Resource Group tab, find the target resource group and click Details to manage its configuration.

    • Purchase Resource Group: If no resource groups are available in the current workspace, click Purchase Resource Group to purchase one.

    • Switch the resource group for a single node: In the Operation column for a target node, click Switch to change the scheduling resource group for that specific node.

  4. Click the 提交 icon to commit the orchestration changes.

  5. After changing a node's resource group, you must deploy it. Once deployed, the node will use the new scheduling resource group in the production environment. For more information about how to deploy nodes, see Deploy tasks.