Manage resource queues
Each workspace includes two default queues, dev_queue and root_queue, for running jobs. If you need to isolate and manage resources, you can create additional resource queues. This topic describes how to manage resource queues.
Prerequisites
You have created a workspace. For more information, see Manage workspaces.
Create a queue
You can isolate and manage resources by adding queues. If you do not add new resource queues, you can use the default dev_queue and root_queue to run jobs. If you purchase a subscription quota, the system automatically creates a subscription_queue queue.
Go to the Resource Management page.
Log on to the E-MapReduce console.
In the navigation pane, choose EMR Serverless > Spark.
On the Spark page, click the name of the target workspace.
On the EMR Serverless Spark page, click Resource Management in the navigation pane.
Select the CPU or GPU tab based on the type of queue that you want to create, and then click Create Queue on the Queues tab.
In the Create Queue dialog box, configure the parameters based on the queue type, and then click OK.
CPU
On the CPU tab, configure the following parameters based on your workspace's billing method.
Pay-as-you-go
Parameter
Description
Queue Name
The queue name must be unique within the workspace.
NoteYou cannot change the Queue Name after the queue is created.
Auto Maximum Concurrency (CU)
Specifies the maximum number of compute units (CUs) that can run concurrently in the queue. Based on the configuration of your session instances, set this value to at least 3 for the selected deployment queue.
The sum of the auto maximum concurrency of all queues cannot exceed the pay-as-you-go quota of the workspace. If the pay-as-you-go quota is insufficient, adjust the pay-as-you-go quota of the workspace first. For more information, see Manage workspaces.
To adjust the concurrency limit for other queues, see Edit a queue.
Application Environment
Development: Recommended for non-critical workloads, such as development and testing.
Production: Recommended for business-critical workloads that require stable, efficient, and secure service.
You can select both Development and Production to share the queue between development and production environments.
Description
The remarks of the queue. The description can be up to 256 characters in length.
Subscription
Parameter
Description
Queue Name
The queue name must be unique within the workspace.
NoteYou cannot change the Queue Name after the queue is created.
Fixed Resource Quota (CU)
The maximum number of CUs you can allocate depends on the remaining subscription quota in the workspace. If the quota is insufficient, you can:
Adjust the resource allocation of other queues. For more information, see Edit a queue.
Purchase additional subscription quota. For more information, see Compute Resources (Subscription).
Application Environment
Development: Recommended for non-critical workloads, such as development and testing.
Production: Recommended for business-critical workloads that require stable, efficient, and secure service.
Hybrid Billing
Parameter
Description
Queue Name
The queue name must be unique within the workspace.
NoteYou cannot change the Queue Name after the queue is created.
Auto Maximum Concurrency (CU)
Specifies the maximum number of compute units (CUs) that can run concurrently in the queue. Based on the configuration of your session instances, set this value to at least 3 for the selected deployment queue.
To adjust the concurrency limit for other queues, see Edit a queue.
Fixed Resource Quota (CU)
The maximum number of CUs you can allocate is determined by the remaining subscription quota in the workspace. If the quota is insufficient, you can adjust the resource allocation of other queues.
For more information, see Edit a queue.
Application Environment
Development: Recommended for non-critical workloads, such as development and testing.
Production: Recommended for business-critical workloads that require stable, efficient, and secure service.
GPU
Switch to the GPU tab and configure the following parameters. After you purchase a subscription GPU quota for the first time, the system automatically creates a
gpu_<spec>_queuequeue. When you add a pay-as-you-go GPU quota for the first time, the system initializes thegpu_dev_queueandgpu_root_queuequeues at a card ratio of 1:4.Parameter
Description
Queue Name
The queue name must be unique within the workspace.
NoteYou cannot change the queue name after the queue is created.
Billing Type
Pay-as-you-go or subscription.
GPU (Cards) Maximum Concurrency
The maximum number of GPU cards that the queue can use to concurrently process jobs.
Instance Type
Limit GPU instance types is enabled by default. When this option is enabled, only the GPU instance types within the selected range can be used.
Application Environment
Development: Recommended for non-critical workloads, such as development and testing.
Production: Recommended for business-critical workloads that require stable, efficient, and secure service.
You can select both Development and Production to share the queue between development and production environments.
Description
The remarks of the queue.
Edit a queue
CPU
On the CPU tab, find the target queue and click Edit Queue in the Actions column. On the Resources tab of the Edit Queue dialog box, you can adjust the queue resource configurations and modify the Application Environment and Description in addition. You cannot change the queue name. The resource configurations vary based on your workspace's billing method.
Pay-as-you-go
On the Queues tab of the Resource Management page, find the target queue and click Edit Queue in the Actions column.
In the Edit Queue dialog box, on the Resources tab, adjust the value for Auto Maximum Concurrency (CU).
Click OK.
Subscription
On the Queues tab of the Resource Management page, find the target queue and click Edit Queue in the Actions column.
In the Edit Queue dialog box, on the Resources tab, adjust the value for Fixed Resource Quota (CU).
Click OK.
Hybrid Billing
On the Queues tab of the Resource Management page, find the target queue and click Edit Queue in the Actions column.
In the Edit Queue dialog box, on the Resources tab, adjust the values for Fixed Resource Quota (CU) and Auto Maximum Concurrency (CU).
Click OK.
GPU
Switch to the GPU tab, find the target queue, and click Edit Queue in the Actions column. On the Resource tab of the Edit Queue dialog box, you can adjust the GPU (Cards) Maximum Concurrency and Instance Type values, and modify the Application Environment and Description in addition. You cannot change the queue name.
Rules for editing GPU queues:
Number of cards: the adjusted value must be greater than or equal to the number of GPU cards in use.
Instance types: you can only add instance types, not remove them.
Edit queue permissions
On the Queues tab of the Resource Management page, find the target queue and click Edit Queue in the Actions column.
In the Edit Queue dialog box, click the Permission tab.
You can add or remove users, and modify their permissions.
Click OK.
Delete a queue
You cannot delete the
root_queue.Before you delete a resource queue, ensure that no running or pending jobs in the workspace are associated with it. Deleting a queue that is in use affects all associated jobs.
Select the CPU or GPU tab based on the type of the target queue, find the target queue on the Queues tab, and then click Delete in the Actions column.
In the Delete Queue dialog box, click Delete Queue.
References
To create and run workflows, see Manage workflows.
To manage permissions, see Manage users and roles.
To manage SQL sessions, see Manage SQL sessions.