Each workspace includes two default queues, dev_queue and root_queue, for running jobs. If you need to isolate and manage resources, you can create additional resource queues. This topic describes how to manage resource queues.
Prerequisites
You have created a workspace. For more information, see Manage workspaces.
Create a queue
You can isolate and manage resources by adding queues. If you do not add new resource queues, you can use the default dev_queue and root_queue to run jobs. If you purchase a subscription quota, the system automatically creates a subscription_queue queue.
-
Go to the Resource Management page.
-
Log on to the E-MapReduce console.
-
In the navigation pane, choose EMR Serverless > Spark.
-
On the Spark page, click the name of the target workspace.
-
On the EMR Serverless Spark page, click Resource Management in the navigation pane.
-
-
On the Queue Resources tab, click Create Queue.
-
In the Create Queue dialog box, configure the parameters based on your workspace's billing method, and then click OK.
Pay-as-you-go
Parameter
Description
Queue Name
The queue name must be unique within the workspace.
NoteYou cannot change the Queue Name after the queue is created.
Auto Maximum Concurrency (CU)
Specifies the maximum number of compute units (CUs) that can run concurrently in the queue. Based on the configuration of your session instances, set this value to at least 3 for the selected deployment queue.
To adjust the concurrency limit for other queues, see Adjust queue resources.
Application Environment
-
Development: Recommended for non-critical workloads, such as development and testing.
-
Production: Recommended for business-critical workloads that require stable, efficient, and secure service.
Subscription
Parameter
Description
Queue Name
The queue name must be unique within the workspace.
NoteYou cannot change the Queue Name after the queue is created.
Fixed Resource Quota (CU)
The maximum number of CUs you can allocate depends on the remaining subscription quota in the workspace. If the quota is insufficient, you can:
-
Adjust the resource allocation of other queues. For more information, see Adjust queue resources.
-
Purchase additional subscription quota. For more information, see Compute Resources (Subscription).
Application Environment
-
Development: Recommended for non-critical workloads, such as development and testing.
-
Production: Recommended for business-critical workloads that require stable, efficient, and secure service.
Hybrid Billing
Parameter
Description
Queue Name
The queue name must be unique within the workspace.
NoteYou cannot change the Queue Name after the queue is created.
Auto Maximum Concurrency (CU)
Specifies the maximum number of compute units (CUs) that can run concurrently in the queue. Based on the configuration of your session instances, set this value to at least 3 for the selected deployment queue.
To adjust the concurrency limit for other queues, see Adjust queue resources.
Fixed Resource Quota (CU)
The maximum number of CUs you can allocate is determined by the remaining subscription quota in the workspace. If the quota is insufficient, you can adjust the resource allocation of other queues.
For more information, see Adjust queue resources.
Application Environment
-
Development: Recommended for non-critical workloads, such as development and testing.
-
Production: Recommended for business-critical workloads that require stable, efficient, and secure service.
-
Edit a queue
Adjust queue resources
The steps vary depending on your workspace's billing method.
Pay-as-you-go
-
On the Queue Resources tab of the Resource Management page, find the target queue and click Edit Queue in the Actions column.
-
In the Edit Queue dialog box, on the Resources tab, adjust the value for Auto Maximum Concurrency (CU).
-
Click OK.
Subscription
-
On the Queue Resources tab of the Resource Management page, find the target queue and click Edit Queue in the Actions column.
-
In the Edit Queue dialog box, on the Resources tab, adjust the value for Fixed Resource Quota (CU).
-
Click OK.
Hybrid Billing
-
On the Queue Resources tab of the Resource Management page, find the target queue and click Edit Queue in the Actions column.
-
In the Edit Queue dialog box, on the Resources tab, adjust the values for Fixed Resource Quota (CU) and Auto Maximum Concurrency (CU).
-
Click OK.
Edit queue permissions
-
On the Queue Resources tab of the Resource Management page, find the target queue and click Edit Queue in the Actions column.
-
In the Edit Queue dialog box, click the Permission tab.
-
You can add or remove users, and modify their permissions.
-
Click OK.
Delete a queue
-
You cannot delete the
root_queue. -
Before you delete a resource queue, ensure that no running or pending jobs in the workspace are associated with it. Deleting a queue that is in use affects all associated jobs.
-
On the Queue Resources tab of the Resource Management page, find the target queue and click Delete in the Actions column.
-
In the Delete Queue dialog box, click Delete Queue.
References
-
To create and run workflows, see Manage workflows.
-
To manage permissions, see Manage users and roles.
-
To manage SQL sessions, see Manage SQL sessions.