Create and manage resource groups
This topic describes how to create and manage resource groups in AnalyticDB for MySQL, including their billing, creation, modification, deletion, and monitoring.
Limitations
A Data Warehouse Edition cluster must meet all of the following requirements:
-
The cluster runs in Elastic mode.
-
The cluster has 32 or more cores of computing resources.
-
The kernel version is 3.1.3.2 or later.
NoteTo view and update the minor version, go to the Configuration Information section on the Cluster Information page in the AnalyticDB for MySQL console.
Billing
Enterprise, Basic, and Data Lakehouse
-
Interactive and Job resource groups are billed for the elastic resources they consume, measured in ACUs.
-
For AI resource groups with the Ray Cluster deployment type:
-
When the Worker Resource Type and Head Node Resource Type are set to CPU, the AI resource group is billed based on the ACUs of elastic resources consumed.
-
When the Worker Resource Type and Head Node Resource Type are set to GPU, the AI resource group is billed based on GPU specifications and quantity.
-
Worker Disk Storage and Head disk space are billed for their configured storage size.
-
You can view the amount of elastic resources used by resource groups in the following ways:
-
For Enterprise Edition and Basic Edition clusters: On the Cluster Management > Resource management > Resource Overview page, you can view the total and reserved resources used by all resource groups. Elastic resource usage is the difference between total and reserved resources.
-
For Data Lakehouse Edition clusters: On the Cluster Management > Resource management > Resource Overview page, you can view the total and reserved computing resources used by all resource groups. Elastic resource usage is the difference between total and reserved computing resources.
Data warehouse edition
Resource group fees are included in your computing resource costs. You are charged only for the computing resources you use.
Create a resource group
Enterprise, Basic, and Data Lakehouse
By default, each cluster has one Interactive resource group (user_default). For newly purchased clusters with kernel version 3.2.2.8 or later, a Job resource group (serverless) is also created by default. If no other resource groups exist, all XIHE queries run in the user_default resource group, and all Spark jobs (including Spark Jar and Spark SQL) run in the serverless resource group. To isolate query resources, you must create additional resource groups.
Log on to the AnalyticDB for MySQL console. In the upper-left corner of the console, select a region. In the left-side navigation pane, click Clusters. Find the cluster that you want to manage and click the cluster ID.
-
In the left-side navigation pane, choose Cluster Management > Resource Management and click the Resource Groups tab. In the upper-right corner of the resource group list, click Create Resource Group.
-
Enter a name for the resource group and select a Job Type.
-
Select Interactive for online workloads that require high QPS and low latency.
An Interactive resource group uses resident computing resources and runs queries in MPP mode to deliver responses in milliseconds.
-
Select Job for offline workloads that require high throughput.
A Job resource group provisions temporary computing resources to run queries in BSP mode, with response times in seconds or minutes. The amount of resources provisioned ranges from 0 ACU to the group's maximum limit, depending on the running task's size.
-
Select AI for heterogeneous computing workloads.
An AI resource group supports heterogeneous computing with GPUs and CPUs and various deployment types such as MLSQL models and Ray-managed computing.
ImportantYou cannot change the task type after the resource group is created.
-
-
The configurable properties depend on the task type you select. After setting the properties, click OK.
Interactive resource group properties
Parameter
Description
Engine
-
XIHE: The resource group can only execute XIHE SQL queries.
-
Spark: The resource group can only execute Spark SQL jobs, which are processed in interactive mode.
ImportantYou cannot change the engine after the resource group is created.
Automatic Stop
If an Interactive resource group is idle for a specified period, its running clusters are automatically released.
This feature reduces costs by preventing resource waste, but restarting resources for the next query introduces a delay.
ImportantThis parameter is available only when the Engine is set to Spark.
Cluster Size
-
If the Engine is XIHE: The size of a single cluster in ACUs. The minimum value is 16 ACU.
-
If the Engine is Spark: The size of a single cluster, which is the number of ACUs allocated to a Spark application. The minimum value is 24 ACU. Each Spark Interactive resource group can run multiple Spark applications. The Minimum Clusters and Maximum Clusters settings determine the number of Spark applications that can run in the resource group.
For information about the mapping between cluster size and Spark Driver/Executor specifications, see Appendix: Mapping between cluster size and Spark Driver and Spark Executor specifications.
Minimum Clusters
Maximum Clusters
Minimum Clusters: The minimum number of clusters that must run in the resource group. The minimum value is 1.
Maximum Clusters: The maximum number of clusters that the resource group can scale out to. The maximum value is 10.
If Minimum Clusters and Maximum Clusters are different, AnalyticDB for MySQL dynamically scales the number of clusters within this range based on query load.
If Minimum Clusters and Maximum Clusters are the same, AnalyticDB for MySQL starts a fixed number of clusters, giving you static control over the group's total computing resources.
NoteIf Minimum Clusters or Maximum Clusters is 2 or greater, the Multi-Cluster feature is enabled for the resource group. For more information, see Multi-Cluster elastic model.
Job Resubmission Rules
Routes queries that exceed the Query Execution Time Threshold to the specified Target Resource Group. For more information, see Job resubmission.
ImportantThis parameter is available only when the Engine is set to XIHE.
Spark Configuration
Sets configuration parameters for all Spark jobs in this resource group. To configure a specific job, set its parameters in the submission code.
For more information about Spark configuration parameters, see Spark application configuration parameters.
ImportantThis parameter is available only when the Engine is set to Spark.
Job resource group properties
Parameter
Description
Minimum Computing Resources
The minimum value is 0 ACU.
ImportantThe minimum computing resources cannot be changed after the resource group is created.
Maximum Computing Resources
In the console, you can set the maximum computing resources up to 1,024 ACU in increments of 8 ACU. To request a higher limit, you can submit a ticket.
Spot Instance
Specifies whether to enable spot instances.
If you enable this option, Spark jobs that run in the Job resource group attempt to use spot instance resources. For more information, see Spot instances.
Spark Configuration
Sets configuration parameters for all Spark jobs in this resource group. To configure a specific job, set its parameters in the submission code.
For more information about Spark configuration parameters, see Spark application configuration parameters.
AI resource group
Parameter
Description
Deployment Mode
Select RayCluster.
Head Node Resource Type
Supports CPU and GPU types.
-
CPU: Use for routine computing tasks, multitasking, or complex logical operations. We recommend using the CPU type for production environments.
-
GPU: Use for large-scale parallel data processing, machine learning, or deep learning training. In development and test environments, you can select the GPU type if you need GPU resources without creating worker nodes.
Head Resource Specifications
-
If the Head Resource Specifications is CPU, you can select specifications such as small, m.xlarge, and m.2xlarge. The CPU core count for each specification is the same as for Spark resource specifications. For more information, see Spark resource specification list.
-
If the Head Resource Specifications is GPU, contact technical support for assistance with model selection due to GPU model and inventory constraints.
ImportantThe head node primarily handles job scheduling. Select a specification based on the overall scale of your Ray Cluster.
Head disk space
Used to store Ray logs, temporary data, and spilled object data. Unit: GB. Range: 30 to 2,000. Default: 100.
ImportantThis disk is for temporary storage only. Do not use it for long-term data.
Allocation Unit
The number of GPUs to allocate per head node. For example, a value of 1/3 assigns one-third of a GPU to each head node.
ImportantThis parameter is required only when the Head Resource Specifications is GPU.
Worker Group Name
A custom name for the Worker Group. An AI resource group can have multiple Worker Groups with different names.
Worker Resource Type
Supports CPU and GPU types.
-
If your workload involves routine computing tasks, multitasking, or complex logical operations, we recommend CPU.
-
If your workload involves large-scale parallel data processing, machine learning, or deep learning training, we recommend GPU.
Worker Resource Specifications
-
If the Worker Resource Type is CPU, you can select specifications such as small, m.xlarge, and m.2xlarge. The CPU core count for each specification is the same as for Spark resource specifications. For more information, see Spark resource specification list.
-
If the Worker Resource Type is GPU, submit a ticket to contact technical support for assistance with model selection due to GPU model and inventory constraints.
Worker Disk Storage
Used to store Ray logs, temporary data, and spilled object data. Unit: GB. Range: 30 to 2,000. Default: 100.
ImportantThis disk is for temporary storage only. Do not use it for long-term data.
Minimum Workers
Maximum Workers
Minimum Workers: The minimum number of workers that must run in a Worker Group. The minimum value is 0.
Maximum Workers: The maximum number of workers that can run in a Worker Group. The maximum value is 8.
Worker Groups support auto-scaling, and each Worker Group can scale independently. If the minimum and maximum worker counts differ, AnalyticDB for MySQL scales the number of workers within this range based on task load. For clusters with multiple Worker Groups, the system automatically selects the best-fit group for a task, preventing individual groups from being overloaded or idle.
Allocation Unit
The number of GPUs to allocate per worker node. For example, a value of 1/3 assigns one-third of a GPU to each worker node.
ImportantThis parameter is required only when the Worker Resource Type is GPU.
-
Data warehouse edition
Log on to the AnalyticDB for MySQL console. In the upper-left corner of the console, select a region. In the left-side navigation pane, click Clusters. Find the cluster that you want to manage and click the cluster ID.
-
In the left-side navigation pane, click Resource Groups.
-
On the Resource Groups page, click Create Resource Group in the upper-right corner of the resource group list.
-
Configure the resource group.
Parameter
Description
Resource Group Name
A custom name for the resource group. The name must be 2 to 30 characters in length, start with a letter, and can contain only letters, digits, and underscores (_).
Query Type
The common SQL query type for this resource group. For more information, see Query execution modes.
-
Default_Type: The default query type.
-
Batch: Suitable for complex queries on large datasets, such as Extract-Transform-Load (ETL) queries. Intermediate results can be written to disk, which may reduce query performance on large datasets but prevents compute nodes from failing due to excessive data volume.
-
Interactive: Suitable for real-time analytical queries that require low latency. This is a fast, memory-based interactive query type. It offers high query performance, but queries may fail if the data volume exceeds the machine's processing capacity.
Resource Amount
The amount of computing resources to allocate to the group.
-
-
Click OK to create the resource group.
Modify a resource group
Enterprise, Basic, and Data Lakehouse
Modifiable properties
-
For custom resource groups (resource groups that you create), you can modify the following properties:
-
For Interactive resource groups: Auto Stop, Cluster size, Minimum clusters, Maximum clusters, Job resubmission rule, and Spark configuration.
-
For Job resource groups: Maximum computing resources, Spot instance, and Spark configuration.
-
For AI resource groups (Ray Cluster deployment type): Head resource specifications, Worker resource type, Worker resource specifications, Worker disk space, Minimum workers, and Maximum workers.
Other properties cannot be modified, including Resource Group Name, Task type, the Engine for an Interactive resource group, the Minimum computing resources for a Job resource group, and the Deployment Type and Worker Group Name for an AI resource group.
-
-
For default resource groups (named
user_defaultandserverless):-
For Enterprise Edition and Basic Edition, you can only modify the job resubmission rule for the
user_defaultresource group. Theserverlessresource group cannot be modified. -
For Data Lakehouse Edition, you can modify the Reserved Computing Resources and job resubmission rule for the
user_defaultresource group. Theserverlessresource group cannot be modified.
-
Procedure
-
On the Resource Groups page, find the target resource group and click Modify in the Actions column.
-
In the Modify Resource Group panel, modify the properties and click OK.
The changes take effect when the resource group's status becomes "Running".
Data warehouse edition
Modifiable properties
After a resource group is created, you can modify its query type or resource amount.
-
For the default resource group (named
user_default), you can only modify the query type. You cannot manually modify the resource amount.NoteThe resource amount for the default resource group is calculated as: Total cluster resources minus the resources used by all other resource groups in the cluster.
-
For custom resource groups (resource groups that you create), you can modify both the query type and the resource amount.
Procedure
-
On the Resource Groups page, find the target resource group and click Modify in the Actions column.
-
Modify the Query Type or Resource Amount as needed.
-
After you are finished, click OK.
Changes to the resource amount of an AnalyticDB for MySQL resource group take effect immediately.
Delete a resource group
Default resource groups (user_default and serverless) cannot be deleted.
Impact of deletion
-
Deleting a resource group interrupts any tasks running in it.
-
If a deleted resource group is specified in a XIHE SQL script or Spark job, you must update the code to target a new group. Otherwise, XIHE SQL jobs will run in the default resource group, and Spark jobs will fail.
Procedure
On the Resource Groups page, find the target resource group and click Delete in the Actions column. In the confirmation dialog box, click OK to delete the resource group.
Monitor resource usage
You can view cluster-level resource usage, resource group-level resources and load, and job-level resource consumption. For detailed descriptions of each monitoring metric, see Resource group monitoring.
Cluster's reserved and elastic resources
-
For Enterprise Edition and Basic Edition clusters: On the Cluster Management > Resource Management>Resource Overview page, view the Total Resources and Reserved Resources for all resource groups in the cluster at a specific point in time. The elastic resource usage is the difference between Total Resources and Reserved Resources.
-
For Data Lakehouse Edition clusters: On the Cluster Management > Resource Management>Resource Overview page, view the Total Computing Resources and Reserved Computing Resources for all resource groups in the cluster at a specific point in time. The elastic resource usage is the difference between Total Computing Resources and Reserved Computing Resources.
Resource group resources and load
You can view the computing resources used by a single resource group and evaluate its load by monitoring metrics such as running and queued queries, Spark engines, and connections.
On the Cluster Management > Resource Management>Resource Groups page, find the target resource group and click Monitoring to view the actual computing resources used by that resource group.
Job resource consumption
The Job Usage Statistics page provides resource consumption statistics for the following jobs: XIHE BSP jobs, Spark jobs, and SLS/Kafka data synchronization and data migration tasks from the AnalyticDB for MySQL console.
On the Cluster Management > Resource Management>Job Usage Statistics page, you can view the total, reserved, elastic, and spot instance resources consumed by a job.
FAQ
My cluster has 32 ACU of reserved resources. Do the default and custom resource groups both consume the 32 ACU of reserved resources at the same time?
For Enterprise Edition or Basic Edition clusters, all reserved resources are allocated exclusively to the default resource group user_default. The serverless default resource group and any custom Job or Interactive resource groups consume only elastic resources.
For Data Lakehouse Edition clusters, you can allocate reserved resources to the default resource group user_default, the serverless default resource group, and custom Job or Interactive resource groups. The amount allocated to user_default becomes its fixed resource limit. Any remaining portion of the cluster's total reserved resources is then available to the other groups.
Related APIs
You can use the OpenAPI to create, modify, and delete resource groups, or to manage their database account bindings:
-
For Enterprise Edition, Basic Edition, or Data Lakehouse Edition, see Resource group management.
-
For Data Warehouse Edition, see Resource group management.