In an E-HPC cluster, compute nodes provide processing power and queues manage how jobs are distributed across those nodes.
Compute nodes
A compute node is an Elastic Compute Service (ECS) instance dedicated to running jobs and processing data within an E-HPC cluster. Each node contributes abundant computing power and storage capacity to the cluster, making it suitable for workloads such as parallel computing, large-scale data processing, and deep learning training.
Combine multiple compute nodes to scale out your cluster and run ultra-large computing tasks efficiently.
Queues
A queue is a resource allocation unit that manages and schedules jobs in an E-HPC cluster. Each queue defines a pool of compute nodes and the rules that govern how jobs are dispatched to them.
Configure the following settings for each queue to match your workload requirements:
| Setting | Description |
|---|---|
| Job priority | Determines the order in which queued jobs are dispatched to nodes |
| Limits | Caps on job configuration based on your business requirements |
| Scheduling policy | Rules that govern how the queue assigns jobs to available nodes |
Organize queues by job type, user group, or resource requirement to balance loads, improve user experience, and manage cluster resources more efficiently across teams and workloads.
How nodes and queues work together
Nodes and queues are complementary: nodes execute the work, and queues decide which node gets which job.
Nodes as queue members
Each compute node belongs to a queue and executes the jobs assigned to it. A node's current state—idle, busy, or maintenance—determines its position and priority within the queue.
Queue as node manager
The queue system receives incoming jobs, sorts them by priority, resource requirements, and node availability, and assigns each job to an available node. It continuously monitors node health to keep job execution on track.
Dynamic resource allocation
The queue system matches each job to a node with the right resources. For example, memory-intensive jobs are routed to nodes with more memory, optimizing resource utilization and improving cluster performance.
Load balancing
The intelligent scheduling algorithm distributes jobs across nodes to prevent any single node from being overloaded or sitting idle, keeping the cluster running at full efficiency.