Overview of elastic resource scheduling, AI task scheduling, heterogeneous resource scheduling, and task queue scheduling to improve cluster resource utilization and job efficiency.
Elastic scheduling
Mix ECS, ECI, and preemptible instances in one application, with priority-based policies controlling which resource type scales out first and releases first.
| Feature | Scenario | References |
|---|---|---|
| Elastic scheduling | Reduce costs by prioritizing cheaper resources during scale-out (for example, ECS before ECI) and releasing them first during scale-in. Supports subscription, pay-as-you-go, and preemptible instances. | Use Elastic Container Instance-based scheduling and Configure priority-based resource scheduling |
Task scheduling
Gang scheduling, Capacity Scheduling, and Kube Queue manage batch and AI workloads.
| Feature | Scenario | References |
|---|---|---|
| Gang scheduling | Distributed training or batch jobs requiring simultaneous task starts. Without gang scheduling, partial starts block resources and deadlock. Gang scheduling starts all correlated processes together to prevent blocking. | Work with gang scheduling |
| Capacity Scheduling | Multi-team clusters where teams use resources at different times. Standard Kubernetes resource quotas allocate fixed amounts per namespace, leaving resources idle when quotas go unused. Capacity Scheduling lets teams share idle resources across quota boundaries. | Use Capacity Scheduling |
| Kube Queue (ack-kube-queue) | Large clusters with AI and batch workloads from multiple users. Pod-level scheduling degrades at high job counts, and jobs from different users interfere. ack-kube-queue manages queues with customizable policies and integrated quotas to maximize utilization. | Use ack-kube-queue to manage job queues |
Heterogeneous resource scheduling
Schedule GPUs with cGPU sharing and topology-aware placement for both CPUs and GPUs. For GPU scheduling node labels, see Labels used by ACK to control GPUs.
GPU sharing with cGPU
cGPU lets multiple pods share a GPU with memory isolation, reducing GPU costs and improving workload stability. ACK Pro clusters support these policies:
| Policy | Use when | Description |
|---|---|---|
| One-pod-one-GPU sharing and memory isolation | Model inference | One pod per GPU with memory isolation between pods sharing the same GPU. |
| One-pod-multi-GPU sharing and memory isolation | Distributed model training | One pod spans multiple GPUs with memory isolation for distributed model training. |
| binpack or spread allocation | Improve GPU utilization and high availability | Allocates GPUs with binpack or spread algorithms to improve utilization and ensure high availability. |
See cGPU Professional Edition for setup instructions.
Topology-aware CPU and GPU scheduling
The scheduler places performance-sensitive workloads based on node hardware topology: GPU-to-GPU paths (NVLink, PCIe Switches) and CPU NUMA topology.
| Feature | References |
|---|---|
| Topology-aware CPU scheduling | Topology-aware CPU scheduling |
| Topology-aware GPU scheduling | Overview |
FPGA scheduling
Use labels to schedule FPGA workloads to accelerated nodes and manage FPGA resources cluster-wide.
| Feature | References |
|---|---|
| FPGA scheduling | Use labels to schedule pods to FPGA-accelerated nodes |
Task queue scheduling
Customize task queue scheduling for AI and batch workloads.