cGPU lets multiple pods share a single physical GPU in ACK Pro clusters by isolating both GPU memory and computing power at the software level, so that multiple workloads can safely share one GPU under strict memory and compute limits.
Prerequisites
Before you begin, ensure that you have:
An ACK Pro cluster running Kubernetes 1.20 or later. For more information, see Create an ACK managed cluster.
A kube-scheduler version that meets the requirement for your cluster version: For the full list of features supported by each kube-scheduler version, see kube-scheduler.
ACK cluster version
Scheduler version
1.30 or later
All scheduler versions are supported.
1.28
1.28.1-aliyun-5.6-998282b9 or later
1.26
v1.26.3-aliyun-4.1-a520c096 or later
1.24
1.24.3-ack-2.0 or later
1.22
1.22.15-ack-2.0 or later
1.20
1.20.4-ack-8.0 or later
The GPU sharing add-on installed with a Helm chart version later than 1.2.0. For more information, see Manage the GPU sharing component.
cGPU 1.0.5 or later installed. For more information, see Update the cGPU version on a node.
Limitations
Job type mixing is not supported on the same node. GPU sharing supports two types of jobs: jobs that request only GPU memory, and jobs that request both GPU memory and computing power.
Computing power values must be a multiple of 5, with a minimum of 5. The scale is 0–100, where 100 represents 100% of a GPU's computing power. For example, a value of 20 means the pod uses 20% of the GPU's computing power. If the specified value is not a multiple of 5, the job cannot be submitted.
Computing power allocation is not supported by direct node labeling. To enable computing power isolation on existing GPU-accelerated nodes, remove them from the cluster first and then add them to a node pool that supports computing power isolation. Running
kubectl label nodes <NODE_NAME> ack.node.gpu.schedule=core_memdirectly on existing nodes does not work.Supported regions only. Computing power allocation is available only in the following regions. Ensure that your cluster's region is one of them:
Region
Region ID
China (Beijing)
cn-beijing
China (Shanghai)
cn-shanghai
China (Hangzhou)
cn-hangzhou
China (Zhangjiakou)
cn-zhangjiakou
China (Shenzhen)
cn-shenzhen
China (Chengdu)
cn-chengdu
China (Heyuan)
cn-heyuan
China (Hong Kong)
cn-hongkong
Indonesia (Jakarta)
ap-southeast-5
Singapore
ap-southeast-1
Thailand (Bangkok)
ap-southeast-7
US (Virginia)
us-east-1
US (Silicon Valley)
us-west-1
Japan (Tokyo)
ap-northeast-1
China East 2 Finance
cn-shanghai-finance-1
Clusters created before March 1, 2022 require a manual scheduler update. Clusters created on or after March 1, 2022 automatically use the scheduler version that supports computing power allocation. For older clusters, follow these steps:
to apply for private preview access to shared GPU scheduling.
If the installed GPU sharing add-on supports GPU memory sharing only (Helm chart version 1.2.0 or earlier), uninstall it:
Log on to the ACK console. In the left navigation pane, click Clusters.
Click the cluster name. In the left navigation pane, choose Applications > Helm.
On the Helm page, find ack-ai-installer and click Delete in the Actions column. In the Delete dialog box, click OK.
Install the latest GPU sharing add-on. For more information, see Manage the GPU sharing component.
Step 1: Create a node pool with computing power allocation
Log on to the ACK console. In the left navigation pane, click Clusters.
Click the cluster name. In the left navigation pane, choose Nodes > Node Pools.
On the Node Pools page, click Create Node Pool.
Configure the node pool with the following settings. For all other parameters, see Create and manage a node pool. Under Node Labels, add these two labels:
Parameter
Description
Node Pool Name
A name for the node pool. This topic uses
gpu-coreas an example.Expected Nodes
The initial number of nodes. Set to
0if you do not want to create nodes immediately.Node Labels
Labels to add to the nodes. Configure both of the following labels. For more information, see Labels for enabling GPU scheduling policies.
Key
Value
Purpose
ack.node.gpu.schedulecore_memEnables both GPU memory isolation and computing power isolation on the node.
ack.node.gpu.placementbinpackUses the binpack algorithm to pack pods onto the fewest GPUs, maximizing GPU utilization.
Step 2: Verify that node pool nodes have computing power allocation enabled
Run the following command to check whether computing power allocation is enabled on a node:
kubectl get nodes <NODE_NAME> -o yamlLook for the aliyun.com/gpu-core.percentage field in the allocatable and capacity sections of the output. Its presence confirms that computing power allocation is active.
Expected output for a node with 4 GPUs (each with 15 GiB of memory):
# Irrelevant fields are omitted.
status:
allocatable:
# 4 GPUs x 100% = 400 total computing power units
aliyun.com/gpu-core.percentage: "400"
aliyun.com/gpu-count: "4"
# 4 GPUs x 15 GiB = 60 GiB total GPU memory
aliyun.com/gpu-mem: "60"
capacity:
aliyun.com/gpu-core.percentage: "400"
aliyun.com/gpu-count: "4"
aliyun.com/gpu-mem: "60"Step 3: Deploy a workload with computing power limits
Without computing power allocation, a pod can use 100% of a GPU's computing power and all 15 GiB of its memory. After enabling this feature, you set explicit limits on both resources.
The following example deploys a job that requests 2 GiB of GPU memory and 30% of computing power.
Create a file named
cuda-sample.yamlwith the following content:apiVersion: batch/v1 kind: Job metadata: name: cuda-sample spec: parallelism: 1 template: metadata: labels: app: cuda-sample spec: containers: - name: cuda-sample image: registry.cn-beijing.aliyuncs.com/ai-samples/gpushare-sample:benchmark-tensorflow-2.2.3 command: - bash - run.sh - --num_batches=500000000 - --batch_size=8 resources: limits: aliyun.com/gpu-mem: 2 # GPU memory in GiB aliyun.com/gpu-core.percentage: 30 # Percentage of GPU computing power; must be a multiple of 5 workingDir: /root restartPolicy: NeverKey resource parameters:
Parameter
Value
Description
aliyun.com/gpu-mem2GPU memory in GiB. The pod can use up to 2 GiB.
aliyun.com/gpu-core.percentage30Percentage of GPU computing power. Must be a multiple of 5; 30 means 30%.
Deploy the job:
kubectl apply -f /tmp/cuda-sample.yamlThe image is large. The initial pull may take several minutes.
Verify that the job is running:
kubectl get po -l app=cuda-sampleExpected output:
NAME READY STATUS RESTARTS AGE cuda-sample-m**** 1/1 Running 0 15sRunningin theSTATUScolumn confirms the pod is active.Check the GPU memory and computing power used by the pod:
kubectl exec -ti cuda-sample-m**** -- nvidia-smiExpected output:
Fri May 23 07:23:13 2025 +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.161.07 Driver Version: 535.161.07 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 Tesla T4 On | 00000000:00:08.0 Off | 0 | | N/A 50C P0 36W / 70W | 2038MiB / 2048MiB | 30% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ +---------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=======================================================================================| +---------------------------------------------------------------------------------------+The output confirms isolation is applied:
GPU memory: The pod is limited to 2,048 MiB (approximately 2 GiB), down from the full 15 GiB available before enabling the feature. Current usage is 2,038 MiB.
Computing power: The pod is limited to 30% (
GPU-Util: 30%), down from 100%.
nvidia-smireports computing power utilization per GPU, not per pod. If n pods each request 30% and n is no greater than 3, all pods are scheduled to the same GPU and the output shows n x 30% utilization.Check the pod logs to observe throttling behavior:
kubectl logs cuda-sample-m**** -fExpected output:
Generating training model Initializing graph Running warm up Done warm up Time Step Img/sec total_loss 2025-05-23 07:17 1 images/sec: 16.9 +/- 0.0 (jitter = 0.0) 7.670 2025-05-23 07:17 10 images/sec: 17.0 +/- 0.2 (jitter = 0.5) 7.947 2025-05-23 07:17 20 images/sec: 17.1 +/- 0.1 (jitter = 0.4) 8.113 2025-05-23 07:17 30 images/sec: 17.1 +/- 0.1 (jitter = 0.6) 7.214 2025-05-23 07:18 40 images/sec: 17.1 +/- 0.1 (jitter = 0.7) 7.878 2025-05-23 07:18 50 images/sec: 17.2 +/- 0.1 (jitter = 0.7) 8.056 2025-05-23 07:18 60 images/sec: 17.2 +/- 0.1 (jitter = 0.7) 7.990 2025-05-23 07:18 70 images/sec: 17.2 +/- 0.1 (jitter = 0.7) 7.892 2025-05-23 07:18 80 images/sec: 17.2 +/- 0.1 (jitter = 0.7) 7.133 2025-05-23 07:18 90 images/sec: 17.2 +/- 0.1 (jitter = 0.6) 7.789Log entries appear at a lower rate than for a pod with unrestricted computing power. This confirms that the 30% computing power limit is in effect (throttled to about 30% of the GPU).
(Optional) Delete the job after verification:
kubectl delete job cuda-sample