Enable Multi-GPU Sharing with Shared GPU Scheduling-Container Service for Kubernetes(ACK)-阿里云帮助中心

During model development, you may need multiple GPU cards without requiring large amounts of resources. Allocating every GPU card to a development platform can waste resources. Multi-GPU sharing lets a Pod draw equal GPU memory allocations from multiple cards without exclusively occupying any. ACK Pro clusters support this with GPU memory isolation for finer-grained resource utilization.

Prerequisites

Ensure the following:

Limitations

Multi-GPU sharing supports GPU memory isolation with computing power sharing only, not computing power allocation.

How it works

Multi-GPU sharing lets a single Pod request GPU memory from multiple cards. Each card contributes an equal share.

Mode	Description
Single-GPU sharing	A Pod uses part of one GPU card's resources.
Multi-GPU sharing	A Pod spans multiple GPU cards, with each card contributing equal GPU memory.

Allocation formula: If a Pod requests N GiB from M GPU cards, each card allocates N/M GiB.

For example, a Pod requesting 8 GiB across 2 cards receives 4 GiB from each.

Constraints:

N/M must be an integer.
All M GPU cards must be on the same Kubernetes node.

Configure multi-GPU sharing

Log on to the ACK console. In the left navigation pane, click Clusters.
On the Clusters page, click the name of your cluster. In the left navigation pane, click Workloads > Jobs.

On the Jobs page, click Create from YAML. Paste the following YAML into Template and click Create.

apiVersion: batch/v1
kind: Job
metadata:
  name: tensorflow-mnist-multigpu
spec:
  parallelism: 1
  template:
    metadata:
      labels:
        app: tensorflow-mnist-multigpu
        aliyun.com/gpu-count: "2"    # Number of GPU cards to use
    spec:
      containers:
      - name: tensorflow-mnist-multigpu
        image: registry.cn-beijing.aliyuncs.com/ai-samples/gpushare-sample:tensorflow-1.5
        command:
        - python
        - tensorflow-sample-code/tfjob/docker/mnist/main.py
        - --max_steps=100000
        - --data_dir=tensorflow-sample-code/data
        resources:
          limits:
            aliyun.com/gpu-mem: 8    # Total GPU memory in GiB across all cards
        workingDir: /root
      restartPolicy: Never

Key parameters:

Parameter	Type	Description
`aliyun.com/gpu-count`	String (Pod label)	Number of GPU cards. Set in `metadata.labels`. Example: `"2"` requests memory from 2 cards.
`aliyun.com/gpu-mem`	Integer (resource limit)	Total GPU memory (GiB) across all cards. Set in `resources.limits`. Example: `8` means 8 GiB total — 4 GiB per card.

Verify GPU memory isolation

After the Job starts, verify the Pod can access only its allocated GPU memory.

On the Clusters page, click the name of your cluster. In the left navigation pane, click Workloads > Pods.

For the Pod (for example, tensorflow-mnist-multigpu-***), click Actions > Terminal and run:

nvidia-smi

Expected output:

Wed Jun 14 03:24:14 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.161.03   Driver Version: 470.161.03   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla V100-SXM2...  On   | 00000000:00:09.0 Off |                    0 |
| N/A   38C    P0    61W / 300W |    569MiB /  4309MiB |      2%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  Tesla V100-SXM2...  On   | 00000000:00:0A.0 Off |                    0 |
| N/A   36C    P0    61W / 300W |    381MiB /  4309MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

Confirm the following in the output:

Two GPU cards are listed (GPU 0 and GPU 1), matching aliyun.com/gpu-count: "2".
Each card shows 4309 MiB total memory — 4 GiB per card, not the physical 16,160 MiB, confirming GPU memory isolation.

For the same Pod, click Actions > Logs. Confirm this output appears twice (once per card):
```
totalMemory: 4.21GiB freeMemory: 3.91GiB
```
A totalMemory of approximately 4 GiB per card — instead of the physical 16,160 MiB — confirms GPU memory isolation is active.