ack-mps-control

Introduction

NVIDIA MPS allows multiple CUDA applications to run concurrently on a single GPU, making it well-suited for multi-user environments or scenarios that run multiple small tasks simultaneously.

Usage notes

• Do not delete the MPS Control Daemon pods. Doing so will make the GPU applications on the node unavailable. MPS clients (GPU applications using the MPS feature) must interact with the MPS Control Daemon. If this daemon restarts, the associated MPS clients will exit unexpectedly.

• The MPS Control Daemon is deployed as a container and requires privileged permissions, which poses potential security risks. Evaluate your security compliance requirements before deploying this add-on.

• The DaemonSet that deploys the MPS Control Daemon is configured with the node selector ack.node.gpu.schedule=mps. If you have already deployed the ack-cgpu add-on in your cluster, labeling a node with ack.node.gpu.schedule=mps will enable both shared GPU scheduling and MPS resource isolation on that node.

• The MPS Control Daemon pods are configured with priorityClassName: system-node-critical to elevate their scheduling priority. This prevents them from being evicted under resource-constrained conditions, ensuring high availability of the MPS service for your workloads.

• Once the MPS capability is enabled on a node, any GPU application pod running on that node must have hostIPC: true configured in its spec.

Changelog