AI node storage usage
This topic describes the storage options supported by PolarDB-X AI nodes, helping you choose the appropriate storage solution for different requirements such as model files, shared data, persistent data, and temporary cache. Pods on AI nodes can mount storage through Kubernetes PersistentVolume (PV), PersistentVolumeClaim (PVC), and emptyDir. Storage resources must match the network environment of the AI node, including the Kubernetes cluster, VPC, and region.
Supported storage options
|
Storage option |
Use case |
Access mode |
Data lifecycle |
|
ESSD cloud disk |
Model caching, intermediate inference results, KV Cache, and data that requires persistent read/write |
Typically suitable for a single Pod or a small number of Pods |
Cloud disks exist independently of Pods and are not released by default after the Pod is deleted. |
|
CPFS |
Large model weights, high-throughput shared files, and shared read/write across multiple Pods |
Multiple Pods can share access |
Data is stored in the CPFS file system and is not deleted when Pods are removed. |
|
OSS Bucket |
Model weights, configuration files, shared read-only data, and archived data |
Multiple Pods can share access |
Data is stored in OSS and is not deleted when Pods are removed. |
|
NAS |
General shared files, configuration files, and lightweight shared read/write data |
Multiple Pods can share access |
Data is stored in the NAS file system and is not deleted when Pods are removed. |
|
Local ephemeral storage |
Temporary compute files and scratch data |
In-Pod access |
Follows the Pod lifecycle and is released when the Pod is deleted. |
|
Memory-backed temporary directory |
|
In-Pod access |
Uses available Pod memory and is released when the Pod is deleted. |
Selection recommendations
|
Requirement |
Recommended solution |
|
High-performance persistent block storage required |
ESSD cloud disk |
|
Multiple inference Pods sharing large model weights with high throughput requirements |
CPFS |
|
Multiple Pods sharing model files, configuration files, or archived files |
OSS Bucket |
|
General shared file system capabilities required |
NAS |
|
Only temporary compute files need to be stored |
Local ephemeral storage |
|
Need to mount |
Memory-backed temporary directory |
Limits
-
Storage resources must be network-connected to the cluster where the AI node resides. We recommend that you select storage resources in the same region and VPC or accessible over the internal network.
-
ESSD cloud disks, CPFS, OSS, and NAS all require creating a PVC first and then referencing the PVC in the Pod.
-
The performance and consistency semantics of shared storage such as OSS, NAS, and CPFS are determined by the corresponding cloud product. Select based on your business read/write model.
-
Local ephemeral storage and memory-backed temporary directories do not provide persistence. Do not store model weights, business results, or other data that requires long-term retention.
Use ephemeral storage
If you only need to store temporary compute files, you can use Kubernetes emptyDir.
apiVersion: v1
kind: Pod
metadata:
name: ai-node-scratch-example
namespace: default
spec:
volumes:
- name: scratch-volume
emptyDir: {}
containers:
- name: app
image: <image>
volumeMounts:
- mountPath: /scratch
name: scratch-volume
If your application requires /dev/shm, you can mount a memory-backed temporary directory by setting emptyDir.medium: Memory.
apiVersion: v1
kind: Pod
metadata:
name: ai-node-shm-example
namespace: default
spec:
volumes:
- name: shm-volume
emptyDir:
medium: Memory
sizeLimit: 32Gi
containers:
- name: app
image: <image>
volumeMounts:
- mountPath: /dev/shm
name: shm-volume
Set sizeLimit based on application concurrency and memory consumption during model execution to avoid excessive memory usage that affects Pod stability.