Mount a CPFS for Lingjun file system on ACS
Container Compute Service (ACS) supports mounting a CPFS for Lingjun file system. Multiple pods can mount it at the same time to share data.
Prerequisites
You have created an ACS cluster.
For ACS, the CSI components (bmcpfs-csi-controller, bmcpfs-csi-node, and cnfs-nas-daemon) are managed by the ACS platform. You do not need to manage component versions or upgrades.
You have created a CPFS for Lingjun file system and recorded the file system ID. For more information, see Create a CPFS for Lingjun file system.
If a pod is scheduled to a node that does not support RDMA, or the scenario crosses compute-storage affinity domains, you must create a VPC mount point in advance and record the mount point domain name (in the format
cpfs-***-vpc-***.<Region>.cpfs.aliyuncs.com). The VPC that the VPC mount point belongs to must be the same VPC as the ACS cluster. For more information, see Manage VPC mount points.
Step 1: Create a PV and a PVC
Choose the appropriate template based on your pod type:
Lingjun GPU pods in the same compute-storage affinity domain: Use "Method A: RDMA."
CPU pods, or Lingjun access across compute-storage affinity domains: Use "Method B: VPC mount point."
When an ACS Lingjun pod is in the same compute-storage affinity domain as CPFS, it accesses the file system through VSC + RDMA direct mount for the best performance. For CPU pods or Lingjun access across compute-storage affinity domains, access goes through a VPC mount point. For the concept and determination method for compute-storage affinity domains, see Overview of mount access.
RDMA network
kubectl CLI
Create a file named
cpfs-pv-pvc.yamlto define the PV and PVC:apiVersion: v1 kind: PersistentVolume metadata: name: cpfs-test labels: alicloud-pvname: cpfs-test spec: accessModes: - ReadWriteMany capacity: storage: 10Ti csi: driver: bmcpfsplugin.csi.alibabacloud.com volumeAttributes: filesystemId: bmcpfs-***** path: / volumeHandle: bmcpfs-***** --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: cpfs-test spec: accessModes: - ReadWriteMany selector: matchLabels: alicloud-pvname: cpfs-test resources: requests: storage: 10TiParameter description:
-
PersistentVolume parameters
Parameter
Description
labelsThe labels for the PersistentVolume. A PersistentVolumeClaim uses a
selectorto bind to a PersistentVolume by matching these labels.accessModesThe access mode of the PersistentVolume.
capacity.storageThe capacity of the volume.
csi.driverThe CSI driver. Set the value to
bmcpfsplugin.csi.alibabacloud.com.csi.volumeAttributesThe attributes of the CPFS volume.
-
filesystemId: The ID of the CPFS for LINGJUN file system. -
path: The directory to mount. The default value is/, which indicates the root directory of the CPFS file system. You can also specify a subdirectory, such as/dir. If the subdirectory does not exist, it is automatically created on mount.
csi.volumeHandleThe ID of the CPFS for LINGJUN file system.
-
-
PersistentVolumeClaim parameters
Parameter
Description
accessModesThe access mode requested by the PersistentVolumeClaim.
selectorThe selector used to find a matching PersistentVolume based on labels.
resources.requests.storageThe amount of storage requested by the PersistentVolumeClaim. The value cannot exceed the capacity of the PersistentVolume.
-
Run the create command:
kubectl create -f cpfs-pv-pvc.yamlConfirm that the PVC is bound to the PV.
kubectl get pvc cpfs-testExample output (a STATUS of Bound indicates success):
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE cpfs-test Bound cpfs-test 10Ti RWX <unset> <unset> 10s
Console
Log on to the ACS console.
On the Clusters page, click the name of the target cluster to go to its management page.
In the left-side navigation pane, choose , click Create, and complete the following configuration:
Parameter
Description
Example
PVC Type
Select CPFS.
CPFS
Name
A custom name for the PVC. For naming requirements, see the on-screen prompts.
cpfs-pvc
Allocation Mode
Select Existing Volume or Create Volume as needed.
Create Volume
CPFS Type
Select CPFS for Lingjun.
CPFS for Lingjun
Access Mode
Supports ReadWriteMany and ReadWriteOnce.
ReadWriteMany
File System ID
Configure the ID of the CPFS for Lingjun file system to mount.
bmcpfs-0115******13q5
View the created PV and PVC.
On the PersistentVolumeClaims page and the Volumes page, you can see the newly created PV and PVC. Confirm that they are bound.
VPC network
Applies to CPU pods, standard GPUs (such as T4 or A10), and Lingjun GPU access across compute-storage affinity domains.
Create a file named
cpfs-pv-pvc.yamlto define the PV and PVC:apiVersion: v1 kind: PersistentVolume metadata: name: cpfs-test labels: alicloud-pvname: cpfs-test # [PV label] Used by the PVC to bind to this PV spec: accessModes: - ReadWriteMany capacity: storage: 10Ti csi: driver: nasplugin.csi.alibabacloud.com volumeAttributes: mountProtocol: efc server: cpfs-***-vpc-***.cn-wulanchabu.cpfs.aliyuncs.com # Replace with your VPC mount point domain name path: / autoSwitch: "true" # Optional. Enables automatic VSC/VPC switching (recommended for Lingjun cross-AZ scenarios) volumeHandle: bmcpfs-***** # Replace with your CPFS for Lingjun file system ID --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: cpfs-test # [PVC name] Referenced by the Deployment spec: accessModes: - ReadWriteMany selector: matchLabels: alicloud-pvname: cpfs-test # [PVC selector] Must match the PV label resources: requests: storage: 10TiParameter description:
-
PersistentVolume parameters
Parameter
Description
labelsThe labels for the PersistentVolume. A PersistentVolumeClaim uses a
selectorto bind to a PersistentVolume by matching these labels.accessModesThe access mode of the PersistentVolume.
capacity.storageThe capacity of the volume.
csi.driverThe CSI driver. Set the value to
nasplugin.csi.alibabacloud.com.csi.volumeAttributesThe attributes of the CPFS volume.
-
mountProtocol: The mount protocol. Set the value toefc. -
server: The VPC mount target domain name of the CPFS file system. -
path: The directory to mount. The default value is/, which indicates the root directory of the CPFS file system. You can also specify a subdirectory, such as/dir. If the subdirectory does not exist, it is automatically created on mount.
csi.volumeHandleThe ID of the CPFS for LINGJUN file system.
-
-
PersistentVolumeClaim parameters
Parameter
Description
accessModesThe access mode requested by the PersistentVolumeClaim.
selectorThe selector used to find a matching PersistentVolume based on labels.
resources.requests.storageThe amount of storage requested by the PersistentVolumeClaim. The value cannot exceed the capacity of the PersistentVolume.
autoSwitch: An optional parameter. When set to"true", it enables automatic VSC/VPC switching. When Lingjun GPUs access across availability zones, the system automatically switches from RDMA to the VPC channel.
NoteThe throughput of the VPC channel is limited by network bandwidth, up to approximately 3.2 GB/s (25 Gbps), which is lower than a direct RDMA connection. For performance-sensitive scenarios, we recommend prioritizing a CPFS instance in the same compute-storage affinity domain.
-
Run the create command:
kubectl create -f cpfs-pv-pvc.yamlConfirm that the PVC is bound to the PV.
kubectl get pvc cpfs-testExample output (a STATUS of Bound indicates success):
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE cpfs-test Bound cpfs-test 10Ti RWX <unset> <unset> 10s
Step 2: Create an application and mount CPFS
Create a Deployment that references the PVC to mount the storage to a specific directory in the container. Choose the CLI or console method based on your preference.
kubectl
Create a file named
cpfs-test.yamlwith the following content.GPU application
The following YAML example creates a Deployment with two Pods. Both Pods use the
alibabacloud.com/compute-class: gpulabel to request GPU-based computing power and a PersistentVolumeClaim (PVC) namedcpfs-testto request storage resources. The mount path for both Pods is/data.NoteFor more information about specific GPU models, see Specify GPU models and driver versions for ACS GPU Pods.
apiVersion: apps/v1 kind: Deployment metadata: name: cpfs-test labels: app: cpfs-test spec: replicas: 2 selector: matchLabels: app: cpfs-test template: metadata: labels: app: cpfs-test # Specify the compute type as GPU. alibabacloud.com/compute-class: gpu # Specify the GPU model, for example, T4. alibabacloud.com/gpu-model-series: T4 alibabacloud.com/compute-qos: default spec: containers: - name: nginx image: registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest ports: - containerPort: 80 volumeMounts: - name: pvc-cpfs mountPath: /data volumes: - name: pvc-cpfs persistentVolumeClaim: claimName: cpfs-testCPU application
The following YAML example creates a Deployment with 2 pods, which request storage resources by using a PVC named
cpfs-testand are both mounted to the/datapath.apiVersion: apps/v1 kind: Deployment metadata: name: cpfs-test labels: app: cpfs-test spec: replicas: 2 selector: matchLabels: app: cpfs-test template: metadata: labels: app: cpfs-test spec: containers: - name: nginx image: registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest ports: - containerPort: 80 volumeMounts: - name: pvc-cpfs mountPath: /data volumes: - name: pvc-cpfs persistentVolumeClaim: claimName: cpfs-testCreate the Deployment and mount the CPFS volume.
kubectl create -f cpfs-test.yamlCheck the status of the Pods in the Deployment.
kubectl get pod | grep cpfs-testThe expected output shows that two Pods are in the 'Running' state.
cpfs-test-****-***a 1/1 Running 0 45s cpfs-test-****-***b 1/1 Running 0 45sCheck the mount path.
This command lists the contents of the mounted directory (/data), which contains data from the CPFS for LINGJUN file system. The directory is empty by default.
kubectl exec cpfs-test-****-***a -- ls /data
Console
In the left-side navigation pane of the cluster management page, choose .
On the Deployments page, click Create from Image.
Configure the parameters for the Deployment and click Create.
The key parameters are described below. You can keep the default values for other parameters. For more information, see Create a stateless application by using a Deployment.
GPU application
Section
Parameter
Description
Example
Basic Information
Application Name
Enter a unique name for the Deployment. The name must conform to the format requirements shown on the page.
cpfs-test
Replicas:
The number of Pod replicas for the Deployment.
2
Instance Type
Specifies the hardware resources for the Pods, such as CPU or a specific GPU model.
NoteFor more information about specific GPU models, see Specify GPU models and driver versions for ACS GPU Pods.
GPU, T4
Container
Image Name
The container image used to deploy the application.
registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest
Required Resources
The amount of GPU, vCPU, and memory resources to allocate to each Pod.
GPU: 1
CPU: 2 vCPU
Memory: 2 GiB
Volume
Click Add PVC and then configure the parameters.
Mount Source: Select the existing PersistentVolumeClaim.
Container Path: Enter the path within the container where the CPFS file system will be mounted.
Mount source: pvc-cpfs
Container path: /data
CPU application
Section
Parameter
Description
Example
Basic Information
Application Name
Enter a unique name for the Deployment. The name must conform to the format requirements shown on the page.
cpfs-test
Replicas:
The number of Pod replicas for the Deployment.
2
Instance Type
The compute type of the Pod.
CPU, General Purpose
Container
Image Name
The container image used to deploy the application.
registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest
Required Resources
The amount of vCPU and memory resources to allocate to each Pod.
CPU: 0.25 vCPU
Memory: 0.5 GiB
Volume
Click Add PVC and then configure the parameters.
Mount Source: Select the existing PersistentVolumeClaim.
Container Path: Enter the path within the container where the CPFS file system will be mounted.
Mount source: pvc-cpfs
Container path: /data
Check the application's status.
On the Deployments page, click the application name.
On the Pods tab, check that the Pods are in the 'Running' state.
Step 3: Verify the mount
The Deployment created in the previous example contains two Pods, both of which mount the same CPFS file system. You can verify its shared storage and persistent storage as follows:
Verify persistent storage by restarting the Deployment and then checking if the data persists in the newly created Pods.
-
Check the Pods.
kubectl get pod | grep cpfs-testExample output:
cpfs-test-****-***a 1/1 Running 0 45s cpfs-test-****-***b 1/1 Running 0 45s -
Verify shared storage.
-
Create a file in one of the Pods.
This example uses the Pod named
cpfs-test-****-***a:kubectl exec cpfs-test-****-***a -- touch /data/test.txt -
Check for the file in the other Pod.
This example uses the Pod named
cpfs-test-****-***b:kubectl exec cpfs-test-****-***b -- ls /dataThe output,
test.txt, confirms that storage is shared.test.txt
-
-
Verify persistent storage.
-
Restart the Deployment.
kubectl rollout restart deploy cpfs-test -
Check the Pod status and wait for the new Pods to be created.
kubectl get pod | grep cpfs-testExample output:
cpfs-test-****-***c 1/1 Running 0 78s cpfs-test-****-***d 1/1 Running 0 52s -
Verify that the data persists in the new Pod.
This example uses the Pod named
cpfs-test-****-***c:kubectl exec cpfs-test-****-***c -- ls /dataThis output confirms that data on the CPFS file system persists and is accessible from the new Pod's mount directory.
test.txt
-