Mount a CPFS for Lingjun file system on ACS

Updated at:
Copy as MD

Container Compute Service (ACS) supports mounting a CPFS for Lingjun file system. Multiple pods can mount it at the same time to share data.

Prerequisites

  • You have created an ACS cluster.

    For ACS, the CSI components (bmcpfs-csi-controller, bmcpfs-csi-node, and cnfs-nas-daemon) are managed by the ACS platform. You do not need to manage component versions or upgrades.
  • You have created a CPFS for Lingjun file system and recorded the file system ID. For more information, see Create a CPFS for Lingjun file system.

  • If a pod is scheduled to a node that does not support RDMA, or the scenario crosses compute-storage affinity domains, you must create a VPC mount point in advance and record the mount point domain name (in the format cpfs-***-vpc-***.<Region>.cpfs.aliyuncs.com). The VPC that the VPC mount point belongs to must be the same VPC as the ACS cluster. For more information, see Manage VPC mount points.

Step 1: Create a PV and a PVC

Choose the appropriate template based on your pod type:

  • Lingjun GPU pods in the same compute-storage affinity domain: Use "Method A: RDMA."

  • CPU pods, or Lingjun access across compute-storage affinity domains: Use "Method B: VPC mount point."

When an ACS Lingjun pod is in the same compute-storage affinity domain as CPFS, it accesses the file system through VSC + RDMA direct mount for the best performance. For CPU pods or Lingjun access across compute-storage affinity domains, access goes through a VPC mount point. For the concept and determination method for compute-storage affinity domains, see Overview of mount access.

RDMA network

kubectl CLI

  1. Create a file named cpfs-pv-pvc.yaml to define the PV and PVC:

    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: cpfs-test
      labels:
        alicloud-pvname: cpfs-test
    spec:
      accessModes:
      - ReadWriteMany
      capacity:
        storage: 10Ti
      csi:
        driver: bmcpfsplugin.csi.alibabacloud.com
        volumeAttributes:
          filesystemId: bmcpfs-*****
          path: /
        volumeHandle: bmcpfs-*****
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: cpfs-test
    spec:
      accessModes:
      - ReadWriteMany
      selector:
        matchLabels:
          alicloud-pvname: cpfs-test
      resources:
        requests:
          storage: 10Ti

    Parameter description:

    • PersistentVolume parameters

      Parameter

      Description

      labels

      The labels for the PersistentVolume. A PersistentVolumeClaim uses a selector to bind to a PersistentVolume by matching these labels.

      accessModes

      The access mode of the PersistentVolume.

      capacity.storage

      The capacity of the volume.

      csi.driver

      The CSI driver. Set the value to bmcpfsplugin.csi.alibabacloud.com.

      csi.volumeAttributes

      The attributes of the CPFS volume.

      • filesystemId: The ID of the CPFS for LINGJUN file system.

      • path: The directory to mount. The default value is /, which indicates the root directory of the CPFS file system. You can also specify a subdirectory, such as /dir. If the subdirectory does not exist, it is automatically created on mount.

      csi.volumeHandle

      The ID of the CPFS for LINGJUN file system.

    • PersistentVolumeClaim parameters

      Parameter

      Description

      accessModes

      The access mode requested by the PersistentVolumeClaim.

      selector

      The selector used to find a matching PersistentVolume based on labels.

      resources.requests.storage

      The amount of storage requested by the PersistentVolumeClaim. The value cannot exceed the capacity of the PersistentVolume.

  2. Run the create command:

    kubectl create -f cpfs-pv-pvc.yaml
  3. Confirm that the PVC is bound to the PV.

    kubectl get pvc cpfs-test

    Example output (a STATUS of Bound indicates success):

    NAME        STATUS   VOLUME           CAPACITY   ACCESS MODES   STORAGECLASS    VOLUMEATTRIBUTESCLASS   AGE
    cpfs-test   Bound    cpfs-test        10Ti       RWX            <unset>         <unset>                 10s

Console

  1. Log on to the ACS console.

  2. On the Clusters page, click the name of the target cluster to go to its management page.

  3. In the left-side navigation pane, choose Storage > PersistentVolumeClaims, click Create, and complete the following configuration:

    Parameter

    Description

    Example

    PVC Type

    Select CPFS.

    CPFS

    Name

    A custom name for the PVC. For naming requirements, see the on-screen prompts.

    cpfs-pvc

    Allocation Mode

    Select Existing Volume or Create Volume as needed.

    Create Volume

    CPFS Type

    Select CPFS for Lingjun.

    CPFS for Lingjun

    Access Mode

    Supports ReadWriteMany and ReadWriteOnce.

    ReadWriteMany

    File System ID

    Configure the ID of the CPFS for Lingjun file system to mount.

    bmcpfs-0115******13q5

  4. View the created PV and PVC.

    On the PersistentVolumeClaims page and the Volumes page, you can see the newly created PV and PVC. Confirm that they are bound.

VPC network

Applies to CPU pods, standard GPUs (such as T4 or A10), and Lingjun GPU access across compute-storage affinity domains.

  1. Create a file named cpfs-pv-pvc.yaml to define the PV and PVC:

    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: cpfs-test
      labels:
        alicloud-pvname: cpfs-test   # [PV label] Used by the PVC to bind to this PV
    spec:
      accessModes:
      - ReadWriteMany
      capacity:
        storage: 10Ti
      csi:
        driver: nasplugin.csi.alibabacloud.com
        volumeAttributes:
          mountProtocol: efc
          server: cpfs-***-vpc-***.cn-wulanchabu.cpfs.aliyuncs.com  # Replace with your VPC mount point domain name
          path: /
          autoSwitch: "true"    # Optional. Enables automatic VSC/VPC switching (recommended for Lingjun cross-AZ scenarios)
        volumeHandle: bmcpfs-*****   # Replace with your CPFS for Lingjun file system ID
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: cpfs-test      # [PVC name] Referenced by the Deployment
    spec:
      accessModes:
      - ReadWriteMany
      selector:
        matchLabels:
          alicloud-pvname: cpfs-test    # [PVC selector] Must match the PV label
      resources:
        requests:
          storage: 10Ti

    Parameter description:

    • PersistentVolume parameters

      Parameter

      Description

      labels

      The labels for the PersistentVolume. A PersistentVolumeClaim uses a selector to bind to a PersistentVolume by matching these labels.

      accessModes

      The access mode of the PersistentVolume.

      capacity.storage

      The capacity of the volume.

      csi.driver

      The CSI driver. Set the value to nasplugin.csi.alibabacloud.com.

      csi.volumeAttributes

      The attributes of the CPFS volume.

      • mountProtocol: The mount protocol. Set the value to efc.

      • server: The VPC mount target domain name of the CPFS file system.

      • path: The directory to mount. The default value is /, which indicates the root directory of the CPFS file system. You can also specify a subdirectory, such as /dir. If the subdirectory does not exist, it is automatically created on mount.

      csi.volumeHandle

      The ID of the CPFS for LINGJUN file system.

    • PersistentVolumeClaim parameters

      Parameter

      Description

      accessModes

      The access mode requested by the PersistentVolumeClaim.

      selector

      The selector used to find a matching PersistentVolume based on labels.

      resources.requests.storage

      The amount of storage requested by the PersistentVolumeClaim. The value cannot exceed the capacity of the PersistentVolume.

    • autoSwitch: An optional parameter. When set to "true", it enables automatic VSC/VPC switching. When Lingjun GPUs access across availability zones, the system automatically switches from RDMA to the VPC channel.

    Note

    The throughput of the VPC channel is limited by network bandwidth, up to approximately 3.2 GB/s (25 Gbps), which is lower than a direct RDMA connection. For performance-sensitive scenarios, we recommend prioritizing a CPFS instance in the same compute-storage affinity domain.

  2. Run the create command:

    kubectl create -f cpfs-pv-pvc.yaml
  3. Confirm that the PVC is bound to the PV.

    kubectl get pvc cpfs-test

    Example output (a STATUS of Bound indicates success):

    NAME        STATUS   VOLUME           CAPACITY   ACCESS MODES   STORAGECLASS    VOLUMEATTRIBUTESCLASS   AGE
    cpfs-test   Bound    cpfs-test        10Ti       RWX            <unset>         <unset>                 10s

Step 2: Create an application and mount CPFS

Create a Deployment that references the PVC to mount the storage to a specific directory in the container. Choose the CLI or console method based on your preference.

kubectl

  1. Create a file named cpfs-test.yaml with the following content.

    GPU application

    The following YAML example creates a Deployment with two Pods. Both Pods use the alibabacloud.com/compute-class: gpu label to request GPU-based computing power and a PersistentVolumeClaim (PVC) named cpfs-test to request storage resources. The mount path for both Pods is /data.

    Note

    For more information about specific GPU models, see Specify GPU models and driver versions for ACS GPU Pods.

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: cpfs-test
      labels:
        app: cpfs-test
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: cpfs-test
      template:
        metadata:
          labels:
            app: cpfs-test
            # Specify the compute type as GPU.
            alibabacloud.com/compute-class: gpu
            # Specify the GPU model, for example, T4.
            alibabacloud.com/gpu-model-series: T4
            alibabacloud.com/compute-qos: default
        spec:
          containers:
          - name: nginx
            image: registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest
            ports:
            - containerPort: 80
            volumeMounts:
              - name: pvc-cpfs
                mountPath: /data
          volumes:
            - name: pvc-cpfs
              persistentVolumeClaim:
                claimName: cpfs-test

    CPU application

    The following YAML example creates a Deployment with 2 pods, which request storage resources by using a PVC named cpfs-test and are both mounted to the /data path.

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: cpfs-test
      labels:
        app: cpfs-test
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: cpfs-test
      template:
        metadata:
          labels:
            app: cpfs-test
        spec:
          containers:
          - name: nginx
            image: registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest
            ports:
            - containerPort: 80
            volumeMounts:
              - name: pvc-cpfs
                mountPath: /data
          volumes:
            - name: pvc-cpfs
              persistentVolumeClaim:
                claimName: cpfs-test
  2. Create the Deployment and mount the CPFS volume.

    kubectl create -f cpfs-test.yaml
  3. Check the status of the Pods in the Deployment.

    kubectl get pod | grep cpfs-test

    The expected output shows that two Pods are in the 'Running' state.

    cpfs-test-****-***a   1/1     Running   0          45s
    cpfs-test-****-***b   1/1     Running   0          45s
  4. Check the mount path.

    This command lists the contents of the mounted directory (/data), which contains data from the CPFS for LINGJUN file system. The directory is empty by default.

    kubectl exec cpfs-test-****-***a -- ls /data

Console

  1. In the left-side navigation pane of the cluster management page, choose Workloads > Deployments.

  2. On the Deployments page, click Create from Image.

  3. Configure the parameters for the Deployment and click Create.

    The key parameters are described below. You can keep the default values for other parameters. For more information, see Create a stateless application by using a Deployment.

    GPU application

    Section

    Parameter

    Description

    Example

    Basic Information

    Application Name

    Enter a unique name for the Deployment. The name must conform to the format requirements shown on the page.

    cpfs-test

    Replicas:

    The number of Pod replicas for the Deployment.

    2

    Instance Type

    Specifies the hardware resources for the Pods, such as CPU or a specific GPU model.

    Note

    For more information about specific GPU models, see Specify GPU models and driver versions for ACS GPU Pods.

    GPU, T4

    Container

    Image Name

    The container image used to deploy the application.

    registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest

    Required Resources

    The amount of GPU, vCPU, and memory resources to allocate to each Pod.

    • GPU: 1

    • CPU: 2 vCPU

    • Memory: 2 GiB

    Volume

    Click Add PVC and then configure the parameters.

    • Mount Source: Select the existing PersistentVolumeClaim.

    • Container Path: Enter the path within the container where the CPFS file system will be mounted.

    • Mount source: pvc-cpfs

    • Container path: /data

    CPU application

    Section

    Parameter

    Description

    Example

    Basic Information

    Application Name

    Enter a unique name for the Deployment. The name must conform to the format requirements shown on the page.

    cpfs-test

    Replicas:

    The number of Pod replicas for the Deployment.

    2

    Instance Type

    The compute type of the Pod.

    CPU, General Purpose

    Container

    Image Name

    The container image used to deploy the application.

    registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest

    Required Resources

    The amount of vCPU and memory resources to allocate to each Pod.

    • CPU: 0.25 vCPU

    • Memory: 0.5 GiB

    Volume

    Click Add PVC and then configure the parameters.

    • Mount Source: Select the existing PersistentVolumeClaim.

    • Container Path: Enter the path within the container where the CPFS file system will be mounted.

    • Mount source: pvc-cpfs

    • Container path: /data

  4. Check the application's status.

    1. On the Deployments page, click the application name.

    2. On the Pods tab, check that the Pods are in the 'Running' state.

Step 3: Verify the mount

The Deployment created in the previous example contains two Pods, both of which mount the same CPFS file system. You can verify its shared storage and persistent storage as follows:

Verify persistent storage by restarting the Deployment and then checking if the data persists in the newly created Pods.

  1. Check the Pods.

    kubectl get pod | grep cpfs-test

    Example output:

    cpfs-test-****-***a   1/1     Running   0          45s
    cpfs-test-****-***b   1/1     Running   0          45s
  2. Verify shared storage.

    1. Create a file in one of the Pods.

      This example uses the Pod named cpfs-test-****-***a:

      kubectl exec cpfs-test-****-***a -- touch /data/test.txt
    2. Check for the file in the other Pod.

      This example uses the Pod named cpfs-test-****-***b:

      kubectl exec cpfs-test-****-***b -- ls /data

      The output, test.txt, confirms that storage is shared.

      test.txt
  3. Verify persistent storage.

    1. Restart the Deployment.

      kubectl rollout restart deploy cpfs-test
    2. Check the Pod status and wait for the new Pods to be created.

      kubectl get pod | grep cpfs-test

      Example output:

      cpfs-test-****-***c   1/1     Running   0          78s
      cpfs-test-****-***d   1/1     Running   0          52s
    3. Verify that the data persists in the new Pod.

      This example uses the Pod named cpfs-test-****-***c:

      kubectl exec cpfs-test-****-***c -- ls /data

      This output confirms that data on the CPFS file system persists and is accessible from the new Pod's mount directory.

      test.txt