Mount a CPFS for Lingjun file system on ACS

Updated at:
Copy as MD

Container Compute Service (ACS) supports mounting a CPFS for Lingjun file system. Multiple pods can mount it at the same time to share data.

Prerequisites

  • You have created an ACS cluster.

    For ACS, the CSI components (bmcpfs-csi-controller, bmcpfs-csi-node, and cnfs-nas-daemon) are managed by the ACS platform. You do not need to manage component versions or upgrades.
  • You have created a CPFS for Lingjun file system and recorded the file system ID. For more information, see Create a CPFS for Lingjun file system.

  • If a pod is scheduled to a node that does not support RDMA, or the scenario crosses compute-storage affinity domains, you must create a VPC mount point in advance and record the mount point domain name (in the format cpfs-***-vpc-***.<Region>.cpfs.aliyuncs.com). The VPC that the VPC mount point belongs to must be the same VPC as the ACS cluster. For more information, see Manage VPC mount points.

Step 1: Create a PV and a PVC

Choose the appropriate template based on your pod type:

  • Lingjun GPU pods in the same compute-storage affinity domain: Use "Method A: RDMA."

  • CPU pods, or Lingjun access across compute-storage affinity domains: Use "Method B: VPC mount point."

When an ACS Lingjun pod is in the same compute-storage affinity domain as CPFS, it accesses the file system through VSC + RDMA direct mount for the best performance. For CPU pods or Lingjun access across compute-storage affinity domains, access goes through a VPC mount point. For the concept and determination method for compute-storage affinity domains, see Overview of mount access.

RDMA network

kubectl CLI

  1. Create a file named cpfs-pv-pvc.yaml to define the PV and PVC:

    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: cpfs-test
      labels:
        alicloud-pvname: cpfs-test
    spec:
      accessModes:
      - ReadWriteMany
      capacity:
        storage: 10Ti
      csi:
        driver: bmcpfsplugin.csi.alibabacloud.com
        volumeAttributes:
          mountpointAutoSwitch: "true" # Enables automatic switchover between the RDMA mount address and the VPC mount address
          vpcMountTarget: cpfs-***-vpc-***.<Region>.cpfs.aliyuncs.com
          filesystemId: bmcpfs-*****
          path: /
        volumeHandle: bmcpfs-*****
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: cpfs-test
    spec:
      accessModes:
      - ReadWriteMany
      selector:
        matchLabels:
          alicloud-pvname: cpfs-test
      resources:
        requests:
          storage: 10Ti

    Parameter description:

    • PV parameters:

      Parameter

      Description

      labels

      Sets a label so that the PVC can use selector to match and bind the PV.

      accessModes

      The access mode of the PV.

      capacity.storage

      The declared capacity of the volume.

      csi.driver

      The driver type. Set the value to bmcpfsplugin.csi.alibabacloud.com.

      csi.volumeAttributes

      The attributes of the CPFS volume. filesystemId: the ID of the CPFS for Lingjun file system. path: the default value / mounts the root directory of the CPFS file system. Optionally, specify a subdirectory, such as /dir. If the subdirectory does not exist, it is automatically created when the volume is mounted.

      csi.volumeHandle

      The ID of the CPFS for Lingjun file system.

    • PVC parameters:

      Parameter

      Description

      accessModes

      The access mode that the PVC requests from the PV.

      selector

      Matches and binds the PV based on the label on the PV.

      resources.requests.storage

      The storage capacity allocated to the pod. The value must not exceed the capacity of the PV.

  2. Run the create command:

    kubectl create -f cpfs-pv-pvc.yaml
  3. Confirm that the PVC is bound to the PV.

    kubectl get pvc cpfs-test

    Example output (a STATUS of Bound indicates success):

    NAME        STATUS   VOLUME           CAPACITY   ACCESS MODES   STORAGECLASS    VOLUMEATTRIBUTESCLASS   AGE
    cpfs-test   Bound    cpfs-test        10Ti       RWX            <unset>         <unset>                 10s

Console

  1. Log on to the ACS console.

  2. On the Clusters page, click the name of the target cluster to go to its management page.

  3. In the left-side navigation pane, choose Storage > PersistentVolumeClaims, click Create, and complete the following configuration:

    Parameter

    Description

    Example

    PVC Type

    Select CPFS.

    CPFS

    Name

    A custom name for the PVC. For naming requirements, see the on-screen prompts.

    cpfs-pvc

    Allocation Mode

    Select Existing Volume or Create Volume as needed.

    Create Volume

    CPFS Type

    Select CPFS for Lingjun.

    CPFS for Lingjun

    Access Mode

    Supports ReadWriteMany and ReadWriteOnce.

    ReadWriteMany

    File System ID

    Configure the ID of the CPFS for Lingjun file system to mount.

    bmcpfs-0115******13q5

  4. View the created PV and PVC.

    On the PersistentVolumeClaims page and the Volumes page, you can see the newly created PV and PVC. Confirm that they are bound.

VPC network

Applies to CPU pods, standard GPUs (such as T4 or A10), and Lingjun GPU access across compute-storage affinity domains.

  1. Create a file named cpfs-pv-pvc.yaml to define the PV and PVC:

    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: cpfs-test
      labels:
        alicloud-pvname: cpfs-test   # [PV label] Used by the PVC to bind to this PV
    spec:
      accessModes:
      - ReadWriteMany
      capacity:
        storage: 10Ti
      csi:
        driver: nasplugin.csi.alibabacloud.com
        volumeAttributes:
          mountProtocol: efc
          server: cpfs-***-vpc-***.cn-wulanchabu.cpfs.aliyuncs.com  # Replace with your VPC mount point domain name
          path: /
          autoSwitch: "true"    # Optional. Enables automatic VSC/VPC switching (recommended for Lingjun cross-AZ scenarios)
        volumeHandle: bmcpfs-*****   # Replace with your CPFS for Lingjun file system ID
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: cpfs-test      # [PVC name] Referenced by the Deployment
    spec:
      accessModes:
      - ReadWriteMany
      selector:
        matchLabels:
          alicloud-pvname: cpfs-test    # [PVC selector] Must match the PV label
      resources:
        requests:
          storage: 10Ti

    Parameter description:

    • PV parameters:

      Parameter

      Description

      labels

      Sets a label so that the PVC can use selector to match and bind the PV.

      accessModes

      The access mode of the PV.

      capacity.storage

      The declared capacity of the volume.

      csi.driver

      The driver type. Set the value to bmcpfsplugin.csi.alibabacloud.com.

      csi.volumeAttributes

      The attributes of the CPFS volume. filesystemId: the ID of the CPFS for Lingjun file system. path: the default value / mounts the root directory of the CPFS file system. Optionally, specify a subdirectory, such as /dir. If the subdirectory does not exist, it is automatically created when the volume is mounted.

      csi.volumeHandle

      The ID of the CPFS for Lingjun file system.

    • PVC parameters:

      Parameter

      Description

      accessModes

      The access mode that the PVC requests from the PV.

      selector

      Matches and binds the PV based on the label on the PV.

      resources.requests.storage

      The storage capacity allocated to the pod. The value must not exceed the capacity of the PV.

    • autoSwitch: An optional parameter. When set to "true", it enables automatic VSC/VPC switching. When Lingjun GPUs access across availability zones, the system automatically switches from RDMA to the VPC channel.

    Note

    The throughput of the VPC channel is limited by network bandwidth, up to approximately 3.2 GB/s (25 Gbps), which is lower than a direct RDMA connection. For performance-sensitive scenarios, we recommend prioritizing a CPFS instance in the same compute-storage affinity domain.

  2. Run the create command:

    kubectl create -f cpfs-pv-pvc.yaml
  3. Confirm that the PVC is bound to the PV.

    kubectl get pvc cpfs-test

    Example output (a STATUS of Bound indicates success):

    NAME        STATUS   VOLUME           CAPACITY   ACCESS MODES   STORAGECLASS    VOLUMEATTRIBUTESCLASS   AGE
    cpfs-test   Bound    cpfs-test        10Ti       RWX            <unset>         <unset>                 10s

Step 2: Create an application and mount CPFS

Create a Deployment that references the PVC to mount the storage to a specific directory in the container. Choose the CLI or console method based on your preference.

kubectl

  1. Create a file named cpfs-test.yaml with the following content.

    GPU application

    The following YAML example creates a Deployment with two Pods. Both Pods use the alibabacloud.com/compute-class: gpu label to request GPU-based computing power and a PersistentVolumeClaim (PVC) named cpfs-test to request storage resources. The mount path for both Pods is /data.

    Note

    For more information about specific GPU models, see Specify GPU models and driver versions for ACS GPU Pods.

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: cpfs-test
      labels:
        app: cpfs-test
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: cpfs-test
      template:
        metadata:
          labels:
            app: cpfs-test
            # Specify the compute type as GPU.
            alibabacloud.com/compute-class: gpu
            # Specify the GPU model, for example, T4.
            alibabacloud.com/gpu-model-series: T4
            alibabacloud.com/compute-qos: default
        spec:
          containers:
          - name: nginx
            image: registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest
            ports:
            - containerPort: 80
            volumeMounts:
              - name: pvc-cpfs
                mountPath: /data
          volumes:
            - name: pvc-cpfs
              persistentVolumeClaim:
                claimName: cpfs-test

    CPU application

    The following YAML example creates a Deployment with 2 pods, which request storage resources by using a PVC named cpfs-test and are both mounted to the /data path.

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: cpfs-test
      labels:
        app: cpfs-test
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: cpfs-test
      template:
        metadata:
          labels:
            app: cpfs-test
        spec:
          containers:
          - name: nginx
            image: registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest
            ports:
            - containerPort: 80
            volumeMounts:
              - name: pvc-cpfs
                mountPath: /data
          volumes:
            - name: pvc-cpfs
              persistentVolumeClaim:
                claimName: cpfs-test
  2. Create the Deployment and mount the CPFS volume.

    kubectl create -f cpfs-test.yaml
  3. Check the status of the Pods in the Deployment.

    kubectl get pod | grep cpfs-test

    The expected output shows that two Pods are in the 'Running' state.

    cpfs-test-****-***a   1/1     Running   0          45s
    cpfs-test-****-***b   1/1     Running   0          45s
  4. Check the mount path.

    This command lists the contents of the mounted directory (/data), which contains data from the CPFS for LINGJUN file system. The directory is empty by default.

    kubectl exec cpfs-test-****-***a -- ls /data

Console

  1. In the left-side navigation pane of the cluster management page, choose Workloads > Deployments.

  2. On the Deployments page, click Create from Image.

  3. Configure the parameters for the Deployment and click Create.

    The key parameters are described below. You can keep the default values for other parameters. For more information, see Create a stateless application by using a Deployment.

    GPU application

    Section

    Parameter

    Description

    Example

    Basic Information

    Application Name

    Enter a unique name for the Deployment. The name must conform to the format requirements shown on the page.

    cpfs-test

    Replicas:

    The number of Pod replicas for the Deployment.

    2

    Instance Type

    Specifies the hardware resources for the Pods, such as CPU or a specific GPU model.

    GPU, T4

    Container

    Image Name

    The container image used to deploy the application.

    registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest

    Required Resources

    The amount of GPU, vCPU, and memory resources to allocate to each Pod.

    • GPU: 1

    • CPU: 2 vCPU

    • Memory: 2 GiB

    Volume

    Click Add PVC and then configure the parameters.

    • Mount Source: Select the existing PersistentVolumeClaim.

    • Container Path: Enter the path within the container where the CPFS file system will be mounted.

    • Mount source: pvc-cpfs

    • Container path: /data

    CPU application

    Section

    Parameter

    Description

    Example

    Basic Information

    Application Name

    Enter a unique name for the Deployment. The name must conform to the format requirements shown on the page.

    cpfs-test

    Replicas:

    The number of Pod replicas for the Deployment.

    2

    Instance Type

    The compute type of the Pod.

    CPU, General Purpose

    Container

    Image Name

    The container image used to deploy the application.

    registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest

    Required Resources

    The amount of vCPU and memory resources to allocate to each Pod.

    • CPU: 0.25 vCPU

    • Memory: 0.5 GiB

    Volume

    Click Add PVC and then configure the parameters.

    • Mount Source: Select the existing PersistentVolumeClaim.

    • Container Path: Enter the path within the container where the CPFS file system will be mounted.

    • Mount source: pvc-cpfs

    • Container path: /data

  4. Check the application's status.

    1. On the Deployments page, click the application name.

    2. On the Pods tab, check that the Pods are in the 'Running' state.

Verify the shared storage and persistent storage of CPFS

The Deployment created in the preceding example contains two pods that mount the same CPFS file system. Verify the storage capabilities in the following ways:

  • Create a file in one pod and then view the file in the other pod to verify shared storage.

  • Restart the Deployment and then check whether the data in the file system still exists in the new pods to verify persistent storage.

  1. View the pod information.

    kubectl get pod | grep cpfs-test

    The following output is returned:

    cpfs-test-****-***a   1/1     Running   0          45s
    cpfs-test-****-***b   1/1     Running   0          45s
  2. Verify shared storage.

  3. Create a file in one pod.

    The following example uses the pod named cpfs-test-****-***a:

    kubectl exec cpfs-test-****-***a -- touch /data/test.txt
  4. View the file in the other pod.

    The following example uses the pod named cpfs-test-****-***b:

    kubectl exec cpfs-test-****-***b -- ls /data

    The following output is returned. The new file test.txt is shared.

    test.txt
  5. Verify persistent storage.

  6. Restart the Deployment to recreate its pods.

    kubectl rollout restart deploy cpfs-test
  7. View the pods and wait until the new pods are created.

    kubectl get pod | grep cpfs-test

    The following output is returned:

    cpfs-test-****-***c   1/1     Running   0          78s
    cpfs-test-****-***d   1/1     Running   0          52s
  8. Check whether the data in the file system still exists in a new pod.

    The following example uses the pod named cpfs-test-****-***c:

    kubectl exec cpfs-test-****-***c -- ls /data

    The following output is returned. The data in the CPFS file system still exists and is available in the mount directory of the new pod.

    test.txt