Mount OSS buckets using ossfs 1.0 dynamic volumes

更新时间:
复制 MD 格式

ossfs 1.0 supports dynamic provisioning, which automatically creates a persistent volume (PV) and mounts an OSS Bucket based on a StorageClass and a PersistentVolumeClaim (PVC). It eliminates manual PV configuration, greatly simplifies storage management, and is ideal for multi-tenant environments or scenarios requiring frequent, on-demand storage provisioning.

Requirements

Your cluster and CSI components (csi-plugin and csi-provisioner) must meet the following requirements:

  • For authentication using RAM Roles for Service Accounts (RRSA): Your cluster must be version 1.26 or later, and the CSI version must be v1.30.4 or later.

    If you used the RRSA feature in a version earlier than v1.30.4, refer to [Product Change] CSI ossfs version upgrade and mount process optimization to add the required RAM Role authorization configuration.
  • For authentication using an AccessKey: To ensure mount stability, we recommend CSI version v1.18.8.45 or later.

To upgrade your cluster, see Manually upgrade a cluster. To upgrade the CSI components, see Upgrade CSI components.

Starting with CSI v1.30.4-*, mounting OSS static volumes depends on the csi-provisioner component.

Step 1: Choose an authentication method

To ensure that your cluster can access OSS Bucket resources securely and compliantly, you must first configure an authentication method.

  • RRSA authentication: Dynamically grants Pods temporary, automatically rotating RAM roles to enable fine-grained, application-level permission isolation. This method is more secure.

  • AccessKey authentication: Stores static, long-term keys in a Secret. This method is simpler to configure but less secure.

Important
  • For clusters running version 1.26 or later, we recommend RRSA authentication to prevent workload restarts and ossfs remounts caused by AccessKey rotation.

  • This example assumes that the cluster and the OSS Bucket are in the same Alibaba Cloud account. To mount an OSS Bucket that belongs to another account, we recommend using RRSA authentication.

RRSA

1. Enable RRSA for your cluster

  1. On the ACK Clusters page, click the name of your cluster. In the left navigation pane, click Cluster Information.

  2. On the Basic Information tab, in the Security and Auditing section, click Enabled to the right of RRSA OIDC. Follow the on-screen instructions to enable RRSA during off-peak hours.

    When the cluster status changes from Updating to Running, RRSA is enabled.

    Important

    After you enable RRSA, the maximum validity period of newly created ServiceAccount tokens in the cluster is limited to 12 hours.

2. Create and authorize a RAM role

Create a RAM role that your Pods can assume for RRSA-authenticated access to the OSS storage volume.

View the steps

  1. Create a RAM role.

    1. Go to the Create Role page on the RAM Console. For Principal Type, select IdP. Then, click Switch to Policy Editor to open the Visual Editor.

    2. For Principal, select IdP. Click Edit and configure the parameters as described in the following table.

      Configure the main parameters and keep the default values for other parameters. For more information, see Create a RAM role for an OIDC identity provider.

      Parameter

      Description

      IdP Type

      OIDC.

      IdP

      Select ack-rrsa-<cluster_id>, where <cluster_id> is the ID of your cluster.

      Condition

      Manually add oidc:sub.

      • Condition key: Select oidc:sub.

      • Operator: Select StringEquals.

      • Condition Value: Enter system:serviceaccount:ack-csi-fuse:csi-fuse-ossfs.

      Role Name

      For example, enter demo-role-for-rrsa.

  2. Create an access policy.

    This example follows the principle of least privilege. Create a custom policy to grant access to the target OSS Bucket. You can grant read-only or read/write permissions.

    1. Go to the Create Policy page on the RAM Console, switch to the JSON Editor, and configure the policy script.

      If you already have a RAM role with OSS permissions, you can reuse it by modifying its trust policy. For more information, see Use an existing RAM role and grant permissions.
      Read-only policy
      Replace <myBucketName> with the actual bucket name.
      {
          "Statement": [
              {
                  "Action": [
                      "oss:Get*",
                      "oss:List*"
                  ],
                  "Effect": "Allow",
                  "Resource": [
                      "acs:oss:*:*:<myBucketName>",
                      "acs:oss:*:*:<myBucketName>/*"
                  ]
              }
          ],
          "Version": "1"
      }
      OSS read/write policy
      Replace <myBucketName> with the actual bucket name.
      {
          "Statement": [
              {
                  "Action": "oss:*",
                  "Effect": "Allow",
                  "Resource": [
                      "acs:oss:*:*:<myBucketName>",
                      "acs:oss:*:*:<myBucketName>/*"
                  ]
              }
          ],
          "Version": "1"
      }
    2. (Optional) If you use a specific CMK ID managed by KMS to encrypt OSS objects, you must also grant KMS permissions to the role. For more information, see Use a specific CMK ID managed by KMS to encrypt data.

  3. Attach the access policy to the RAM role.

    1. Go to the Roles page on the RAM Console, find the role you created, and in the Actions column, click Attach Policy.

    2. In the Policies section, search for and select the access policy you created.

AccessKey

Create a RAM user with permissions to access the target OSS Bucket, and then obtain an AccessKey for that user.

  1. Create a RAM user. You can skip this step if you already have a RAM user.

    Go to the Create User page on the RAM Console and follow the on-screen instructions to create a RAM user. You must specify parameters such as the logon name and password.

  2. Create an access policy.

    This example follows the principle of least privilege. Create a custom policy to grant access to the target OSS Bucket. You can grant read-only or read/write permissions.

    1. Go to the Create Policy page on the RAM Console, switch to the JSON Editor, and configure the policy script.

      Read-only policy

      Replace <myBucketName> with the actual bucket name.
      {
          "Statement": [
              {
                  "Action": [
                      "oss:Get*",
                      "oss:List*"
                  ],
                  "Effect": "Allow",
                  "Resource": [
                      "acs:oss:*:*:<myBucketName>",
                      "acs:oss:*:*:<myBucketName>/*"
                  ]
              }
          ],
          "Version": "1"
      }

      OSS read/write policy

      Replace <myBucketName> with the actual bucket name.
      {
          "Statement": [
              {
                  "Action": "oss:*",
                  "Effect": "Allow",
                  "Resource": [
                      "acs:oss:*:*:<myBucketName>",
                      "acs:oss:*:*:<myBucketName>/*"
                  ]
              }
          ],
          "Version": "1"
      }

      If you want to create a PV in the console, you also need the oss:ListBuckets permission.

      {
        "Effect": "Allow",
        "Action": "oss:ListBuckets",
        "Resource": "*"
      }
    2. (Optional) If you use a specific CMK ID managed by KMS to encrypt OSS objects, you must also grant KMS permissions to the RAM user. For more information, see Use a specific CMK ID managed by KMS to encrypt data.

  3. Attach the access policy to the RAM user.

    1. Go to the Users page on the RAM Console, find the user you created, and in the Actions column, click Add Permissions.

    2. In the Policies section, search for and select the access policy you created.

  4. The AccessKey you create will be stored as a Secret for the PV.

    1. Go to the Users page on the RAM Console, click the name of your RAM user, and then in the AccessKey section, click Create AccessKey.

    2. Follow the on-screen instructions to create an AccessKey. Copy and securely store the AccessKey ID and AccessKey Secret.

Step 2: Create a StorageClass

Create a StorageClass to define a template for provisioning persistent volumes.

RRSA method

  1. Create a file named sc-oss.yaml.

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: sc-oss
    parameters:
      # Replace with the name of your OSS bucket.
      bucket: bucket  
      # The mount path relative to the bucket's root directory. You can specify a subdirectory.
      path: /
      # The endpoint of the region where the bucket is located.
      url:  "http://oss-cn-hangzhou-internal.aliyuncs.com"  
      # Use the RRSA method for authentication.
      authType: rrsa
      # The RAM role that you created or modified.
      roleName: demo-role-for-rrsa
      # Custom parameters.
      otherOpts: "-o umask=022 -o max_stat_cache_size=100000 -o allow_other"
      # The volume access mode.
      volumeAs: sharepath
    # Fixed value for the Alibaba Cloud OSS CSI plug-in.
    provisioner: ossplugin.csi.alibabacloud.com
    # The reclaim policy for the dynamically provisioned persistent volume.
    reclaimPolicy: Retain
    # The volume binding mode.
    volumeBindingMode: Immediate

    Parameter

    Description

    bucket

    The OSS bucket to mount.

    path

    The version of the CSI component must be v1.14.8.32-c77e277b-aliyun or later.

    The mount path relative to the root directory of the bucket. The default value is /, which mounts the entire bucket.

    If the ossfs version is earlier than 1.91, the specified path must exist in the OSS bucket. For more information, see New features in ossfs 1.91 and later.

    url

    The endpoint of the OSS Bucket to mount.

    • Use an internal endpoint if the mount node and the bucket are in the same region, or if a VPC connection is established.

    • Use a public endpoint if the mount node and the bucket are in different regions.

    The following are common formats for different access ports:

    • Internal network format: http://oss-{{regionName}}-internal.aliyuncs.com or https://oss-{{regionName}}-internal.aliyuncs.com.

      The internal endpoint format vpc100-oss-{{regionName}}.aliyuncs.com is deprecated. Switch to the new format promptly.
    • Public network format: http://oss-{{regionName}}.aliyuncs.com or https://oss-{{regionName}}.aliyuncs.com.

    authType

    Set this parameter to rrsa to use RRSA for authentication.

    roleName

    The RAM role that you created or modified.

    To configure different permissions for different StorageClasses, you can create multiple RAM roles and specify different roleName values in each StorageClass.

    otherOpts

    Custom parameters for the OSS persistent volume, in the format of -o *** -o ***. For example: -o umask=022 -o max_stat_cache_size=100000 -o allow_other.

    Expand for details

    • umask: Sets the permission mask for files and directories.

      For example, umask=022 changes the file permissions to 755. This resolves the permission issue for files that are uploaded by using other methods, such as SDKs or the OSS console, which have a default permission of 640. We recommend that you configure this parameter for read/write splitting or multi-user access.

    • max_stat_cache_size: Sets the upper limit of metadata cache entries, for example, 100000. Metadata is cached in memory to improve the performance of operations such as ls and stat.

      However, this cache cannot immediately detect file modifications made through the OSS console, SDKs, or ossutil. This may cause applications to read inconsistent data. If you have strict data consistency requirements, set this parameter to 0 to disable the cache, or reduce the cache expiration time by using the stat_cache_expire parameter. This, however, may reduce read performance.

    • allow_other: Allows users other than the user who mounted the volume to access files and directories in the mount path. This is suitable for multi-user shared environments where non-mounting users also need to access data.

    For more information about optional parameters, see Mount options and Best practices for configuring ossfs 1.0.

    provisioner

    The driver type. This value is fixed at ossplugin.csi.alibabacloud.com when using the Alibaba Cloud OSS CSI plug-in.

    reclaimPolicy

    The reclaim policy for dynamically provisioned persistent volumes. OSS persistent volumes support only Retain, which means that when you delete a PVC, the corresponding persistent volume and the data in the OSS bucket are not deleted.

    volumeBindingMode

    The volume binding mode.

    OSS persistent volumes do not require availability zone-based node affinity. You can use the default value Immediate.

    volumeAs

    The volume access mode. The default value is sharepath. Valid values:

    • sharepath: All persistent volumes share the same mount path. Data is stored at <bucket>:<path>/.

    • subpath: Creates a dedicated subdirectory for each persistent volume to isolate data. The data is stored at <bucket>:<path>/<pv-name>/.

      This parameter takes effect only when the CSI component version is 1.31.3 or later.

    sigVersion

    The signature version for requests sent to the OSS server.

  2. Create the StorageClass.

    kubectl apply -f sc-oss.yaml

AccessKey method

kubectl

1. Create a StorageClass

  1. Create a Secret in the same namespace as your application.

    Replace akId and akSecret with your AccessKey ID and AccessKey Secret.

    kubectl create secret generic oss-secret --from-literal='akId=<yourAccessKey ID>' --from-literal='akSecret=<yourAccessKey Secret>'
  2. Create the StorageClass.

    1. Create a file named sc-oss.yaml.

      apiVersion: storage.k8s.io/v1
      kind: StorageClass
      metadata:
        name: sc-oss
      parameters:
        # Replace with the name of your OSS bucket.
        bucket: bucket
        # The mount path relative to the bucket's root directory. You can specify a subdirectory.
        path: /
        # The endpoint of the region where the bucket is located.
        url: "http://oss-cn-hangzhou-internal.aliyuncs.com" 
        # The name of the Secret that stores the AccessKey information.
        csi.storage.k8s.io/node-publish-secret-name: oss-secret
        # The namespace where the Secret that stores the AccessKey information is located.
        csi.storage.k8s.io/node-publish-secret-namespace: default
        # Custom parameters.
        otherOpts: "-o umask=022 -o max_stat_cache_size=100000 -o allow_other"
      # Fixed value for the Alibaba Cloud OSS CSI plug-in.
      provisioner: ossplugin.csi.alibabacloud.com
      # The reclaim policy for the dynamically provisioned persistent volume.
      reclaimPolicy: Retain
      # The volume binding mode.
      volumeBindingMode: Immediate
      

      Parameter

      Description

      name

      The name of the StorageClass.

      bucket

      The OSS bucket to mount.

      path

      The version of the CSI component must be v1.14.8.32-c77e277b-aliyun or later.

      The mount path relative to the root directory of the bucket. The default value is /, which mounts the entire bucket.

      If the ossfs version is earlier than 1.91, the specified path must exist in the OSS bucket. For more information, see New features in ossfs 1.91 and later.

      url

      The endpoint of the OSS Bucket to mount.

      • Use an internal endpoint if the mount node and the bucket are in the same region, or if a VPC connection is established.

      • Use a public endpoint if the mount node and the bucket are in different regions.

      The following are common formats for different access ports:

      • Internal network format: http://oss-{{regionName}}-internal.aliyuncs.com or https://oss-{{regionName}}-internal.aliyuncs.com.

        The internal endpoint format vpc100-oss-{{regionName}}.aliyuncs.com is deprecated. Switch to the new format promptly.
      • Public network format: http://oss-{{regionName}}.aliyuncs.com or https://oss-{{regionName}}.aliyuncs.com.

      csi.storage.k8s.io/node-publish-secret-name

      The name of the Secret that stores the AccessKey information.

      csi.storage.k8s.io/node-publish-secret-namespace

      The namespace where the Secret that stores the AccessKey information is located.

      otherOpts

      Custom parameters for the OSS persistent volume, in the format of -o *** -o ***. For example: -o umask=022 -o max_stat_cache_size=100000 -o allow_other.

      Expand for details

      • umask: Sets the permission mask for files and directories.

        For example, umask=022 changes the file permissions to 755. This resolves the permission issue for files that are uploaded by using other methods, such as SDKs or the OSS console, which have a default permission of 640. We recommend that you configure this parameter for read/write splitting or multi-user access.

      • max_stat_cache_size: Sets the upper limit of metadata cache entries, for example, 100000. Metadata is cached in memory to improve the performance of operations such as ls and stat.

        However, this cache cannot immediately detect file modifications made through the OSS console, SDKs, or ossutil. This may cause applications to read inconsistent data. If you have strict data consistency requirements, set this parameter to 0 to disable the cache, or reduce the cache expiration time by using the stat_cache_expire parameter. This, however, may reduce read performance.

      • allow_other: Allows users other than the user who mounted the volume to access files and directories in the mount path. This is suitable for multi-user shared environments where non-mounting users also need to access data.

      For more information about optional parameters, see Mount options and Best practices for configuring ossfs 1.0.

      provisioner

      The driver type. This value is fixed at ossplugin.csi.alibabacloud.com when using the Alibaba Cloud OSS CSI plug-in.

      reclaimPolicy

      The reclaim policy for dynamically provisioned persistent volumes. OSS persistent volumes support only Retain, which means that when you delete a PVC, the corresponding persistent volume and the data in the OSS bucket are not deleted.

      volumeBindingMode

      The volume binding mode.

      OSS persistent volumes do not require availability zone-based node affinity. You can use the default value Immediate.

      volumeAs

      The volume access mode. The default value is sharepath. Valid values:

      • sharepath: All persistent volumes share the same mount path. Data is stored at <bucket>:<path>/.

      • subpath: Creates a dedicated subdirectory for each persistent volume to isolate data. The data is stored at <bucket>:<path>/<pv-name>/.

        This parameter takes effect only when the CSI component version is 1.31.3 or later.

      sigVersion

      The signature version for requests sent to the OSS server.

    2. Create the StorageClass.

      kubectl apply -f sc-oss.yaml

Console

  1. Store the AccessKey from Step 1 as a Secret for the persistent volume.

    1. On the ACK Clusters page, click the name of your cluster. In the left navigation pane, click Configurations > Secrets.

    2. Click Create from YAML and follow the on-screen instructions to create the Secret.

      apiVersion: v1
      kind: Secret
      metadata:
        name: oss-secret
        # Must match the namespace of the application.
        namespace: default
      stringData:
        # Replace with your AccessKey ID.
        akId: <your AccessKey ID>
        # Replace with your AccessKey secret.
        akSecret: <your AccessKey Secret>
  2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Volumes > StorageClasses.

  3. On the StorageClasses page, click Create, set PV Type to OSS, and follow the on-screen instructions to configure the StorageClass.

    Parameter

    Description

    Access certificate

    The Secret used to access OSS, which contains the AccessKey ID and AccessKey Secret.

    Bucket ID:

    The OSS bucket to use.

    The console displays only the buckets that the configured AccessKey can access.

    OSS path

    The version of the CSI component must be v1.14.8.32-c77e277b-aliyun or later.

    The mount path relative to the root directory of the bucket. The default value is /, which mounts the entire bucket.

    If the ossfs version is earlier than 1.91, the specified path must exist in the OSS bucket. For more information, see New features in ossfs 1.91 and later.

    Volume Mode

    The volume access mode. The default is Shared Directory. Valid values:

    The Subdirectory mode takes effect only when the CSI component version is 1.31.3 or later. Otherwise, the Shared Directory mode is used.
    • Shared Directory (sharepath): All persistent volumes share the same mount path. Data is stored at <bucket>:<path>/.

    • Subdirectory (subpath): Creates a dedicated subdirectory for each persistent volume to isolate data. Data is stored at <bucket>:<path>/<pv-name>/.

    Endpoint

    The endpoint of the OSS Bucket to mount.

    • Use an internal endpoint if the mount node and the bucket are in the same region, or if a VPC connection is established.

    • Use a public endpoint if the mount node and the bucket are in different regions.

    The following are common formats for different access ports:

    • Internal network format: http://oss-{{regionName}}-internal.aliyuncs.com or https://oss-{{regionName}}-internal.aliyuncs.com.

      The internal endpoint format vpc100-oss-{{regionName}}.aliyuncs.com is deprecated. Switch to the new format promptly.
    • Public network format: http://oss-{{regionName}}.aliyuncs.com or https://oss-{{regionName}}.aliyuncs.com.

    By default, the console uses HTTP to access OSS over an internal network. To use HTTPS, you must create the PV by using kubectl.

    Reclaim policy

    The reclaim policy for dynamically provisioned persistent volumes. OSS persistent volumes support only Retain, which means that when you delete a PVC, the corresponding persistent volume and the data in the OSS bucket are not deleted.

    Optional Parameters:

    Custom parameters for the OSS persistent volume, in the format of -o *** -o ***. For example: -o umask=022 -o max_stat_cache_size=100000 -o allow_other.

    Expand for details

    • umask: Sets the permission mask for files and directories.

      For example, umask=022 changes the file permissions to 755. This resolves the permission issue for files that are uploaded by using other methods, such as SDKs or the OSS console, which have a default permission of 640. We recommend that you configure this parameter for read/write splitting or multi-user access.

    • max_stat_cache_size: Sets the upper limit of metadata cache entries, for example, 100000. Metadata is cached in memory to improve the performance of operations such as ls and stat.

      However, this cache cannot immediately detect file modifications made through the OSS console, SDKs, or ossutil. This may cause applications to read inconsistent data. If you have strict data consistency requirements, set this parameter to 0 to disable the cache, or reduce the cache expiration time by using the stat_cache_expire parameter. This, however, may reduce read performance.

    • allow_other: Allows users other than the user who mounted the volume to access files and directories in the mount path. This is suitable for multi-user shared environments where non-mounting users also need to access data.

    For more information about optional parameters, see Mount options and Best practices for configuring ossfs 1.0.

Step 3: Create a PVC

Create a PVC to dynamically request storage resources. The CSI plug-in automatically creates a PV based on the StorageClass.

kubectl

  1. Create a file named pvc-oss.yaml:

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      # PVC name
      name: pvc-oss
    spec:
      # Configure the access mode. ReadOnlyMany means that ossfs mounts the OSS bucket in read-only mode.
      accessModes:
      - ReadOnlyMany
      volumeMode: Filesystem
      resources:
        requests:
          # Declare the requested storage capacity. This value does not limit the actual capacity of the OSS volume.
          storage: 20Gi
      # Specify the StorageClass to use.
      storageClassName: sc-oss

    Parameter

    Description

    accessModes

    Specifies the access mode. Supported values are ReadOnlyMany and ReadWriteMany.

    If set to ReadOnlyMany, ossfs mounts the OSS bucket in read-only mode.

    storage

    The requested capacity of the volume. This value does not limit the actual capacity of the OSS volume.

    storageClassName

    The name of the StorageClass.

  2. Create the PVC:

    kubectl apply -f pvc-oss.yaml
  3. Verify that the PVC is created and in the Bound state:

    kubectl get pvc pvc-oss

    Expected output:

    NAME           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS         VOLUMEATTRIBUTESCLASS   AGE
    pvc-oss        Bound    oss-251d111d-3b0b-4879-81a0-eb5a19xxxxxx   20Gi       ROX            sc-oss             <unset>                 4d20h

Console

  1. On the Clusters page, click the name of your cluster. In the left navigation pane, click Volumes > Persistent Volume Claims.

  2. On the Persistent Volume Claims page, click Create. Set PVC Type to OSS and configure the parameters.

    Parameter

    Description

    Allocation Mode

    Select Use StorageClass.

    Existing Storage Class

    Click Select Storage Class and choose the previously created StorageClass.

    Capacity

    The requested capacity of the volume. This value does not limit the actual capacity of the OSS volume.

    Access Mode

    Specifies the access mode. Supported values are ReadOnlyMany and ReadWriteMany.

    If set to ReadOnlyMany, ossfs mounts the OSS bucket in read-only mode.

Step 4: Create an application and mount a volume

To mount the volume, reference the PVC in your application.

kubectl

  1. Create an oss-workload.yaml file.

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: oss-workload
      labels:
        app: nginx
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6
            ports:
            - containerPort: 80
            volumeMounts:
              # The mount path in the container.
              - name: pvc-oss
                mountPath: "/data"
            # Configure a health check.
            livenessProbe:
              exec:
                command:
                - ls
                - /data
              initialDelaySeconds: 30
              periodSeconds: 30
          volumes:
            - name: pvc-oss
              persistentVolumeClaim:
                # Reference the PVC that you created.
                claimName: pvc-oss
  2. Create the application.

    kubectl create -f oss-workload.yaml
  3. Verify the mount.

    • Confirm that the Pods are running.

      kubectl get pod -l app=nginx
    • Access a Pod and check the mount point.

      kubectl exec -it <pod-name> -- ls /data

      The output should list the files in the OSS mount path.

Console

  1. On the ACK Clusters page, click the name of your cluster. In the left navigation pane, click Workloads > Deployments.

  2. On the Deployments page, click Create from Image.

  3. Click Create from Image and follow the on-screen instructions to configure the application parameters.

    The key parameters are described in the following table. For other parameters, you can use the default values. For more information, see Create a Deployment.

    Configuration page

    Parameter

    Description

    Basic Information

    Replicas:

    The number of Pod replicas for the Deployment.

    Container

    Image name

    The address of the container image for the application, such as anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6.

    Required Resources

    The amount of vCPU and memory to allocate to each container.

    Volumes

    Click Add PVC and then configure the parameters.

    • Mount Source: Select the PVC that you created earlier.

    • Container Path: The path within the container where the volume will be mounted, such as /data.

    Labels and Annotations

    Pod label

    For example, set the key to 'app' and the value to 'nginx'.

  4. Check the application deployment status.

    On the Deployments page, click the name of your application. On the Pods tab, verify that the Pods have a Running status.

Step 5: Verify shared and persistent storage

Verify shared storage

To verify shared storage, create a file in one Pod and then view it in another.

  1. Get the names of the application Pods.

    kubectl get pod -l app=nginx
  2. This example uses the Pod oss-workload-66fbb85b67-d****:

    • ReadWriteMany: Create the tmpfile file in the /data path.

      kubectl exec oss-workload-66fbb85b67-d**** -- touch /data/tmpfile
    • ReadOnlyMany: Upload the tmpfile file to the corresponding path in the OSS bucket by using the OSS console or by uploading a file with cp.

  3. Check for the file in the mount path of another Pod.

    Take a Pod named oss-workload-66fbb85b67-l**** with the mount path data as an example.

    kubectl exec oss-workload-66fbb85b67-l**** -- ls /data | grep tmpfile

    The presence of the file in the output confirms that the Pods can share data.

    tmpfile
    If you do not see the expected output, make sure that your CSI component version is v1.20.7 or later.

Verify persistent storage

To verify persistent storage, delete a Pod and then check that the file remains after the Pod is automatically recreated.

  1. Delete an application Pod. The Deployment controller automatically recreates it.

    kubectl delete pod oss-workload-66fbb85b67-d****
  2. Check the status of the Pods. Wait for the new Pod to start and become Running.

    kubectl get pod -l app=nginx
  3. View the files in the /data path.

    This example uses a Pod named oss-workload-66fbb85b67-z**** with the mount path data.

    kubectl exec oss-workload-66fbb85b67-z**** -- ls /data | grep tmpfile

    The output shows the tmpfile file still exists, which confirms that data persists even after the Pod is recreated.

    tmpfile

Known impacts

  • Data integrity risks

    • Concurrent write consistency risk: To improve write stability, we recommend upgrading the CSI component to v1.28 or later. However, in single-file concurrent write scenarios, the OSS "overwrite upload" feature can still overwrite data. You must ensure data consistency at the application layer.

    • Data synchronization and accidental deletion risk: When a volume is mounted, deleting or modifying files in the mount path on an application Pod or host node directly synchronizes those changes with the source files in the OSS Bucket. To prevent accidental data deletion, we recommend enabling versioning for your OSS Bucket.

  • Application stability risks

    • OOM risk: When you perform a readdir operation (such as the ls command in a shell script) on a large number of files (for example, more than 100,000, depending on the node memory) for the first time, ossfs loads all metadata at once and consumes a large amount of memory. This may cause the process to be OOM Killed and make the mount point unavailable.

      To mitigate this risk, we recommend mounting a subdirectory of the OSS Bucket or optimizing your directory structure.

    • Extended mount time: Configuring securityContext.fsgroup in an application causes kubelet to recursively modify file permissions (chmod/chown) when mounting a volume. If the volume contains a large number of files, the mount time is significantly extended, which can cause severe Pod startup delays.

      If you must configure this parameter but want to reduce the mount time, see Increased mount time for OSS volumes.

    • Key invalidation risk (AccessKey authentication): If an AccessKey referenced by a PV becomes invalid or its permissions change, the associated application immediately loses access.

      To restore access, you must update the credentials in the Secret and restart the application Pod to force a remount. This causes a service interruption, so perform this operation during a maintenance window. For more information, see Solutions.

  • Cost risks

    • Part costs: When uploading files larger than 10 MB, ossfs performs a multipart upload, splitting the file into parts. If an upload is unexpectedly interrupted, for example, due to an application restart, you must manually delete the parts or delete them by using lifecycle rules to avoid storage costs from incomplete parts.

Resource release

  1. Delete workloads

    • Action: Delete all workloads, such as Deployments and StatefulSets, that use the PVC. This stops the Pods and unmounts the volume.

    • Example command: kubectl delete deployment <your-deployment-name>

  2. Delete the PVC

    • Action: Delete the PVC. The reclaim policy (reclaimPolicy) for dynamically provisioned OSS volumes supports only Retain. When you delete the PVC, its bound PV enters the Released state, and the backend OSS bucket retains the corresponding subdirectory and data.

    • Example command: kubectl delete pvc <your-pvc-name>

  3. Delete the PV

    • Action: Delete the PV in the Released state. This operation only removes the PV resource object from Kubernetes and does not delete any data in the backend OSS bucket.

    • Example command: kubectl delete pv <your-pv-name>

  4. Delete the StorageClass and Secret (optional)

    • Action: Delete the StorageClass and the associated Secret if they are no longer needed. This operation only removes the resource objects from Kubernetes and does not affect the backend OSS bucket.

    • Example commands: kubectl delete sc <your-storageclass-name>, kubectl delete secret <your-secret-name>

Related documents