Use a static OSS volume

Updated at:

Object Storage Service (OSS) volumes give your application shared, persistent storage for unstructured data such as images, audio, and video files. Data written to an OSS bucket stays available to every pod that mounts the volume and survives pod restarts.

Background

OSS is a secure, cost-effective, high-capacity, and highly reliable cloud storage solution. It is ideal for storing data that is rarely modified after being written, as well as unstructured data such as images, audio, and video files. For more information, see Storage overview.

OSS volume clients

An OSS volume is a type of volume that uses a Filesystem in Userspace (FUSE) client to mount object storage as a local file system (FS). Compared with traditional local storage and block storage, OSS volumes have some POSIX compatibility limitations. Alibaba Cloud Container Compute Service (ACS) supports the following OSS volume clients.

Use cases

Client

Type

Description

Most scenarios, including read/write operations and user permission configuration.

ossfs 1.0

FUSE

Supports most POSIX operations, including append write, random write, and setting user permissions.

Read-only scenarios or workloads that only involve sequential append writes. Examples include AI model training and inference, big data processing, and autonomous driving.

ossfs 2.0

FUSE

Supports full reads and sequential append writes. Ideal for read-heavy scenarios, such as AI model training and inference, big data processing, and autonomous driving. This client can significantly improve data read performance.

Before you choose a client, consider the following:

  • Unknown read/write pattern — If you are unsure about the read/write model of your workload, use ossfs 1.0. It offers better POSIX compatibility and ensures greater operational stability for your workloads.

  • Separable read and write paths — For scenarios where read and write operations can be separated, such as when reads and writes do not occur simultaneously or target different files (for example, saving checkpoints or persistent logging), use separate volumes. For example, mount a read-only path using an ossfs 2.0 volume and a write path using an ossfs 1.0 volume.

  • Compute power required by ossfs 2.0 — ossfs 2.0 supports only GPU compute power. To use CPU compute power, submit a ticket to request access.

  • Mount method supported by ossfs 2.0 — ACS supports mounting static ossfs 2.0 volumes only by using kubectl.

POSIX API support

The following table compares the POSIX API support of ossfs 1.0 and ossfs 2.0.

Feature category

Operation

ossfs 1.0

ossfs 2.0

Basic object operations

open

Supported

Supported

Basic object operations

flush

Supported

Supported

Basic object operations

close

Supported

Supported

File reads and writes

read

Supported

Supported

File reads and writes

write

Supports random writes (requires a disk cache)

Supports only sequential writes (no disk cache required)

File reads and writes

truncate

Supported (can be truncated to any size)

Only supports truncating an object to zero length

File meta operations

create

Supported

Supported

File meta operations

unlink

Supported

Supported

File meta operations

rename

Supported

Supported

Directory operations

mkdir

Supported

Supported

Directory operations

readdir

Supported

Supported

Directory operations

rmdir

Supported

Supported

Permissions and attributes

getattr

Supported

Supported

Permissions and attributes

chmod

Supported

Unsupported

Permissions and attributes

chown

Supported

Unsupported

Permissions and attributes

utimes

Supported

Supported

Extended features

setxattr

Supported

Unsupported

Extended features

symlink

Supported

Unsupported

Extended features

lock

Unsupported

Unsupported

Performance benchmark

ossfs 2.0 delivers significant performance improvements over ossfs 1.0 in sequential read/write operations and high-concurrency small-file reads.

  • Sequential write performance: In single-threaded, large-object sequential write scenarios, ossfs 2.0 provides nearly 18 times the bandwidth of ossfs 1.0.

  • Sequential read performance: In single-threaded or multi-threaded (4 threads) large-object sequential read scenarios, ossfs 2.0 provides more than 3 times the bandwidth of ossfs 1.0.

  • Concurrent small-object read performance: In high-concurrency (128 threads) small-object read scenarios, ossfs 2.0 provides more than 20 times the bandwidth of ossfs 1.0.

    If the read/write performance, such as latency or throughput, does not meet your expectations, see Best practices for tuning the performance of OSS volumes.

Prerequisites

The managed-csiprovisioner add-on is installed in the ACS cluster.

Note

Go to the ACS cluster management page in the ACS console. In the left-side navigation pane of the cluster management page, choose Operations > Add-ons. On the Storagetab, check whether managed-csiprovisioner is installed.

Notes

The following notes mainly apply to general read/write scenarios that use the ossfs 1.0 client. The ossfs 2.0 client is read-focused and supports only a subset of POSIX operations.

  • ACS supports only OSS static volumes, not dynamic volumes.

  • Hard links are not supported.

  • You cannot mount buckets whose storage class is Archive, Cold Archive, or Deep Cold Archive.

  • Random or append writes to a file create a new file locally and then re-upload it to the OSS server. Due to the nature of object storage, note the following:

    • Rename operations for files and directories are not atomic.

    • Avoid concurrent writes or operations such as compression and decompression directly in the mount directory. Coordinate concurrent writes on the client side, for example by serializing the writes that target the same object. Otherwise, concurrent writes, compression, and decompression in the mount directory can leave data or metadata inconsistent, and ACS does not protect against such inconsistencies.

  • By default, the readdir operation on ossfs 1.0 volumes sends numerous HeadObject requests to retrieve extended information for all objects in a directory, which can degrade overall ossfs performance in directories with many files. If your workload is not sensitive to file attributes such as permissions, you can enable the -o readdir_optimize parameter to improve performance. For more information, see readdir optimization.

Create an OSS bucket and get details

  1. Create an OSS bucket.

    1. Log on to the OSS console. In the left-side navigation pane, click Buckets.

    2. Click Create Bucket.

    3. In the Create Bucket panel, configure the required parameters and click Create.

      The following table describes the key parameters. For more information, see Create buckets.

      Parameter

      Description

      Bucket Name

      Enter a custom name. The bucket name must be globally unique in OSS and cannot be changed after creation. Follow the formatting requirements displayed in the UI.

      Region

      Select Specific Region and choose the same region as your ACS cluster. This allows pods in the cluster to access the bucket over the internal network.

  2. (Optional) To mount a subdirectory from an OSS bucket, create the subdirectory first.

    1. On the Buckets page, click the name of the target bucket.

    2. On the bucket details page, in the left-side navigation pane, choose Object Management > Objects.

    3. Click Create Directory to create directories in the bucket as needed.

  3. Get the endpoint of the OSS bucket.

    1. On the Buckets page, click the name of the target bucket.

    2. On the bucket details page, click the Overview tab. In the Port section, copy the endpoint that matches your deployment:

      • If the OSS bucket and the ACS cluster are in the same region, copy the internal endpoint.

      • If the OSS bucket is region-agnostic or is in a different region from the ACS cluster, copy the public endpoint.

  4. Get an AccessKey ID and an AccessKey Secret to access OSS. For more information, see Obtain an AccessKey pair.

    If you need to mount an OSS bucket that belongs to another Alibaba Cloud account, you must get the AccessKey pair from that account.

Mount an OSS volume

Select the tab for the client that you chose. Each procedure creates a Persistent Volume (PV) and a Persistent Volume Claim (PVC), and then mounts the volume in an application.

ossfs 1.0 volume

kubectl

Step 1: Create a PV
  1. Save the following YAML content as oss-pv.yaml.

    apiVersion: v1
    kind: Secret
    metadata:
      name: oss-secret
      namespace: default
    stringData:
      akId: <your AccessKey ID>
      akSecret: <your AccessKey Secret>
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: oss-pv
      labels:
        alicloud-pvname: oss-pv
    spec:
      storageClassName: test 
      capacity:
        storage: 20Gi
      accessModes:
        - ReadWriteMany
      persistentVolumeReclaimPolicy: Retain
      csi:
        driver: ossplugin.csi.alibabacloud.com
        volumeHandle: oss-pv
        nodePublishSecretRef:
          name: oss-secret
          namespace: default
        volumeAttributes:
          bucket: "<your OSS Bucket Name>"
          url: "<your OSS Bucket Endpoint>"
          otherOpts: "-o umask=022 -o allow_other"
    Note

    This YAML creates a Secret and a PV. The Secret stores your AccessKey pair, which the PV uses for secure access. Replace the values of akId and akSecret with your actual AccessKey ID and AccessKey Secret.

    The following table describes the PV parameters:

    Parameter

    Description

    alicloud-pvname

    The label of the PV, used to bind it to a PVC.

    storageClassName

    This value is for PVC binding only and does not need to correspond to an existing StorageClass object.

    storage

    The capacity of the OSS volume. The capacity specified for a static OSS volume is for declarative purposes only. The actual capacity is not limited, and the available capacity is based on the usage displayed in the OSS console.

    accessModes

    The access mode. Valid values: ReadOnlyMany, which lets multiple pods mount the volume as read-only, and ReadWriteMany, which lets multiple pods mount the volume as read-write.

    persistentVolumeReclaimPolicy

    The reclaim policy of the PV.

    driver

    The driver type. Set this to ossplugin.csi.alibabacloud.com to use the Alibaba Cloud OSS Container Storage Interface (CSI) plugin.

    volumeHandle

    The UID (unique identifier) of the PV. This must be the same as metadata.name.

    nodePublishSecretRef

    References the Secret that contains the AccessKey pair for authorization.

    bucket

    The name of the OSS bucket. Replace the value of bucket with your actual OSS bucket name.

    url

    The endpoint of the OSS bucket. Replace the value of url with the actual endpoint of your OSS bucket.

    • If the OSS bucket and the ACS cluster are in the same region, use the internal endpoint. For example, oss-cn-shanghai-internal.aliyuncs.com.

    • If the OSS bucket is region-agnostic or is in a different region from the ACS cluster, use the public endpoint. For example, oss-cn-shanghai.aliyuncs.com.

    otherOpts

    Specifies custom mount options for the OSS volume in the format -o *** -o ***. For example, -o umask=022 -o max_stat_cache_size=100000 -o allow_other.

    Expand to view details

    • umask: changes the permissions that ossfs uses to read files. For example, umask=022 changes the permissions of ossfs files to 755. This resolves insufficient permissions inside the mount point for objects that were uploaded by other means, such as an SDK or the OSS console, which have default permissions of 640. (Recommended) Set this option for read/write separation or multi-user access.

    • max_stat_cache_size: sets the maximum number of entries in the metadata cache, such as 100000. Caching object metadata in memory improves the performance of operations such as ls and stat. However, this cache does not immediately detect changes that are made through the OSS console, an SDK, or ossutil, so your application can read inconsistent data. If you have strict data consistency requirements, set it to 0 to disable the cache, or use the stat_cache_expire parameter to shorten the cache expiration time. Both settings reduce read performance.

    • allow_other: allows users other than the mounting user to access the files and directories in the mount point. Use this option in multi-user shared environments where users other than the mounting user also need to access the data.

    For more configurations, see Mount options and Best practices.

  2. Create the Secret and the PV.

    kubectl create -f oss-pv.yaml
  3. View the PV.

    kubectl get pv

    Expected output:

    NAME     CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   VOLUMEATTRIBUTESCLASS   REASON   AGE
    oss-pv   20Gi       RWX            Retain           Available           test           <unset>                          9s
Step 2: Create a PVC
  • Save the following YAML content as oss-pvc.yaml.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: oss-pvc
spec:
  storageClassName: test
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 20Gi
  selector:
    matchLabels:
      alicloud-pvname: oss-pv

The following table describes the parameters:

Parameter

Description

storageClassName

This value is for PVC binding only and does not need to correspond to an existing StorageClass object. It must be the same as the spec.storageClassName of the PV.

accessModes

The access mode.

storage

The amount of storage to request. This value must not be greater than the PV's capacity.

alicloud-pvname

The label of the PV to bind. It must be the same as the metadata.labels.alicloud-pvname of the PV.

  • Create the PVC.

kubectl create -f oss-pvc.yaml
  • View the PVC.

kubectl get pvc

The expected output shows that the PVC is bound to the PV created in Step 1.

NAME      STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
oss-pvc   Bound    oss-pv   20Gi       RWX            test           <unset>                 6s
Step 3: Create an application and mount the volume
  • Create a file named oss-test.yaml with the following content.

    The following YAML example creates a Deployment with two pods. Both pods request storage by using the PVC named oss-pvc, and the mount path for both is /data.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: oss-test
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest
        ports:
        - containerPort: 80
        volumeMounts:
          - name: pvc-oss
            mountPath: /data
      volumes:
        - name: pvc-oss
          persistentVolumeClaim:
            claimName: oss-pvc
  • Create the Deployment and mount the OSS volume.

kubectl create -f oss-test.yaml
  • Check the status of the pods in the Deployment.

kubectl get pod | grep oss-test

The example output indicates that two pods have been created.

oss-test-****-***a   1/1     Running   0          28s
oss-test-****-***b   1/1     Running   0          28s
  • Check the mount path.

    The following command lists the data at the OSS bucket mount point. By default, the output is empty.

kubectl exec oss-test-****-***a -- ls /data

Console

Step 1: Create a PV
  1. Log on to the ACS console.

  2. On the Clusters page, click the name of the cluster to go to the cluster management page.

  3. In the left-side navigation pane of the cluster management page, choose Volumes > Persistent Volumes.

  4. On the Persistent Volumes page, click Create.

  5. In the Create dialog box, configure the parameters and click the Create button.

    Parameter

    Description

    Example

    PV Type

    The volume type. Select OSS.

    OSS

    Volume Name:

    The name of the PV. Specify a custom name according to the format requirements displayed in the UI.

    oss-pv

    Capacity

    The capacity of the OSS volume. The capacity specified for a static OSS volume is for declarative purposes only. The actual capacity is not limited, and the available capacity is based on the usage displayed in the OSS console.

    20Gi

    Access Mode

    The access mode. Select an option based on your requirements:

    • ReadOnlyMany: The volume can be mounted as read-only by multiple pods.

    • ReadWriteMany: The volume can be mounted as read-write by multiple pods.

    ReadWriteMany

    Access Certificate

    For security, store the AccessKey information in a Secret. This example uses the Create Secret option.

    • Create Secret

    • namespace: default

    • Name: oss-secret

    • AccessKey ID: ********

    • AccessKey Secret: ********

    Bucket ID:

    Select an OSS bucket.

    oss-acs-***

    OSS Path

    The directory to mount. This defaults to the root directory (/). If you specify a subdirectory, such as /dir, ensure it exists.

    /

    Endpoint:

    The endpoint of the OSS bucket.

    • If the OSS bucket and the ACS cluster are in the same region, select the Internal Endpoint option.

    • If the OSS bucket is region-agnostic or is in a different region from the ACS cluster, select the Public Endpoint option.

    Private Endpoint

    After the PV is created, you can view its information on the Persistent Volumes page. The PV is not yet bound to a PVC.

Step 2: Create a PVC
  1. In the left-side navigation pane of the cluster management page, choose Volumes > Persistent Volume Claims.

  2. On the Persistent Volume Claims page, click Create.

  3. In the dialog box that appears, configure the parameters and click the Create button.

    Parameter

    Description

    Example

    PVC Type

    The volume type. Select OSS.

    OSS

    Name

    The name of the PVC. Specify a custom name according to the format requirements displayed in the UI.

    oss-pvc

    Allocation Mode

    The allocation mode. Select the Existing Volumes option.

    Existing PV

    Existing Volumes

    Select the PV that you created.

    oss-pv

    Capacity

    The amount of storage to request. This value cannot exceed the PV's capacity.

    20Gi

    After the PVC is created, you can view it on the Persistent Volume Claims page. The PVC is now bound to the PV (the OSS volume).

Step 3: Create an application and mount the volume
  1. In the left-side navigation pane of the cluster management page, choose Workloads > Deployments.

  2. On the Deployments page, click the Create from Image button.

  3. Configure the parameters for the Deployment and click the Create button.

    The following table describes key parameters. Keep the default values for other parameters. For more information, see Create a Deployment.

    Section

    Parameter

    Description

    Example

    Basic Information

    Application Name

    The name of the Deployment. Specify a custom name according to the format requirements displayed in the UI.

    oss-test

    Basic Information

    Replicas:

    The number of replicas (pods) for the Deployment.

    2

    Container

    Image Name

    The image address used to deploy the application.

    registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest

    Container

    Required Resources

    Set the required vCPU and memory resources.

    0.25 vCPU, 0.5 GiB

    Volume

    Add PVC

    Click the button and configure the parameters:

    • Mount Source: Select the PVC that you created.

    • Container Path: Enter the container path where you want to mount the OSS bucket.

    • Mount source: oss-pvc

    • Container path: /data

  4. Check the status of the application Deployment.

    1. On the Deployments page, click the application name.

    2. On the Pods tab, make sure the pods are in the Running state.

ossfs 2.0 volume

ACS supports mounting static ossfs 2.0 volumes only by using kubectl.

Step 1: Create a PV

  1. Save the following YAML content as oss-pv.yaml.

    apiVersion: v1
    kind: Secret
    metadata:
      name: oss-secret
      namespace: default
    stringData:
      akId: <your AccessKey ID>
      akSecret: <your AccessKey Secret>
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: oss-pv
      labels:
        alicloud-pvname: oss-pv
    spec:
      storageClassName: test 
      capacity:
        storage: 20Gi
      accessModes:
        - ReadWriteMany
      persistentVolumeReclaimPolicy: Retain
      csi:
        driver: ossplugin.csi.alibabacloud.com
        volumeHandle: oss-pv
        nodePublishSecretRef:
          name: oss-secret
          namespace: default
        volumeAttributes:
          fuseType: ossfs2 # Explicitly specifies the ossfs 2.0 client
          bucket: "<your OSS Bucket Name>"
          url: "<your OSS Bucket Endpoint>"
          otherOpts: "-o close_to_open=false" # Note: Supported mount options are not compatible with the ossfs 1.0 client.
    Note

    This YAML creates a Secret and a PV. The Secret stores your AccessKey pair, which the PV uses for secure access. Replace the values of akId and akSecret with your actual AccessKey ID and AccessKey Secret.

    The following table describes the PV parameters:

    Parameter

    Description

    alicloud-pvname

    The label of the PV, used to bind it to a PVC.

    storageClassName

    This value is for PVC binding only and does not need to correspond to an existing StorageClass object.

    storage

    The capacity of the OSS volume. The capacity specified for a static OSS volume is for declarative purposes only. The actual capacity is not limited, and the available capacity is based on the usage displayed in the OSS console.

    accessModes

    The access mode. Valid values: ReadOnlyMany, which lets multiple pods mount the volume as read-only, and ReadWriteMany, which lets multiple pods mount the volume as read-write.

    persistentVolumeReclaimPolicy

    The reclaim policy of the PV.

    driver

    The driver type. Set this to ossplugin.csi.alibabacloud.com to use the Alibaba Cloud OSS Container Storage Interface (CSI) plugin.

    volumeHandle

    The UID (unique identifier) of the PV. This must be the same as metadata.name.

    nodePublishSecretRef

    References the Secret that contains the AccessKey pair for authorization.

    fuseType

    Must be ossfs2 for the ossfs 2.0 client.

    bucket

    The name of the OSS bucket. Replace the value of bucket with the name of your OSS bucket.

    url

    The endpoint of the OSS bucket. Replace the value of url with the actual endpoint of your OSS bucket.

    • If the OSS bucket and the ACS cluster are in the same region, use the internal endpoint. For example, oss-cn-shanghai-internal.aliyuncs.com.

    • If the OSS bucket is region-agnostic or is in a different region from the ACS cluster, use the public endpoint. For example, oss-cn-shanghai.aliyuncs.com.

    otherOpts

    Specifies custom mount options for the OSS volume in the format -o *** -o ***. For example, -o close_to_open=false.

    close_to_open: Disabled by default. If you enable it, the system sends a GetObjectMeta request to OSS each time a file is opened. This retrieves the latest metadata and keeps it up to date. However, in scenarios that involve reading a large number of small files, frequent metadata queries can significantly increase latency.

    For more information about optional parameters, see ossfs 2.0 mount options.

  2. Create the Secret and the PV.

    kubectl create -f oss-pv.yaml
  3. View the PV.

    kubectl get pv

    Expected output:

    NAME     CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   VOLUMEATTRIBUTESCLASS   REASON   AGE
    oss-pv   20Gi       RWX            Retain           Available           test           <unset>                          9s

Step 2: Create a PVC

  1. Save the following YAML content as oss-pvc.yaml.

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: oss-pvc
    spec:
      storageClassName: test
      accessModes:
        - ReadWriteMany
      resources:
        requests:
          storage: 20Gi
      selector:
        matchLabels:
          alicloud-pvname: oss-pv

    The following table describes the parameters:

    Parameter

    Description

    storageClassName

    This value is for PVC binding only and does not need to correspond to an existing StorageClass object. It must be the same as the spec.storageClassName of the PV.

    accessModes

    The access mode.

    storage

    The amount of storage to request. This value must not be greater than the PV's capacity.

    alicloud-pvname

    The label of the PV to bind. It must be the same as the metadata.labels.alicloud-pvname of the PV.

  2. Create the PVC.

    kubectl create -f oss-pvc.yaml
  3. View the PVC.

    kubectl get pvc

    The expected output shows that the PVC is bound to the PV created in Step 1.

    NAME      STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
    oss-pvc   Bound    oss-pv   20Gi       RWX            test           <unset>                 6s

Step 3: Create an application and mount the volume

  1. Create a file named oss-test.yaml with the following content.

    The following YAML example creates a Deployment with two pods. Both pods request storage by using the PVC named oss-pvc, and the mount path for both is /data.

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: oss-test
      labels:
        app: nginx
    spec:
      replicas: 2
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: registry.cn-hangzhou.aliyuncs.com/acs-sample/nginx:latest
            ports:
            - containerPort: 80
            volumeMounts:
              - name: pvc-oss
                mountPath: /data
          volumes:
            - name: pvc-oss
              persistentVolumeClaim:
                claimName: oss-pvc
  2. Create the Deployment and mount the OSS volume.

    kubectl create -f oss-test.yaml
  3. Check the status of the pods in the Deployment.

    kubectl get pod | grep oss-test

    The example output indicates that two pods have been created.

    oss-test-****-***a   1/1     Running   0          28s
    oss-test-****-***b   1/1     Running   0          28s
  4. Check the mount path.

    The following command lists the data at the OSS bucket mount point. By default, the output is empty.

    kubectl exec oss-test-****-***a -- ls /data

Verify OSS shared storage and persistent storage

The Deployment created in the preceding example contains two pods. Both pods mount the same OSS bucket. You can verify shared storage and persistent storage as follows:

  • To verify shared storage, create a file in one pod and then check that it is visible in the other pod.

  • To verify persistent storage, restart the Deployment and then check if the original data is still accessible from a new pod.

  1. Get the pod information.

    kubectl get pod | grep oss-test

    Sample output:

    oss-test-****-***a   1/1     Running   0          40s
    oss-test-****-***b   1/1     Running   0          40s
  2. Verify shared storage.

    1. Create a file in one of the pods.

      This example uses the pod named oss-test-****-***a.

      kubectl exec oss-test-****-***a -- touch /data/test.txt
    2. Check for the file in the other pod.

      This example uses the pod named oss-test-****-***b.

    kubectl exec oss-test-****-***b -- ls /data

    This output confirms that the test.txt file is shared.

    test.txt
  3. Verify persistent storage.

    1. Restart the Deployment.

      kubectl rollout restart deploy oss-test
    2. Check the pod status and wait for the new pods to enter the Running state.

      kubectl get pod | grep oss-test

      Sample output:

      oss-test-****-***c   1/1     Running   0          67s
      oss-test-****-***d   1/1     Running   0          49s
    3. In one of the new pods, check if the data in the file system still exists.

      This example uses the pod named oss-test-****-***c.

      kubectl exec oss-test-****-***c -- ls /data

      This output confirms that the data in the OSS bucket persists and is accessible from the new pod's mount directory.

      test.txt