strmvol volume overview-Container Service for Kubernetes(ACK)-阿里云帮助中心

You can use strmvol volumes to mount Object Storage Service (OSS) data through virtual block devices. This improves the read performance in scenarios where ultra-large numbers of OSS objects are mounted.

Solution overview

OSS supports massive data storage and the pay-as-you-go billing method, and provides convenient access features based on the HTTP protocol. In containerized scenarios, a FUSE file system is usually used to mount OSS data. In small file reading scenarios, you can use the virtual block device solution to mount OSS data as strmvol volumes.

strmvol volumes enable OSS data access through virtual block devices. This solution resolves the performance bottlenecks at the FUSE intermediate layer and allows you to directly go to the storage driver layer to accelerate data access. This solution is ideal for scenarios where you need to quickly traverse millions of small files, such as loading AI training datasets and analyzing time series data.

Features of the solution that uses FUSE

This solution uses a FUSE file system, such as ossfs. Features:

Data changes on the OSS server can be detected. If you disable metadata caching, data changes take effect in real time.
Write operations are supported. Extended file system information, including group ID (gid) and user ID (uid), is supported.
Read and write requests are directed to the ossfs client through the FUSE layer. Scenarios involving high peak requests, such as traversing numerous small files, consume high performance overhead.
In earlier versions of the Container Storage Interface (CSI) plug-in, the ossfs client runs as a process on the node where it resides. In the current version of the CSI plug-in, the ossfs client runs as a separate pod in the cluster. Pods on the same node and persistent volume (PV) share the same ossfs client for OSS access.

Features of the virtual block device solution

This solution is based on the underlying virtual block device and kernel-state file system, such as Enhanced Read-Only File System (EROFS). Features:

During initialization, all file metadata under the OSS bucket mount target is retrieved and indexed. The current version does not support data synchronization for the OSS server during runtime.
In read-only scenarios, extended information parsing is not supported for OSS objects.
Performance overhead at the FUSE layer is eliminated. The preset memory size is used as the data storage medium to accelerate prefetching in small file reading scenarios.
In addition to the CSI plug-in, this solution requires an additional DaemonSet named strmvold to manage the virtual block devices on the nodes. After the devices are attached to the nodes, the preset node resources are occupied for data caching.

Scenarios

If your business meets the following requirements, you can use the virtual block device solution (strmvol volumes).

Note

The solution that uses FUSE is suitable for most scenarios. For more information, see Use an ossfs 1.0 statically provisioned volume, Use ossfs 1.0 dynamically provisioned volumes, and Use ossfs 2.0 volumes.

Data is stored in OSS buckets and does not need to be updated during business running.
Your business is not sensitive to the extended information of the file system.
Read-only scenarios, especially those that involve numerous small files or random reads.

How to use strmvol volumes

Before you use strmvol volumes, you must install the strmvol-csi-driver component on the Marketplace page in the Container Service for Kubernetes (ACK) console. After you deploy strmvol-csi-driver, the following resources are created.

Resource name	Resource type	Feature
strmvold	DaemonSet	Manages the virtual block devices on the node.
strmvol-csi-plugin	DaemonSet	Implements strmvol volume mounting and unmounting.
strmvol-csi-provisioner	Deployment	Implements dynamic strmvol volume creation.

上一篇: ossfs 2.0 persistent volume FAQ 下一篇: Use strmvol volumes