CPFS for AI performance testing-Cloud Parallel File Storage(CPFS)-阿里云帮助中心

Prerequisites

A CPFS for Lingjun file system is created and associated with a Lingjun GPU server during server creation.

Background

Use the FIO tool to benchmark your CPFS for Lingjun file system. Other benchmarking tools may produce results that do not reflect actual performance. The performance data in this topic comes from FIO tests on Linux and serves as the basis for the product specifications.

The data in the product specifications represents theoretical values obtained in an Alibaba Cloud lab environment. Actual performance varies depending on your operating environment. Factors such as I/O size, concurrency, storage-side load, and read/write ratio affect test results.

Procedure

Log on to the Lingjun GPU server.

Install FIO. If FIO is already installed, skip this step.

Operating system	Installation command
CentOS	`sudo yum install fio`
Red Hat	`sudo yum install fio`
Alibaba Cloud Linux	`sudo yum install fio`
Ubuntu	Run the following commands in sequence: a. `sudo apt-get update` b. `sudo apt-get install fio`
Debian	Run the following commands in sequence: a. `sudo apt-get update` b. `sudo apt-get install fio`

Run the performance test commands.

In the following commands, /cpfs-test/ is the mount path for the CPFS for Lingjun file system. Replace this path with your actual mount path.

4 KB random read IOPS:

fio -numjobs=64 -iodepth=4 -ioengine=libaio -direct=1 -rw=randread -bs=4K --group_reporting -size=1G -time_based -runtime=180 -name=4depth_64file_4k_direct_randread_iops -directory=/cpfs-test/

4 KB random write IOPS:

fio -numjobs=64 -iodepth=4 -ioengine=libaio -direct=1 -rw=randwrite -bs=4K --group_reporting -size=1G -time_based -runtime=180 -name=4depth_64file_4k_direct_randwrite_iops -directory=/cpfs-test/

4 MB sequential read throughput:

fio -numjobs=128 -iodepth=2 -ioengine=libaio -direct=1 -rw=read -bs=4M --group_reporting -size=1G -time_based -runtime=180 -name=2depth_128file_4M_direct_read_bw -directory=/cpfs-test/

4 MB sequential write throughput:

fio -numjobs=128 -iodepth=2 -ioengine=libaio -direct=1 -rw=write -bs=4M --group_reporting -size=1G -time_based -runtime=180 -name=2depth_128file_4M_direct_write_bw -directory=/cpfs-test/

FIO parameter reference

The following table describes the FIO parameters used in the test commands.

Parameter	Description
direct	Specifies whether to use direct I/O. 1 (default): Uses direct I/O, bypassing the I/O cache to write data directly to storage. 0: Does not use direct I/O.
iodepth	The I/O queue depth, which controls the maximum number of outstanding I/O requests.
rw	The I/O pattern: `randread`: Random read `randwrite`: Random write `read`: Sequential read `write`: Sequential write `randrw`: Mixed random read and write
ioengine	The I/O engine. Use `libaio` for asynchronous I/O, which closely mimics typical application behavior. For more options, see the official FIO documentation.
bs	The block size per I/O operation. Default: 4 KiB. You can specify separate values for read and write in `read,write` format. If omitted, the default is used.
size	The total size of the test file. If omitted, FIO uses the entire file or device. A percentage from 1 to 100 is also accepted. For example, `size=20%` uses 20% of the file or device.
numjobs	The number of concurrent threads. In this topic, 64 is used for IOPS tests and 128 for throughput tests.
runtime	The test duration in seconds. If omitted, FIO runs until the amount of data specified by `size` is fully processed.
group_reporting	Aggregates statistics across all jobs into a single report. When specified, FIO reports combined results instead of reporting per job.
directory	The mount path of the file system to test. Replace `/cpfs-test/` with your actual mount path.
name	A descriptive name for the test job. Set this to any identifier string.

For more information about FIO parameters, see the FIO documentation.