How to quickly get started with a CPFS-POSIX file system-Cloud Parallel File Storage(CPFS)-阿里云帮助中心

This topic uses Alibaba Cloud Linux as an example to demonstrate how to create a Cloud Parallel File Storage (CPFS) file system and mount it using a CPFS-POSIX client. The CPFS-POSIX client connects to multiple data nodes simultaneously and provides standard POSIX and MPI-IO interfaces for applications, ensuring both high performance and application compatibility.

Note

The CPFS-POSIX client offers higher I/O performance than the CPFS-NFS client. When you create a POSIX mount target, the system automatically creates three pay-as-you-go Elastic Compute Service (ECS) instances in your Alibaba Cloud account to manage the CPFS-POSIX client cluster. The instance type is ecs.g*.large (g6 or later), and you are charged based on the standard ECS pricing for the region. Ensure your Alibaba Cloud account has permissions to purchase this instance type in the target zone. For pricing details, see ECS Pricing.
If high performance is not a critical requirement, we recommend using a CPFS-NFS client to access the CPFS file system. This method does not require the three additional ECS instances. For more information about protocol services, see Protocol services.

Prerequisites

Before you begin, ensure the following requirements are met:

Activate the CPFS service: When you log on to the CPFS console for the first time, follow the on-screen instructions to activate the service.

ECS instance: An ECS instance running an operating system supported by the CPFS-POSIX client. For information about supported operating systems and instance requirements, see Create an ECS instance.

Operating system	Distribution	Kernel version
Alibaba Cloud Linux	Alibaba Cloud Linux 2.1903 64-bit	4.19.91-27.4.al7.x86_64 and earlier
RHEL or CentOS	8.4	4.18.0-305.19.1.el8_4
	8.3	4.18.0-240.22.1.el8_3
	8.2	4.18.0-193.28.1.el8_2
	8.1	4.18.0-147.8.1.el8_1
	8.0	4.18.0-80.11.2.el8_0
	7.9	3.10.0-1160.42.2.el7
	7.8	3.10.0-1127.19.1.el7
	7.7	3.10.0-1062.18.1.el7
	7.6	3.10.0-957.54.1.el7
	7.5	3.10.0-862.14.4.el7
	7.4	3.10.0-693.2.2.el7
	7.3	3.10.0-514.26.2.el7
	7.2	3.10.0-514.26.2.el7
Ubuntu	20.04.3 LTS	5.4.0-86-generic

Note

The CPFS-POSIX client software requires a minimum of 2 CPU cores and 4 GiB of memory to run properly.
If your operating system kernel version is not supported by the POSIX client, use the NFS client to access CPFS.
For CentOS users: The CentOS Project has announced the end of life (EOL) for CentOS Linux. The CentOS Linux public images on Alibaba Cloud are from the official CentOS source. After CentOS Linux reaches its EOL, Alibaba Cloud will no longer provide support for the operating system. To avoid service disruptions, migrate to a different operating system. For more information about migrating your operating system, see Operating system migration.

This topic uses an ECS instance with the following configuration:

Operating system: Alibaba Cloud Linux 2.1903 LTS 64-bit
Kernel version: 4.19.91-27.4.al7.x86_64
vCPU: 2 vCPUs
Memory: 8 GiB
Region: China (Shanghai)
Zone: Shanghai Zone B
Internal IP address: 192.168.1.249

Procedure

The deployment process consists of five stages:

Create a CPFS file system: Create the file system in the target region and zone.
Add a POSIX mount target and join the quorum node security group: Create the mount target and add the ECS instance to the quorum node security group to ensure that the client can connect to the control plane.
Install the CPFS-POSIX client: Configure password-free logon and run the remote installation script.
Mount the file system: Run the mount command on the ECS instance.
Verify the mount: Confirm that the file system is accessible.

Step 1: Create a file system

Log on to the CPFS console.
On the Overview page, in the file system selection section, click Create under CPFS.

On the Create panel, configure the required parameters. You can use the default values for other parameters or specify custom values as needed. For more information, see Create a file system.

Parameter	Description
Region	From the drop-down list, select China (Shanghai).
Zone	Select Shanghai Zone B. The file system must be in the same zone as the ECS instance.
Type	Select 100 MB/s/TiB Baseline.
Storage Capacity	Specify the storage capacity for the file system. This example uses 3600 GiB.
VPC	Select the VPC that contains your ECS instance.
vSwitch	Select the vSwitch that is available in the selected VPC.

Click Buy Now and complete the purchase as prompted.

Step 2: Add mount target and join security group

In the left-side navigation pane, go to File Systems > File System List and click the name of the file system you created.
On the file system details page, click the Mount Targets tab.

Add a POSIX client mount target.

Click Add Mount Target for POSIX Client and configure the required parameters. Use the default values for other parameters or configure them as needed.

Parameter	Description
vSwitch	Select the vSwitch that is in the same VPC as your ECS instance.

Click OK.

Obtain the security group information of the quorum nodes.
1. In the Client Management Nodes section, click the first ECS instance in the list.
2. On the details page of the ECS instance, click the Security Groups tab and record the security group information.
Add the target ECS instance to the quorum node security group.
1. In the left-side navigation pane, choose Instances & Images > Instances.
2. Click the ID of the ECS instance where you plan to mount the CPFS file system.
3. Click the Security Groups tab and click Join Security Group. Select the quorum node security group you recorded previously (<FSID>-<GENID>-qr-sg) and click OK.

Step 3: Install the CPFS-POSIX client

In the left-side navigation pane, choose Instances & Images > Instances.
In the Operation column for the management node (an ECS instance with a name similar to cpfs-****-000001-qr-001), click Connect. For information about how to connect to an instance, see Connect to an instance.

To find the logon password for the first client management node, go to the Mount Targets page for your file system in the NAS console. In the Client Management Nodes section, record the value from the Initial Password column.
Configure password-free logon from the management node to the target ECS instance.

Note
Before you configure password-free logon for multiple ECS instances, we recommend that you set the same logon password for all target ECS instances. Otherwise, you will need to create a separate client.list for each instance and repeat the operations in Steps 6 to 8.
1. Create a client.list file and add the internal IP address of the target ECS instance.
  1. Create the client.list file.
```
touch client.list
```
  2. Open the client.list file and add the internal IP address of the target ECS instance.
```
192.168.1.249
```
    Note
    To add multiple ECS instances, enter each internal IP address on a new line.
  3. Save and close the file.
2. Create a qr.list file and add the quorum node IP addresses.
  1. Create the qr.list file.
```
touch qr.list
```
  2. Open the qr.list file and add the quorum node IP addresses. The following provides an example:
```
192.168.*.***
192.168.*.***
192.168.*.***
```
    You can find the IP addresses of the quorum nodes in the Client Management node section of the console.
  3. Save and close the file.
3. Create and configure the auto_ssh.sh script.
  1. Create the auto_ssh.sh file.
```
touch auto_ssh.sh
```
  2. Open the auto_ssh.sh file and add the following content.
```
#!/usr/bin/expect
set timeout 10
set username [lindex $argv 0]
set password [lindex $argv 1]
set hostname [lindex $argv 2]
spawn ssh-copy-id -i /root/.ssh/id_rsa.pub $username@$hostname
expect {
 #first connect, no public key in ~/.ssh/known_hosts
 "Are you sure you want to continue connecting (yes/no)?" {
 send "yes\r"
 expect "password:"
 send "$password\r"
 }
 #already has public key in ~/.ssh/known_hosts
 "password:" {
 send "$password\r"
 }
 "Now try logging into the machine" {
 #it has authorized, do nothing!
 }
 "already exist on the remote" {
 #it has authorized, do nothing!
 }
}
expect eof
```
  3. Save and close the file.
Make the auto_ssh.sh script executable.
```
chmod +x auto_ssh.sh
```
Set the password variable for the target ECS instance.
```
PASSWD='<your_ecs_instance_password>'
```
Note
If the ECS instances have different login passwords, you need to create a client.list file for each instance and repeat Step 6 to Step 8.

Configure password-free logon for the client nodes.

cat client.list | xargs -I {} ./auto_ssh.sh root $PASSWD {}

Remotely install the CPFS client on the ECS instances in client.list. The returned message Complete! indicates that the installation is successful.

region_id=`curl http://100.100.100.200/latest/meta-data/region-id`;
wget https://cpfs-${region_id}-pre.oss-${region_id}-internal.aliyuncs.com/cpfs/CPFS2.3.4-CentOS.tar.gz -O /root/CPFS2.3.4-CentOS.tar.gz;
cat client.list | xargs -I {} scp /root/CPFS2.3.4-CentOS.tar.gz {}:/root/
pssh -ih client.list "tar xzvf /root/CPFS2.3.4-CentOS.tar.gz"
pssh -ih client.list "cd /root/CPFS2.3.4-CentOS/CentOS7;yum install -y gpfs.adv-*.x86_64.rpm gpfs.base-*.x86_64.rpm gpfs.docs-*.noarch.rpm gpfs.gpl-*.noarch.rpm gpfs.gskit-*.x86_64.rpm gpfs.gss.pmsensors-*.x86_64.rpm gpfs.license.dm-*.x86_64.rpm gpfs.msg.en_US-*.noarch.rpm"

Remotely compile the kernel extension on the ECS instances listed in the client.list file.
```
pssh -ih client.list "/usr/bin/yum -y install cpp gcc-c++ elfutils-libelf-devel" 
pssh -ih client.list "export LINUX_DISTRIBUTION=KERNEL_ORG_LINUX; /usr/lpp/mmfs/bin/mmbuildgpl"
```
After the environment is configured, you can create a custom image from the instance. This lets you create new ECS instances from the image without repeating the configuration. For more information about how to create a custom image, see Overview of custom images.

Step 4: Mount the CPFS file system

Update the hosts file on the control plane nodes and target ECS instances.

pssh -ih client.list "hostname;ifconfig eth0 | grep -v inet6 | grep inet | awk '{print \$2}'" | grep -v SUCC | sed 'N;s/\n/ /' > /tmp/ip-host-client.list
pssh -ih qr.list "hostname;ifconfig eth0 | grep -v inet6 | grep inet | awk '{print \$2}'" | grep -v SUCC | sed 'N;s/\n/ /' > /tmp/ip-host-qr.list
cat /tmp/ip-host-client.list | awk '{print $2,$1,$1"t_MAGICTAG"}' > /etc/client-hosts.append
cat /tmp/ip-host-qr.list | awk '{print $2,$1,$1"t_MAGICTAG"}' > /etc/qr-hosts.append
pssh -ih client.list "cp /etc/hosts hosts.bak"
pssh -ih qr.list "cp /etc/hosts hosts.bak"
cat client.list | xargs -I {} scp /etc/client-hosts.append {}:/etc/
cat client.list | xargs -I {} scp /etc/qr-hosts.append {}:/etc/
cat qr.list | xargs -I {} scp /etc/client-hosts.append {}:/etc/
pssh -ih client.list "cat /etc/client-hosts.append >> /etc/hosts"
pssh -ih client.list "cat /etc/qr-hosts.append >> /etc/hosts"
pssh -ih qr.list "cat /etc/client-hosts.append >> /etc/hosts"

Add the target ECS instances to the CPFS cluster.

Run the following commands.

mmaddnode -N client.list
mmchlicense client --accept -N client.list
mmchnode --perfmon -N client.list
mmstartup -N client.list
mmgetstate -a

Verify the output.

If the GPFS state of each target ECS instance node shows active, the nodes are successfully added to the CPFS cluster.

 Node number  Node name                            GPFS state  
---------------------------------------------------------------
           1  cpfs-2989198****323d3-000001-qr-001  active
           2  cpfs-2989198****323d3-000001-qr-002  active
           3  cpfs-2989198****323d3-000001-qr-003  active
           4  iZ0jl91t3p5kehvr6i5****              active

Confirm that the node list includes the target ECS instances.

Run the command.
```
mmlscluster
```

Verify the output.

 Node  Daemon node name                     IP address    Admin node name                      Designation
-----------------------------------------------------------------------------------------------------------
   1   cpfs-2989198****323d3-000001-qr-001  192.168.3.57  cpfs-2989198****323d3-000001-qr-001  quorum-manager-perfmon
   2   cpfs-2989198****323d3-000001-qr-002  192.168.3.58  cpfs-2989198****323d3-000001-qr-002  quorum-manager-perfmon
   3   cpfs-2989198****323d3-000001-qr-003  192.168.3.56  cpfs-2989198****323d3-000001-qr-003  quorum-manager-perfmon
   4   iZ0jl91t3p5kehvr6i5****             192.168.3.59  iZ0jl91t3p5kehvr6i5****            perfmon

Step 5: Verify the mount

After the file system is mounted, it functions as a standard directory on the ECS instance, where you can read and write files.

Connect to the target ECS instance. For more information, see Connect to an ECS instance.

Run the following command to verify that the mount target is listed.

df -h

Example output:

Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                 7.6G     0  7.6G   0% /dev
tmpfs                    7.6G     0  7.6G   0% /dev/shm
tmpfs                    7.6G  512K  7.6G   1% /run
tmpfs                    7.6G     0  7.6G   0% /sys/fs/cgroup
/dev/vda1                 40G  3.4G   37G   9% /
tmpfs                    1.6G     0  1.6G   0% /run/user/0
01a392bac3****-000001    3.6T  432M  3.6T   1% /cpfs/01a392bac3****-000001

In the output, /cpfs/01a392bac3****-000001 is the mount target.

Run the following command to change to the CPFS file system directory.
```
cd <mount_target>
```
You can use the CPFS file system like any other directory.