Deploy a high-availability web service with shared storage and multi-zone NAS disaster recovery-Elastic Compute Service(ECS)-阿里云帮助中心

As your business expands, you need a web architecture that can handle increasing traffic and frequent content updates. This guide shows you how to build a highly available solution with shared storage to ensure 24/7 service uptime, maintain data consistency, and lower storage costs.

Solution architecture

The following diagram shows the default architecture for this solution. You can customize the settings during deployment, but the resulting environment will be similar.

The architecture includes the following components:

A Virtual Private Cloud (VPC) provides an isolated network environment for your cloud resources, such as CLB, ECS, and NAS file systems.
Two vSwitches connect the two ECS instances within the same VPC to enable communication and provide basic network segmentation.
Two ECS instances run in different zones within the same region to form a high-performance, highly available pool of backend servers.
An internet-facing Classic Load Balancer (CLB) distributes incoming traffic across the backend ECS instances. This eliminates single points of failure and ensures service stability by automatically routing requests to healthy instances if an instance fails.
Two NAS file systems provide shared storage for multiple web servers, ensuring data consistency. An automation script uses inotify-tools and rsync to enable incremental backups and automatic failover between the primary and backup NAS file systems.

Solution deployment

Quick start

Use Resource Orchestration Service (ROS) to automatically create and configure all the required resources with a single click. This lets you quickly deploy and experience the highly available web service with shared storage.

Create one Virtual Private Cloud (VPC).
Create two vSwitches.
Create a security group and configure its rules.
Create two spot instances and deploy a web service on them.
Create one internet-facing Classic Load Balancer (CLB) and add the two ECS instances as backend servers.
Create two General-purpose NAS file systems in different zones.
Use inotify-tools, rsync, and an automation script to implement incremental backup and automatic failover between the primary and backup NAS file systems.
About the incremental backup and failover scripts
The scripts are located in the following paths:
- Incremental backup script: /etc/systemd/system/sync_nas.sh
  
  This script automatically synchronizes data to the backup NAS file system whenever data on the primary NAS file system changes.
- Failover script: /etc/systemd/system/sync_check_switch.sh
  
  This script automatically checks the availability of the primary NAS file system. If the primary system is available, the script uses it. Otherwise, the system fails over to the backup NAS file system. After the primary system recovers, the script automatically switches back.
Important
This disaster recovery solution for NAS is a basic example and has limitations. You must enhance it based on your specific business requirements.

For example, this solution is unidirectional (from the primary to the backup system). In a real-world scenario that involves both read and write operations, you must implement additional strategies for when the primary NAS file system fails:

If high write availability is not a critical requirement, you can disable write operations during the failover to the backup NAS file system.

If high write availability is required, do not immediately switch operations back to the primary NAS file system after it recovers. First, synchronize the new data generated on the backup system during the outage back to the primary system. Then, switch back. Ensure that no new file changes occur on the backup system during the switch-over, or that any such changes are also synchronized to the primary system.

Click Deploy Now. In the upper-left corner, select a region. This tutorial uses China (Hangzhou) as an example.
On the Configure Template Parameters page, specify a name for the stack, select the zones for the ECS instances and NAS file systems, select an instance type for the ECS instances, and configure a password for the ECS instances. After you enter the required information, click Create to start the deployment.
When the status on the Stack Information tab changes to Created., the resources are created.
Click the Output tab to view and access the created resources and verify the solution. After the stack is successfully created, the Output tab displays the following information: FileSystemId1 (ID of the primary NAS file system), FileSystemId2 (ID of the backup NAS file system), ECS1URL (URL of the ECS 1 instance), ECS2URL (URL of the ECS 2 instance), SlbIpAddress (public IP address), MountInfo1 (mount directory /nas_master), and MountInfo2 (mount directory /nas_backup).

Manual deployment

Follow these steps to manually deploy the highly available web service with shared storage:

(Optional) Step 1: Create an ECS instance and deploy a web service
Step 2: Deploy NAS for shared storage and disaster recovery
Step 3: Deploy CLB for high availability

(Optional) Step 1: Create an ECS instance

Important

If you already have an ECS instance with a web service configured, you can skip this step.
This step uses Alibaba Cloud Linux 3 as an example to demonstrate how to deploy a web service. The procedure may differ for other operating systems. For more tutorials on how to build a website, see Build a website.

Go to the instance creation page, select the Custom Launch tab, and follow the on-screen instructions to create an ECS instance. In this tutorial, the instance is named ECS_01.

Note the following parameters:
- Billing Method: spot instance.
  Important
  - To save costs for this tutorial, we use a spot instance, which offers a discount compared to pay-as-you-go instances. You can typically use a spot instance for at least one hour. After one hour, the instance might be automatically released if the market price exceeds your bid or if there is insufficient instance capacity. Make sure to back up your data.
  - If you need to use the ECS instance long-term, select subscription or pay-as-you-go for Billing Method.
- Region: Select a region close to your business. This tutorial uses China (Hangzhou) as an example.
- Image: Alibaba Cloud Linux 3.
- Public IP: Select Assign Public IPv4 Address.
- Security Group: Create a new security group and open ports 22 and 80 for IPv4 traffic.
- Logon Credential: Select Custom Password. For Logon Name, select ecs-user and set a password.
- Advanced Options - Resource Group: Create and select a Resource Group named High-availability Test Resource Group.
Configure other parameters as required. For more information, see Create an instance by using the custom launch tab.
Deploy the web service.
1. Connect to the ECS instance. For more information, see Connect to a Linux instance by using Workbench.
2. Run the following command to install the sample service.
```
curl -fsSL https://help-static-aliyun-doc.aliyuncs.com/tech-solution/install-ha-web.sh|bash
```
3. In your browser, enter the public IP address of the instance, such as http://123.1.XX.XX. If you can access the website, you have successfully deployed the web service.

Step 2: Deploy NAS

This solution uses NAS to provide shared storage. It also uses inotify-tools and rsync to implement the following features:

Incremental backup (Primary -> Backup): When data on the primary NAS file system changes, it is automatically synced to the backup NAS file system.
Primary-backup failover: The system automatically checks the availability of the primary NAS file system. If the primary system is available, it is used. Otherwise, the system fails over to the backup NAS file system. After the primary system recovers, the system automatically switches back.

Important

This disaster recovery solution for NAS is a basic example and has limitations. You must enhance it based on your specific business requirements.

For example, this solution is unidirectional (from the primary to the backup system). In a real-world scenario that involves both read and write operations, you must implement additional strategies for when the primary NAS file system fails:

If high write availability is not a critical requirement, you can disable write operations during the failover to the backup NAS file system.
If high write availability is required, do not immediately switch operations back to the primary NAS file system after it recovers. First, synchronize the new data generated on the backup system during the outage back to the primary system. Then, switch back. Ensure that no new file changes occur on the backup system during the switch-over, or that any such changes are also synchronized to the primary system.

Ensure that the instances to which you want to mount the file systems are in the Running state.

Log on to the NAS console.
In the left-side navigation pane, choose File System>File System List.
Create two NAS file systems in different zones.
1. Create a file system to be used as the primary NAS file system.
  
  Click Create File System, select General-purpose NAS, and then click Create General-purpose NAS. The NAS file system must meet the following requirements. For more information about the parameters, see Create a file system:
  - Region: Select the same region as the ECS instance (for example, China (Hangzhou)).
  - Zone: China (Hangzhou) Zone F.
  - Storage Type: Capacity.
  - Protocol Type: NFS.
  - VPC: Select the same VPC as the ECS instance.
2. Repeat the preceding step to create another NAS file system to be used as the backup NAS file system. Make sure this file system is in a different zone than the primary NAS file system, for example, China (Hangzhou) Zone K.

Mount both NAS file systems on the ECS instance.

This guide uses /nas_master for the primary NAS file system's mount path and /nas_backup for the backup's.

On the file system list, find the file system and click Attach in the **Actions** column.

Follow the on-screen instructions to complete the mount process.

Note the following parameters. Configure other parameters as required. For more information, see Mount an NFS file system by using the Apsara File Storage NAS console.

Parameter	Description	Example
Select ECS instance	The target ECS instance.	ECS_01
Mount path	The mount path on the ECS instance. Must be an absolute path.	/nas_master
Automatic mount	Select Auto Mount on Start-up so the file system mounts automatically on reboot.	Auto Mount on Start-up
Protocol type	Select NFSv3.	NFSv3

Repeat the preceding steps to mount the other file system on the ECS instance. Set the mount path to /nas_backup.

Move the files to be shared to the primary NAS file system's mount path (/nas_master).
1. Connect to the ECS instance. For more information, see Connect to a Linux instance by using Workbench.
2. Move the web application's root directory to the primary NAS mount path.
```
sudo cp -Lvr <absolute_path_of_shared_file> <nas_file_system_mount_path>
```
  Example: Move the html folder and its files for the website's frontend (located at /usr/share/nginx/html) to the NAS storage (mounted at /nas_master).
```
sudo cp -Lvr /usr/share/nginx/html /nas_master
```
Connect to the ECS instance. For more information, see Connect to a Linux instance by using Workbench.
Install inotify-tools and rsync.
```
sudo yum install inotify-tools rsync
```
Prepare the scripts to enable multi-AZ disaster recovery for NAS.
1. Download the scripts.
  - sync_nas.sh: Uses inotify and rsync to sync data from the primary NAS file system to the backup NAS file system when changes occur. You can adjust the script as needed.
  - sync_check_switch.sh: Checks the availability of the primary NAS file system and fails over to the backup if needed. You can adjust the script as needed.
```
sudo wget -P /etc/systemd/system/ https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20231017/pftz/sync_nas.sh
sudo wget -P /etc/systemd/system/ https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/en-US/20230925/wmaj/sync_check_switch.sh
```
2. Grant the execute permission to the scripts.
```
sudo chmod +x /etc/systemd/system/sync_nas.sh
sudo chmod +x /etc/systemd/system/sync_check_switch.sh
```

Configure the scripts to automatically start on system startup. This ensures that the NAS availability check and disaster recovery services can run properly after a server failure and recovery.

Create the sync-check-switch.service file.

sudo vi /etc/systemd/system/sync-check-switch.service

Add the following content to the file. Then, press Esc, type :wq, and press Enter to save and close the file.

[Unit]
Description=Sync Check Switch
After=network.target
[Service]
ExecStart=/etc/systemd/system/sync_check_switch.sh
RestartSec=3
Restart=always
[Install]
WantedBy=default.target

Create the sync-nas.service file.

sudo vi /etc/systemd/system/sync-nas.service

Add the following content to the file. Then, press Esc, type :wq, and press Enter to save and close the file.

[Unit]
Description=Sync NAS Service
After=network.target
[Service]
ExecStart=/etc/systemd/system/sync_nas.sh
Restart=always
RestartSec=3
[Install]
WantedBy=default.target

Run the following command to reload the systemd configuration.
```
sudo systemctl daemon-reload
```

Run the following commands to start the services and enable them to start on system startup.

sudo systemctl start sync-nas.service
sudo systemctl enable sync-check-switch.service
sudo systemctl start sync-check-switch.service
sudo systemctl enable sync-nas.service

Step 3: Deploy CLB

Replicate the web server instance

A high-availability architecture requires at least two ECS instances. You can create a custom image to quickly create a new instance that has the same configurations and data as the original instance, including the operating system, web server, and database.

Create a custom image from the web server instance (ECS_01).
1. Log on to the ECS console.
2. In the left-side navigation pane, choose Instances & Images > Instance.
3. In the top navigation bar, select the region where the target resource is located.
4. In the instance list, find the target instance. In the Actions column, choose > Disks and Images > Create Custom Image.
5. Enter an Image Name, select High-availability Test Resource Group for the Resource Group, and click OK.
6. In the left-side navigation pane, click Instances & Images > Images, and then select the Custom Images tab to view the image creation progress.
  
  Image creation takes some time. When the Progress of the target image reaches 100% and its status is Available, the image is created.
Use the custom image to create a new ECS instance (ECS_02).
1. In the left-side navigation pane, click Instances & Images > Images.
2. On the Custom Images tab, find the custom image that you created. In the Actions column, click Create Instance.
3. On the Custom Launch page, the Images section is already set to your selected custom image. Follow the on-screen instructions to complete the other configurations and purchase an ECS instance.
  Note the following parameters:
  - Billing Method: spot instance.
  - Region: Select the same region as the source instance. This tutorial uses China (Hangzhou).
  - Network and Zone: Select the same VPC as the source instance, but in a different zone.
  - Public IP Address: Deselect Assign Public IPv4 Addresses.
  - Security Group: Select the same security group as the source instance.
  - Logon Credential: Select Custom Password. For Logon Name, select ecs-user and set a password.
  - Advanced Options - Resource Group: Select High-availability Test Resource Group.
  Configure other parameters as required. For more information, see Create an instance by using the custom launch tab.

Configure the CLB instance

CLB eliminates single points of failure. If a server fails, CLB automatically routes requests to the remaining healthy servers, which improves service continuity and stability.

Log on to the Classic Load Balancer (CLB) console.
In the top navigation bar, select the resource group (for example, High-availability Test Resource Group) and the region.
On the Instances page, click Create CLB.

The CLB instance must meet the following requirements. Configure other parameters as required. For more information, see Create and manage a CLB instance.
- Billing Method: pay-as-you-go.
- Region: Must be in the same region as the ECS instances (for example, China (Hangzhou)).
- Instance Billing Method: Pay-by-traffic.
- Instance Type: Public.
- IP Version: IPv4.
- Resource Group: Select High-availability Test Resource Group.

After the instance is created, find it on the CLB Instances page. In the Actions column, click Configure Listener and follow the on-screen instructions.

Configure other parameters as required. For more information, see Add an HTTP listener.

Configuration Item	Requirements
Protocol & Listener	Listener Protocol: `HTTP`. Listening Port: `80`.
backend server	Select Default Server Group and click Continue to Add. Add the two ECS instances and set the port for both to 80.
Health check	We recommend that you enable health checks to monitor the health of your servers and determine their service availability.

Verify the solution

1. Simulate a server failure by stopping an ECS instance to verify service high availability

Stop an Elastic Compute Service (ECS) instance to simulate a failure and verify service availability.

Log on to the ECS console. Choose Overview. In the resource list, select an ECS instance, such as ECS_01, and click Stop.
After the ECS instance enters the Stopped state, enter the public IP address of the load balancer to access the website. If the website is accessible, the service is highly available.

Open the Classic Load Balancer (CLB) console. The health check for the CLB instance is abnormal, which indicates that a failure was detected on a backend server. This does not affect the user experience. In the listener list for the CLB instance, the health check status for port 80 of ECS_01 is Abnormal. This shows that CLB detected the unavailable backend server and automatically forwarded traffic to other healthy instances.

2. Test file synchronization to verify the shared storage feature

Open the ECS console. Start the ECS instance that you previously stopped, and then click Remote Connection. In the command line, run the following command: echo "Test content" > /nas_master/testfile.txt.
Remotely connect to the other ECS instance. To check if the file in the /nas_master path is synchronized, run the following command: cat /nas_master/testfile.txt.

[root@iZbp1a2rx5ap1jvrtr7bbqZ ~]# cat /nas_master/testfile.txt
Test_content

The Apsara File Storage NAS (NAS) file systems are mounted to the /nas_master and /nas_backup directories. If you update, add, or delete a file on one ECS instance, the file in the corresponding directory on the other ECS instance is also synchronized. If the files are synchronized, shared storage is successful. This ensures data consistency and sharing.

3. Perform a primary/standby switchover to verify the disaster recovery and high availability of NAS

Open the NAS console. Choose File Systems > File System List. Find the primary NAS file system and click Manage in the Actions column.

On the management page, click Mount & Use in the navigation pane on the left. Remove the current mount target. For more information, see Manage mount targets.

After about 4 seconds, enter the public IP address of the load balancer again to access the website. If the website is accessible, a successful switchover from the primary NAS file system to the backup NAS file system has occurred. This verifies the disaster recovery and high availability of NAS.

Important

NAS disaster recovery strategies must be customized for your business. This topic provides a simple example for reference only.
If you cannot remount the file system to the ECS instance after you remove the mount target, run the sudo umount -f <mount_address> command or restart the ECS instance. Then, add a mount target and mount the file system to the ECS instance. For more information, see How do I add a mount target?