Local disks do not provide high data availability. To help you promptly identify and respond to issues, Alibaba Cloud provides several maintenance features for instances with local disks. This topic describes common maintenance scenarios and their related system events.
View and monitor system events
View system events for instances with local disks:
View events in the ECS console or by using Alibaba Cloud CLI. For more information, see Query and handle events.
View events in the CloudMonitor console. For more information, see View system events.
Monitor system events for instances with local disks:
To ensure the stability of services on your ECS instances and to automate maintenance, set up event notifications to monitor changes in the underlying environment. After you configure event notifications, you will receive messages through your specified notification methods.
Configure alert rules in the CloudMonitor console to receive event notifications. For more information, see Subscribe to system event notifications.
Use a DingTalk chatbot to send event notifications. For more information, see Send event notifications with a DingTalk chatbot.
For ECS Bare Metal instances, you can also install the xdragon_hardware_detect_plugin plug-in to periodically check the health of local disks. For more information, see Install the monitoring plug-in.
Common maintenance scenarios and related system events
The following figure shows common maintenance scenarios for instances with local disks and their related system events.
You can customize the automatic recovery method for an instance by modifying its maintenance attributes. For example, if an instance's maintenance attribute is set to automatic redeployment, the instance enters a redeployment scenario by default during automatic recovery. For more information, see Modify instance maintenance attributes.
For more information about the system events in the preceding scenarios, see the following sections:
To avoid business disruptions, ensure service availability and back up your data before you perform maintenance tasks. For example, you can switch traffic at the application layer, remove the ECS instance from an SLB instance, and back up data on your disks.
Scenario ①
The following workflow describes how to handle a SystemMaintenance.Reboot event:
You receive a notification that your instance is scheduled for a restart.
Respond to the event as needed.
If you do not want the instance to be restarted during the scheduled time window, modify the scheduled restart time. For more information, see Modify scheduled restart time.
Restart the instance within the user operation window. For more information, see Restart an instance.
NoteYou must restart the instance in the ECS console or by calling the RebootInstance API. Restarting the instance from within the operating system has no effect.
Wait for the system to automatically restart the instance.
Check whether the instance and your applications are working as expected.
For more information about the event states supported by SystemMaintenance.Reboot, see System event summary. For a diagram that shows typical transitions between event states, see States and windows of system events.
Scenario ②
The following workflow describes how to handle a SystemMaintenance.Redeploy event:
You receive a notification that an instance with a local disk is scheduled to be redeployed.
Complete the prerequisite tasks, such as modifying the /etc/fstab file and backing up data.
For more information about the prerequisites, see the Prerequisites section of Redeploy instance.
Respond to the event.
After you back up your data, respond to the event. This action automatically triggers the instance migration and local disk replacement. For more information, see Redeploy instance.
NoteAfter you redeploy an instance with a local disk, the instance is migrated to a new host and its local disk is re-initialized. All data on the local disk is erased.
Check whether the instance and your applications are working as expected, and synchronize data as needed.
For more information about the event states supported by SystemMaintenance.Redeploy, see System event summary. For a diagram that shows typical transitions between event states, see States and windows of system events.
Scenario ③
The following workflow describes how to handle a SystemFailure.Reboot event:
The system automatically restarts the instance.
You receive a notification that the instance is restarting.
The instance is automatically restarting. No action is required. Wait for the process to complete.
Check whether the instance and your applications are working as expected.
For more information about the event states supported by SystemFailure.Reboot, see System event summary. For a diagram that shows typical transitions between event states, see States and windows of system events.
Scenario ④
The following workflow describes how to handle a SystemFailure.Redeploy event:
You receive a notification that an instance with a local disk is scheduled to be redeployed.
Complete the prerequisite tasks, such as modifying the /etc/fstab file and backing up data.
For more information about the prerequisites, see the Prerequisites section of Redeploy instance.
Respond to the event as needed.
After you back up your data, respond to the event. This action automatically triggers the instance migration and local disk replacement. For more information, see Redeploy instance.
NoteAfter you redeploy an instance with a local disk, the instance is migrated to a new host and its local disk is re-initialized. All data on the local disk is erased.
Check whether the instance and your applications are working as expected, and synchronize data as needed.
For more information about the event states supported by SystemFailure.Redeploy, see System event summary. For a diagram that shows typical transitions between event states, see States and windows of system events.
Scenario ⑤
In Scenario ⑤, you can redeploy the instance to another host or replace only the damaged disk. When you replace a damaged disk, note the following:
Not all disks on these instances can be isolated. You can isolate a damaged disk only when the operation for the system event includes disk isolation.
Disk isolation and disk repair are independent operations. Disk isolation is a prerequisite for disk repair, but does not guarantee that a repair is possible. Not all instances support local disk repair. You can initiate a repair only after you receive a disk recovery notification from Alibaba Cloud.
Redeploying the instance quickly restores local disk availability, but erases all data on the local disks. For more information, see Redeploy instance.
Replacing a damaged disk preserves the data on other local disks. The maintenance workflow is as follows:
You receive notifications about a disk failure and a scheduled isolation of the damaged disk.
Complete the prerequisite tasks, such as modifying the /etc/fstab file and backing up data.
If the system event includes a damaged disk isolation event or operation, respond to the notification to authorize the isolation.
If the system event includes a Reboot event or operation, you must restart the instance.
Alibaba Cloud removes the damaged disk from the host, inserts a new one, and then sends a disk recovery notification.
If the system event includes a disk recovery event or operation, respond to the notification to authorize the disk recovery.
If the system event includes a Reboot event or operation, you must restart the instance.
NoteReplacing a damaged disk requires cooperation between you and Alibaba Cloud. For more information, see Isolate/repair local disks and Isolate damaged local disks (CLI).
The following figure shows the supported event states for system events related to damaged disk replacement and the transitions between them.
Scenario ⑥
In Scenario ⑥, you can redeploy the instance to another host or perform an in-place repair. When you perform an in-place repair, note the following:
An in-place repair cannot guarantee 100% data integrity or a 100% success rate. Back up critical business data before you authorize the repair.
Stop-and-repair is not supported on all instances with local disks.
During the repair period, the instance cannot be started, but billing continues according to its billing method.
The in-place repair cycle is 14 business days. During this period, you can terminate the repair process by redeploying or releasing the instance.
Redeploying the instance quickly restores local disk availability, but erases all data on the local disks. For more information, see Redeploy instance.
The workflow to authorize a repair is as follows:
You receive a system event notification about an in-place repair for your instance with a local disk.
Respond to the event as needed.
Within the user operation window, stop the instance and authorize the in-place repair.
Wait for the system to automatically stop the instance and perform the repair.
Alibaba Cloud repairs the hardware on the host. After the repair is complete, Alibaba Cloud sends a repair completion event.
Check whether the instance and your applications are working as expected, and synchronize data as needed.
For more information about the event states supported by SystemMaintenance.StopAndRepair, see System event summary. For a diagram that shows typical transitions between event states, see States and windows of system events.
References
You can call the AcceptInquiredSystemEvent API to accept the default operation for a system event and authorize the system to perform it.