Simulate a downtime drill
Simulate kernel faults on an ECS instance to test recovery, verify monitoring alerts, and develop response strategies.
How it works
The ecs-fault-oscrash Cloud Assistant plugin triggers a kernel panic through the sysrq module. The system then automatically restarts and resumes normal operation.
This drill may affect your business. Ensure your system has high availability (HA) capabilities and kdump is enabled. See How do I enable the kdump service on a Linux instance?
Procedure
Prerequisites
-
Cloud Assistant Agent is installed on the target ECS instance.
-
Cloud Assistant status is Normal on the instance. See Check Cloud Assistant status and troubleshoot exceptions.
Inject a fault
-
Connect to the ECS instance as a user with sudo privileges. See Log on to a Linux instance using Workbench.
-
Run the
ecs-fault-oscrashCloud Assistant plugin:sudo acs-plugin-manager --exec --plugin ecs-fault-oscrash --params injectThe following output indicates that the
ecs-fault-oscrashplugin ran successfully.
-
Verify the fault injection.
-
If an Instance Restart Due To Instance Error event appears in unexpected O&M events, the fault was injected.

-
Run the
uptimecommand on the instance to check the system runtime and confirm whether a restart occurred.The following output shows the instance restarted at 18:21:46, confirming fault injection.

-
Recover from the fault
The ECS instance automatically restarts and resumes normal operation after the drill. If the instance fails to restart, force restart it in the ECS console. See Restart an instance.