Fault drills are essential before you launch a business system. You can use them to test and improve your emergency plans. Drills help increase the stability of products, clusters, and data centers. They also reduce faults and improve emergency response efficiency, making your product more competitive.
A complete fault drill process consists of the following steps: Create an atomic fault service > Create a drill scenario > Create and publish a drill plan > Create and run a drill execution > View the drill report > Export the drill plan.
The drill scenario module lets you design and manage custom fault drill scenarios based on your disaster recovery needs. The module provides out-of-the-box templates built on atomic capabilities.
Create a drill scenario
When you configure each step of a drill scenario, you can specify whether to run the step automatically during the drill.
Procedure:
Log on to the High Availability (HA) Management Console.
In the navigation pane on the left, click Fault Drill > Drill Scenarios.
Click the Create Drill Scenario button to open the creation page.
After you configure the basic information, click Next.
Scenario Name: Enter a custom name for the scenario. This parameter is required.
Tag: Select the tag to associate.
Scenario Description: Enter a description of the fault scenario.
Expected Goal: Describe the potential impact of the fault.
Problem Severity: Select a severity level.
NoteFor problem severity, P0 indicates the highest severity and P4 indicates the lowest.
Associated Risk Scenarios: Select associated risk scenarios. To associate multiple risk scenarios, click Add.
(Optional) After you configure the pre-checks, click Next.
Specify whether to run this step automatically during the drill. By default, this step runs automatically. To disable autorun, set Autorun to No.
Add check items. You can add multiple check items using one of the following two methods: inspection-based checks or test image-based checks.
If you select Inspection-based Check, configure the following parameters:
Inspection Rule: The rule to run for the check.
Execution Data Center: The data center where the inspection rule runs.
Expected Result: The expected result of the inspection, either pass or fail.
If you select Test Image-based Check, configure the following parameters:
Image Product: Select a test image. The image product must be the same as the product selected in the basic information.
Expected Result: The expected result of the inspection, either pass or fail.
Inject the fault. After you complete the following configurations, click Next.
Specify whether to run this step automatically during the drill. By default, this step runs automatically. To disable autorun, set Autorun to No.
Click Add Fault Injection and configure the fault injection.
Fault Node Type: You can select Ant Product, User Application, AKE Host Node, or Custom Server.
If you select Ant Product, configure the following parameters.
Parameter
Description
Fault Injection Product
Select the product to inject the fault into.
Application
Select the application to inject the fault into.
IP List
Specify the IP addresses for the drill. Use commas to separate multiple IP addresses.
NoteIf you specify IP addresses, you cannot sync this drill scenario across multiple environments.
Atomic Operation
Select an atomic operation as needed.
ImportantMake sure the fault injection component is installed in the cluster. You can install it from Container Service for Kubernetes (AKS) > Cluster Management > Component Management.
If you select User Application, configure the following parameters.
Parameter
Description
Fault Injection Application
Select the application to inject the fault into. You can select multiple applications.
Workspace Group
Select the workspace group where the application is located. You can select multiple workspace groups.
Namespace
Select the namespace where the application is located. You can select multiple namespaces.
Application Service
Select one or more application services.
IP List
Specify the IP addresses for the drill. Use commas to separate multiple IP addresses.
NoteIf you specify IP addresses, you cannot sync this drill scenario across multiple environments.
label
Specify the label for the physical machine.
Atomic Operation
Select an atomic operation as needed.
ImportantMake sure the fault injection component is installed in the cluster. You can install it from Container Service for Kubernetes (AKS) > Cluster Management > Component Management.
If you select AKE Host Node or Custom Server, configure the following parameters.
Configuration setting
Description
IP List
Specify the IP addresses for the drill. Use commas to separate multiple IP addresses.
NoteIf you specify IP addresses, you cannot sync this drill scenario across multiple environments.
Atomic Operation
Select an atomic operation as needed.
ImportantMake sure the fault injection component is installed in the cluster. You can install it from Container Service for Kubernetes (AKS) > Cluster Management > Component Management.
Atomic Operation Settings: Set the parameter values for the selected atomic operation.
(Optional) Configure fault recovery. After you complete the following configurations, click Finish.
Specify whether to run this step automatically during the drill. By default, this step does not run automatically. To enable autorun, set Autorun to Yes.
Select an emergency plan.
More operations
Modify a drill scenario
You can modify a drill scenario in the following ways:
On the Drill Scenarios page, click Edit in the Actions column of the target scenario. The subsequent steps are the same as those for creating a drill scenario.
Click the name of the target drill scenario to go to its details page. Then, click Edit in the upper-right corner of the page to modify the steps in the drill scenario.
Clone a drill scenario
Cloning a drill scenario means copying an existing scenario to create a new one.
On the Drill Scenarios page, find the target drill scenario, click 
in the Actions column, and select Clone. Then, modify the configuration of the cloned drill scenario as needed. The modification process is the same as that for editing a drill scenario.
Abandon a drill scenario
You can deprecate drill scenarios that you no longer use.
On the Drill Scenarios page, find the target drill scenario, click in the Actions column, and select Deprecate. After a drill scenario is deprecated, it remains in the list of drill scenarios, but its status changes to Deprecated.
If you are sure you no longer need a deprecated drill scenario, you can delete it. For more information, see Delete a drill scenario.
Delete a drill scenario
A drill scenario cannot be recovered after it is deleted. Proceed with caution.
You can delete a drill scenario as follows:
On the Drill Scenarios page, find the target drill scenario, click in the Actions column, and select Delete. After the drill scenario is deleted, it is removed from the drill scenario list.
More operations
On the Drill Scenarios page, you can also quickly create a drill plan from a scenario. The steps are as follows:
On the Drill Scenarios page, find the target drill scenario, click
in the Actions column, and select Create Drill Plan.On the Create Drill Plan page, enter a name and description for the drill plan. For more information, see Create a drill plan.