Simulate high disk utilization

Updated at:

Fill a disk on an ECS instance to a target utilization and verify system stability under high disk usage.

How it works

The Cloud Assistant plug-in ACS-ECS-FillDisk uses fallocate to generate a temporary file that fills the disk to the specified utilization. You must manually delete the temporary file to recover.

Procedure

Prerequisites

Fault injection

  1. Log on to the ECS instance. See Log on to a Linux instance using Workbench.

  2. Run the ACS-ECS-FillDisk plug-in as a sudo user.

    sudo acs-plugin-manager --exec --plugin ACS-ECS-FillDisk --params inject,[dir=paramA],[percent=paramB]

    Parameters in brackets ([]) are optional:

    • dir: the target directory. Default value: /.

    • percent: the target disk utilization. Default value: 80%.

    Note

    If fallocate fails due to insufficient space or other reasons, the generated temporary file is deleted.

    The following output indicates that the ACS-ECS-FillDisk plug-in ran successfully.

    [root@iZ0jlj5r9xxx            /]# sudo acs-plugin-manager --exec --plugin ACS-ECS-FillDisk --params inject,dir=/usr,percent=80
    params: dir:/usr, percent:80
    Will attempt to fallocate 21.39 GB...
    Filling up disk space. Temporary file: /usr/AliFaultFillDisk.tmp
    Done
  3. Run df {dir} to verify the fault injection.

    The following output indicates that the injection succeeded.

    [root@iZxxxZ /]# df /usr/
    Filesystem     1K-blocks     Used Available Use% Mounted on
    /dev/vda3      41828332 33462628   8365704  80% /
    [root@iZxxxZ /]#

Fault recovery

Log on to the ECS instance and run rm -f {dir}/AliFaultFillDisk.tmp to delete the temporary file. Replace dir with the path specified during injection.

Next steps

  • Resize disks whose capacity is insufficient.

  • In Container Service for Kubernetes (ACK) scenarios, automatically extend disk volumes when utilization exceeds a threshold. See Configure automatic expansion.