Create application-consistent snapshots

更新时间:
复制 MD 格式

When you create a snapshot of a running ECS instance, data from applications such as databases may still be in memory or in the process of being written to storage. A standard snapshot cannot capture this in-flight data, which can lead to data loss or inconsistencies when you restore data from the snapshot. To address this issue, Alibaba Cloud provides the application-consistent snapshot feature. This feature integrates with Cloud Assistant to ensure your application data is consistent upon restoration.

Background information

When you create a snapshot-consistent group, you can enable the application-consistent snapshot feature. Based on your configuration, the system creates either an application-consistent snapshot or a file system-consistent snapshot. The following table describes the differences between these two types of snapshots.

Feature

Application-consistent snapshot

File system-consistent snapshot

Focus

Ensures consistency between stored data and the data within the application, such as a database.

Ensures data consistency at the file system level.

Implementation

  • Linux: Implemented by running custom shell scripts that you create to meet your application's requirements.

  • Windows: Implemented by using the built-in Volume Shadow Copy Service (VSS).

    Note

    Volume Shadow Copy Service (VSS) is a Microsoft Windows technology that creates snapshots of files or volumes for backup and recovery, even while they are in use. For more information, see the official Microsoft documentation for Volume Shadow Copy Service.

  • Linux: If you do not configure a shell script or if the script fails to run, a file system-consistent snapshot is created by default.

  • Windows: If a VSS Writer is not configured, a file system-consistent snapshot is created by default.

    Note

    A Writer is a key component in the VSS architecture. The Windows operating system includes many built-in Writers that ensure data consistency during a backup. For more information, see the official Microsoft documentation for Volume Shadow Copy Service.

Use cases

Ideal for applications that require strict data consistency, such as databases and mission-critical business systems.

Suitable for scenarios that require the overall integrity of the file system, such as for file servers or document management systems.

Snapshot tag

APPConsistent:True

FsConsistent:True

How it works

Application-consistent snapshots work as follows.

Linux instances

After you enable the application-consistent snapshot feature for a Linux instance, the system defaults to creating a file system-consistent snapshot. If you upload pre-configured shell scripts (a prescript.sh script and a postscript.sh script), the system runs them to perform an application consistency check. If the scripts run successfully, the snapshot is tagged as an application-consistent snapshot. Otherwise, it is tagged as a file system-consistent snapshot.

The following table describes the prescript.sh and postscript.sh scripts.

Name

Function

Type

Path

Permissions

Content

prescript.sh

Pauses application write I/O operations before creating a snapshot. This action ensures data remains consistent and is not altered during the snapshot process.

Shell

/tmp/prescript.sh

Only the root user has read, write, and execute permissions.

You must write the scripts based on your specific business requirements. For an example, see Step 1: Create the prescript.sh and postscript.sh scripts.

postscript.sh

Resumes application write I/O operations after the snapshot process is complete.

Shell

/tmp/postscript.sh

Important

The script names, script type, and upload paths for creating application-consistent snapshots are fixed. You must use these exact values when you create your custom scripts. Otherwise, the scripts may fail to run.

  1. For the target ECS instance, you enable the application-consistent snapshot feature and verify that Cloud Assistant is installed.

    Note

    If Cloud Assistant is not installed on the instance, the Cloud Assistant plug-in is automatically installed after you enable the application-consistent snapshot feature.

  2. Cloud Assistant accesses the ECS instance and automatically runs a sequence of commands: it runs the prescript.sh script, pauses I/O operations, and creates the snapshot.

    • If the script is correctly configured and runs successfully, an application-consistent snapshot is created.

    • If the script has an error, such as incorrect permissions, upload path, or script name, or fails to run, a file system-consistent snapshot is created instead.

  3. Cloud Assistant runs the postscript.sh script to resume I/O operations.

image

Windows instances

After you enable the application-consistent snapshot feature for a Windows instance, Cloud Assistant uses the built-in Volume Shadow Copy Service (VSS) to perform an application consistency check. If the check is successful, the snapshot is tagged as an application-consistent snapshot. Otherwise, it is tagged as a file system-consistent snapshot.

  1. For the target ECS instance, you enable the application-consistent snapshot feature and verify that Cloud Assistant is installed.

    Note

    If Cloud Assistant is not installed on the instance, the Cloud Assistant plug-in is automatically installed after you enable the application-consistent snapshot feature.

  2. Cloud Assistant accesses the ECS instance and automatically runs commands to pause and resume I/O operations, create the snapshot, and perform other related tasks.

    1. Cloud Assistant invokes the VSS service and pauses all I/O operations.

    2. The system checks whether default Writers are included in the process.

      • If yes, an application-consistent snapshot is created.

      • If no, a file system-consistent snapshot is created.

  3. I/O operations are resumed.

image

Limitations

  • You can create application-consistent snapshots only for ESSD cloud disks with the multi-attach feature disabled.

  • In the ECS console, you can create application-consistent snapshots for the cloud disks of only a single ECS instance at a time.

Prerequisites

  • Make sure that your ECS instance uses one of the following operating systems:

    • Windows: Windows Server 2012 R2 or later.

    • Linux: CentOS 7.6 or later, Ubuntu 18.04 or later, or Alibaba Cloud Linux 2.

  • The ECS instance is in the Running state and the Cloud Assistant client is running correctly. To learn how to check the client status, see View the status of the Cloud Assistant client and handle exceptions.

  • You have configured a RAM role and a custom policy with the required permissions for application-consistent snapshots and attached the role to the ECS instance. For more information, see Step 1: Create a RAM role.

    Note

    Because creating an application-consistent snapshot requires Cloud Assistant to access the ECS instance and run commands, you must grant it the necessary permissions through a RAM role.

    • RAM role: A custom role, for example, AppSnapshotRoleName.

    • Custom policy: The following policy grants permissions to query snapshots and disk information, create snapshots, and add tags to resources during the snapshot creation process.

      {
          "Version": "1",
          "Statement": [
              {
                  "Effect": "Allow",
                  "Action": [
                      "ecs:DescribeSnapshot*",
                      "ecs:CreateSnapshot*",
                      "ecs:TagResources",
                      "ecs:DescribeDisks"
                  ],
                  "Resource": [
                      "*"
                  ],
                  "Condition": {}
              }
          ]
      }

Procedure

Console

Linux instances

  1. Upload your prepared prescript.sh and postscript.sh scripts to the ECS instance.

    For more information, see Upload files to or download files from an instance by using Workbench.

    Note

    For more information about the prescript.sh and postscript.sh scripts, see How it works.

  2. Go to ECS console - Snapshot-consistent Groups.

  3. In the top navigation bar, select the region and resource group of the resource that you want to manage. Region

  4. On the Snapshot-consistent Groups tab, click Create Snapshot-consistent Group.

  5. In the Create Snapshot dialog box, configure the parameters for the snapshot-consistent group.

    1. For Resource Types, Instance is selected by default.

    2. Select an instance and the ESSD cloud disks of the instance.

    3. Expand Advanced Settings and configure application-consistent snapshot settings.

      • If you select Enable Application-consistent Snapshot and have correctly configured the scripts, an application-consistent snapshot is created.

      • If you select Enable Application-consistent Snapshot but have not configured the scripts or have configured them incorrectly, a file system-consistent snapshot is created.

        Note

        If Cloud Assistant is not installed on the instance, the Cloud Assistant plug-in is automatically installed after you select Enable Application-consistent Snapshot.

      • You can also select File System I/O Suspension and Resume and set a timeout period.

        Note

        If you enable the File System I/O Suspension and Resume feature, file system write activities are paused during snapshot creation to prevent data inconsistencies caused by ongoing write operations. For example, if you set a timeout of 16 seconds and the I/O suspension does not complete within that time, the suspension fails and may cause data inconsistency.

      • For information about other parameters, see Create a snapshot-consistent group.

    4. Click OK.

      After the snapshot is created, a Cloud Assistant command execution ID is returned. You can use this ID to view the creation result.

  6. Click the command execution ID to view the execution result on the Cloud Assistant page.

    image

    As shown in the figure, an Exit Code of 0 indicates that Cloud Assistant successfully ran the command to create the application-consistent snapshot. The command output includes the ID of the snapshot-consistent group.

    Note

    If the Exit Code is not 0, troubleshoot the issue based on the Exit Code information. For more information, see Error codes.

  7. View information about the created snapshot-consistent group and snapshot.

    1. Go to ECS console - Snapshot-consistent Groups.

    2. On the Snapshot-consistent Groups tab, find the group that you created and click its ID to view details.

    3. In the Snapshot Information section, check the snapshot's tags to determine whether it is an application-consistent or a file system-consistent snapshot.

      • The tag APPConsistent:True indicates that an application-consistent snapshot was created.

        image

      • The tag FsConsistent:True indicates that a file system-consistent snapshot was created.

        image

Windows instances

  1. Go to ECS console - Snapshot-consistent Groups.

  2. In the top navigation bar, select the region and resource group of the resource that you want to manage. Region

  3. On the Snapshot-consistent Groups tab, click Create Snapshot-consistent Group.

  4. In the Create Snapshot dialog box, configure the parameters for the snapshot-consistent group.

    1. For Resource Types, Instance is selected by default.

    2. Select an instance and the ESSD cloud disks of the instance.

    3. Expand Advanced Settings and configure application-consistent snapshot settings.

      • To create an application-consistent snapshot, select both Enable Application-consistent Snapshot and Contain Writers by Default. The Contain Writers by Default option becomes visible only after you select Enable Application-consistent Snapshot.

      • If you select only Enable Application-consistent Snapshot, a file system-consistent snapshot is created.

        Note

        If Cloud Assistant is not installed on the instance, the Cloud Assistant plug-in is automatically installed after you select Enable Application-consistent Snapshot.

      • For information about other parameters, see Create a snapshot-consistent group.

    4. Click OK.

      After the snapshot is created, a Cloud Assistant command execution ID is returned.

  5. Click the command execution ID to view the execution result on the Cloud Assistant page.

    image

    As shown in the figure, an Exit Code of 0 indicates that Cloud Assistant successfully ran the command to create the application-consistent snapshot. The command output includes the ID of the snapshot-consistent group.

    Note

    If the Exit Code is not 0, troubleshoot the issue based on the Exit Code information. For more information, see Error codes.

  6. View information about the created snapshot-consistent group and snapshot.

    1. Go to ECS console - Snapshot-consistent Groups.

    2. On the Snapshot-consistent Groups tab, find the group that you created and click its ID to view details.

    3. In the Snapshot Information section, check the snapshot's tags to determine whether it is an application-consistent or a file system-consistent snapshot.

      • The tag APPConsistent:True indicates that an application-consistent snapshot was created.

        image

      • The tag FsConsistent:True indicates that a file system-consistent snapshot was created.

        image

API

  1. Call the RunCommand API operation to create application-consistent snapshots for one or more ECS instances.

    You need to set parameters such as RegionId, Type, CommandContent, and InstanceId based on your actual environment. An example and description of the CommandContent parameter are as follows:

    Linux instances

    acs-plugin-manager --exec --plugin app-snapshot-plugin --params=-RamRoleName=\"AppSnapshotRoleName\",-EnableFsFreeze=true,-TimeoutInSeconds=30,-PreScriptPath=\"/tmp/prescript.sh\",-PostScriptPath=\"/tmp/postscript.sh\",-ExcludeDiskId=\"\",-Name=\"LinuxApp1\",-Description=\"LinuxApp\"

    • acs-plugin-manager --exec --plugin app-snapshot-plugin: Executes the Cloud Assistant plug-in app-snapshot-plugin.

    • --params=: Specifies the configuration parameters for the plug-in. The following table describes the parameters.

      Parameter

      Type

      Required

      Description

      RamRoleName

      String

      Yes

      The RAM role that is attached to the ECS instance.

      EnableFsFreeze

      Boolean

      No

      Specifies whether to use the Linux fsfreeze mechanism to place the file system in a read-only consistent state before snapshot creation.

      Default value: True.

      TimeoutInSeconds

      Integer

      No

      The timeout period for I/O operations.

      Default value: 30 seconds.

      PreScriptPath

      String

      No

      The path to the prescript.sh script (/tmp/prescript.sh). The prescript.sh script must meet the following conditions:

      • Permissions: Only the root user has read, write, and execute permissions (permission code 700).

      • Content: The script content must be customized for your application.

      Important

      This parameter is required when you create an application-consistent snapshot for a Linux instance. If the script is configured incorrectly, for example, with incorrect permissions, path, or filename, a file system-consistent snapshot is created instead.

      PostScriptPath

      String

      No

      The path to the postscript.sh script (/tmp/postscript.sh). The postscript.sh script must meet the following conditions:

      • Permissions: Only the root user has read, write, and execute permissions (permission code 700).

      • Content: The script content must be customized for your application.

      Important

      This parameter is required when you create an application-consistent snapshot for a Linux instance. If the script is configured incorrectly, for example, with incorrect permissions, path, or filename, a file system-consistent snapshot is created instead.

      ExcludeDiskId

      String

      No

      The IDs of cloud disks on the ECS instance that you want to exclude from the snapshot.

      Name

      String

      Yes

      The name of the snapshot-consistent group.

      Description

      String

      No

      The description of the snapshot-consistent group.

    Windows instances

    acs-plugin-manager --exec --plugin app-snapshot-plugin-win --params=-RamRoleName=\"AppSnapshotRoleName\",-EnableWriters=true,-ExcludeDiskId=\"\",-Name=\"APPSnapshot-1\",-Description=\"AppSnapshot\"

    • acs-plugin-manager --exec --plugin app-snapshot-plugin-win: Executes the Cloud Assistant plug-in app-snapshot-plugin-win.

    • --params=: Specifies the configuration parameters for the plug-in. The following table describes the parameters.

      Parameter

      Type

      Required

      Description

      RamRoleName

      String

      Yes

      The RAM role that is attached to the ECS instance.

      EnableWriters

      Boolean

      No

      Specifies whether to create an application-consistent snapshot. Valid values:

      • true: Creates an application-consistent snapshot.

      • false: Creates a file system-consistent snapshot.

      Default value: true.

      ExcludeDiskId

      String

      No

      The IDs of cloud disks on the ECS instance that you want to exclude from the snapshot.

      Name

      String

      Yes

      The name of the snapshot-consistent group.

      Description

      String

      No

      The description of the snapshot-consistent group.

  2. Based on the returned InvokeId, call the DescribeInvocationResults API operation.

  3. Check whether the application-consistent snapshot was created successfully.

    • ExitCode returns the error code.

      • A value of 0 indicates that the snapshot was created successfully.

      • If the value is not 0, troubleshoot the issue based on the error code information. For more information, see Error code information.

    • Output is the information returned after the command is run. If the creation is successful, the output contains information such as the snapshot-consistent group ID.

      [time=\"2025-03-04 16:09:25.8200239\"][message=\"Finish SnapshotGroup=ssg-2zefohc25d7n1grq**** Creation, TotalCost=3.8204978s, QueryCost=2.9307022s\"]\n[level=\"info\"][time=\"2025-03-04 16:09:25.8200239\"][message=\"Prepare to Thaw FileSystem or Applications\"]\n[requestor=\"\n\"]\n[level=\"info\"][time=\"2025-03-04 16:09:27.7133096\"][message=\"Thaw Write Request Done\"]\n[level=\"info\"][time=\"2025-03-04 16:09:28.0355042\"][message=\"Tag snapshots with AppConistent\"]\n[level=\"info\"][time=\"2025-03-04 16:09:28.0365941\"][message=\"Take AppConsistent snapshots successfully\"]

Error codes

Error code

Description

0

The snapshot was created successfully.

1

A condition check failed. Possible causes include:

  • The cloud disk type is not supported.

  • The snapshot name does not meet requirements.

  • A network connection issue occurred.

  • The ECS instance lacks the required RAM role authorization.

  • The operating system version is not supported.

2

The type or number of parameters specified after --params is incorrect.

3

Possible causes include:

  • No ESSD cloud disks are attached to the ECS instance.

  • The instance's RAM role lacks the required permissions for snapshot-related API operations.

4

Failed to create the snapshot-consistent group.

5

The snapshot-consistent group is in an invalid state.

6

Timed out while creating the snapshot-consistent group.

7

An individual cloud disk snapshot within the snapshot-consistent group is in an unexpected state.

8

Failed to add a tag to the snapshot.

9

Failed to run the prescript.sh script.

10

Failed to run the postscript.sh script.

11

Failed to pause I/O operations.

12

Failed to resume I/O operations.

13

The ECS instance lacks the required RAM role authorization.

14

The number of snapshots exceeds the limit.

15

The snapshot is in an invalid state.

16

A snapshot creation is already in progress. You cannot create another snapshot until it completes.

255

An unknown failure occurred.

Related topics

To create application-consistent snapshots for MySQL or SQL Server databases, see Best practices for creating application-consistent snapshots for MySQL (Linux) and Best practices for creating application-consistent snapshots for SQL Server (Windows).