Automatic backup and restoration

Enable automatic backup and configure cluster alerts

View backup snapshots

After enabling automatic backup, use the _snapshot API in the Kibana console to view snapshots. Connect to a cluster through Kibana.

• View the snapshot repository.

  GET _snapshot

  Example response:

  {
    "aliyun_auto_snapshot" : {
      "type" : "oss",
      "settings" : {
        "compress" : "true",
        "base_path" : "cf95b9-185320276651****/es-cn-09k2053us0003****",
        "endpoint" : "http://oss-cn-hangzhou-internal.aliyuncs.com"
      }
    }
  }

  Parameter	Description
  aliyun_auto_snapshot	The automatic snapshot repository, created during the first snapshot. Always named aliyun_auto_snapshot. A repository stores multiple snapshots, each containing backup data for all, some, or a single index.
  type	Storage medium. oss indicates Object Storage Service (OSS).
  compress	Whether to compress snapshot metadata files (index mappings, settings). Does not affect data files. Default: false.
  base_path	Storage path in OSS.
  endpoint	OSS bucket endpoint for the instance region.

• View information about all snapshots in the aliyun_auto_snapshot repository.

  GET _snapshot/aliyun_auto_snapshot/_all

  Example response:

  {
    "snapshots": [
      {
        "snapshot": "es-cn-09k2053us0003****_20210117030003",
        "uuid": "vIdSCkthTeGa0nSj4D****",
        "version_id": 5050399,
        "version": "5.5.3",
        "indices": [
          ".kibana"
        ],
        "state": "SUCCESS",
        "start_time": "2018-06-28T01:22:39.609Z",
        "start_time_in_millis": 1530148959609,
        "end_time": "2018-06-28T01:22:39.923Z",
        "end_time_in_millis": 1530148959923,
        "duration_in_millis": 314,
  
        "failures": [ ],
  
        "shards": {
          "total": 1,
          "failed": 0,
          "successful": 1
        }
      },
      {
        "snapshot": "es-cn-09k2053us0003****_20210118030004",
        "uuid": "XKO_Uwz_Qu6mZrU3Am****",
        "version_id": 5050399,
        "version": "5.5.3",
        "indices": [
          ".kibana"
        ],
        "state": "SUCCESS",
        "start_time": "2018-06-28T01:25:00.764Z",
        "start_time_in_millis": 1530149100764,
        "end_time": "2018-06-28T01:25:01.482Z",
        "end_time_in_millis": 1530149101482,
        "duration_in_millis": 718,
  
        "failures": [ ],
  
        "shards": {
          "total": 1,
          "failed": 0,
          "successful": 1
        }
      }
    ]
  }

  Automatic backups are scheduled in the local time of the instance's region, but the API response returns timestamps in UTC. China Standard Time (UTC+8) is 8 hours ahead of UTC.

  The state field indicates snapshot status:

  Snapshot status	Description
  IN_PROGRESS	Snapshot in progress.
  SUCCESS	Snapshot completed. All shard data stored.
  FAILED	Snapshot failed.
  PARTIAL	Partial success. At least one shard was not stored.
  INCOMPATIBLE	Snapshot incompatible with the Alibaba Cloud Elasticsearch instance version.

  Automatic backup snapshots have these additional default parameters:

  Parameter	Description
  max_snapshot_bytes_per_sec	Maximum snapshot throughput per node. Default: 40 MB/s.
  max_restore_bytes_per_sec	Maximum restore throughput per node. Default: 40 MB/s.
  chunk_size	Chunk size for splitting large files during snapshot (for example, 1g, 10m, 5k). Default: null (no limit).

• View detailed information about a specific snapshot in the aliyun_auto_snapshot repository.

  GET _snapshot/aliyun_auto_snapshot/<snapshot>/_status

  <snapshot>: Name of the automatic backup snapshot (from the all-snapshots command above). Example: es-cn-09k2053us0003****_20210118030004.

  Example response:

  {
    "snapshots": [
      {
        "snapshot": "es-cn-09k2053us0003****_20210118030004",
        "repository": "aliyun_auto_snapshot",
        "uuid": "XKO_Uwz_Qu6mZrU3Am****",
        "state": "SUCCESS",
        "shards_stats": {
          "initializing": 0,
          "started": 0,
          "finalizing": 0,
          "done": 1,
          "failed": 0,
          "total": 1
        },
        "stats": {
          "number_of_files": 4,
          "processed_files": 4,
          "total_size_in_bytes": 3296,
          "processed_size_in_bytes": 3296,
          "start_time_in_millis": 1530148959688,
          "time_in_millis": 77
        },
        "indices": {
          ".kibana": {
            "shards_stats": {
              "initializing": 0,
              "started": 0,
              "finalizing": 0,
              "done": 1,
              "failed": 0,
              "total": 1
            },
            "stats": {
              "number_of_files": 4,
              "processed_files": 4,
              "total_size_in_bytes": 3296,
              "processed_size_in_bytes": 3296,
              "start_time_in_millis": 1530148959688,
              "time_in_millis": 77
            },
            "shards": {
              "0": {
                "stage": "DONE",
                "stats": {
                  "number_of_files": 4,
                  "processed_files": 4,
                  "total_size_in_bytes": 3296,
                  "processed_size_in_bytes": 3296,
                  "start_time_in_millis": 1530148959688,
                  "time_in_millis": 77
                }
              }
            }
          }
        }
      }
    ]
  }

Restore data from an automatic backup

Restoring system indexes (prefix .) may cause Kibana to become inaccessible. Do not restore system indexes.

In the Kibana console, run the following commands to restore index data from a snapshot:

• Restore all indexes from a specific snapshot (this runs in the background).

  POST _snapshot/aliyun_auto_snapshot/<snapshot>/_restore

  <snapshot>: The name of the automatic backup snapshot, for example, es-cn-abcdefghij****_20180627091600.

• Restore all indexes from a specific snapshot and wait for the task to complete.

  The _restore API is asynchronous. Add wait_for_completion to wait for completion:

  POST _snapshot/aliyun_auto_snapshot/<snapshot>/_restore?wait_for_completion=true

  <snapshot>: The name of the automatic backup snapshot, for example, es-cn-abcdefghij****_20180627091600.

• Restore specific indexes from a snapshot and rename them during the process (this runs in the background).

  POST _snapshot/aliyun_auto_snapshot/<snapshot>/_restore
  {
  "indices": "index_1",
  "rename_pattern": "index_(.+)",
  "rename_replacement": "restored_index_$1"
  }

  Parameter	Description
  <snapshot>	The name of the automatic backup snapshot, for example, es-cn-abcdefghij****_20180627091600.
  indices	The name of the index to restore.
  rename_pattern	Optional. A regular expression to match the names of the indexes you want to restore.
  rename_replacement	Optional. The replacement pattern for renaming the matched indexes.

Disable automatic backup

1. Log on to the Alibaba Cloud Elasticsearch console.

2. In the left-side navigation pane, click Elasticsearch Clusters.

3. In the top menu bar, select a resource group and a region.

4. On the Elasticsearch Clusters page, click the ID of the target instance.

5. In the left-side navigation pane, click Data Backup.

6. In the Snapshots (Free Trial) area, turn off the Auto Snapshot switch.

FAQ