Migrate Elasticsearch data with elasticsearch-dump-Elasticsearch(ES)-阿里云帮助中心

Use the open-source elasticsearch-dump tool to migrate index settings, mappings, and data between Alibaba Cloud Elasticsearch clusters or between on-premises and cloud environments.

Overview

elasticsearch-dump is a command-line tool that exports data from one Elasticsearch index into another index or a local file.

Best for:

Migrating small volumes of data.
Moving specific indexes.
Backing up mappings or settings to local JSON files.

Official documentation: elasticdump.

Prerequisites

Source/Destination clusters: Alibaba Cloud Elasticsearch clusters must be created. See Create an Alibaba Cloud Elasticsearch cluster.
Auto-indexing: The destination cluster must have Auto Indexing enabled, or the target index must be created manually in advance. Configure the YML file.
Migration node: An Elastic Compute Service (ECS) is required to run the tool.
- If the ECS instance and Elasticsearch cluster are in the same Virtual Private Cloud (VPC), use internal endpoints for faster, free data transfer.
- If they are in different regions/VPCs, use public endpoints and ensure the ECS IP is added to the cluster whitelist. See Manage IP address whitelists.

Install elasticsearch-dump

Connect to the ECS instance.

Connect to a Linux instance by using a password or key.

Install Node.js.

Download the package:

wget https://nodejs.org/dist/v16.18.0/node-v16.18.0-linux-x64.tar.xz

Decompress and move:
```
tar -xf node-v16.18.0-linux-x64.tar.xz
```

Configure environment variables:

Temporary (current session only):

export PATH=$PATH:/root/node-v16.18.0-linux-x64/bin/

Permanent (add to shell profile):

vim ~/.bash_profile
export PATH=$PATH:/root/node-v16.18.0-linux-x64/bin/
source ~/.bash_profile

Install elasticsearch-dump:
```
npm install elasticdump -g
```

Examples

Important

If your password contains special characters (e.g., #, $, @), standard URL strings may fail. See the FAQ and troubleshooting section for the --httpAuthFile solution.

Migrate data between clusters (cloud-to-cloud)

To fully migrate an index, run the command for settings, mappings, and data in that order.

Migrate index settings:

elasticdump --input=http://"<UserName>:<YourPassword>"@<YourEsHost>/<YourEsIndex> --output=http://"<OtherName>:<OtherPassword>"@<OtherEsHost>/<OtherEsIndex> --type=settings

Migrate mappings:

elasticdump --input=http://"<UserName>:<YourPassword>"@<YourEsHost>/<YourEsIndex> --output=http://"<OtherName>:<OtherPassword>"@<OtherEsHost>/<OtherEsIndex> --type=mapping

Migrate documents (data)

elasticdump --input=http://"<UserName>:<YourPassword>"@<YourEsHost>/<YourEsIndex> --output=http://"<OtherName>:<OtherPassword>"@<OtherEsHost>/<OtherEsIndex> --type=data

Export to local file (backup)

Migrate settings

elasticdump --input=http://"<UserName>:<YourPassword>"@<YourEsHost>/<YourEsIndex> --output=<YourLocalFile> --type=settings

Migrate mappings

elasticdump --input=http://"<UserName>:<YourPassword>"@<YourEsHost>/<YourEsIndex> --output=<YourLocalFile> --type=mapping

Migrate documents (data)

elasticdump --input=http://"<UserName>:<YourPassword>"@<YourEsHost>/<YourEsIndex> --output=<YourLocalFile> --type=data

Migrate data based on a query

elasticdump --input=http://"<UserName>:<YourPassword>"@<YourEsHost>/<YourEsIndex> --output=<YourLocalFile> ----searchBody="<YourQuery>"

Import from local file (restore)

Restore documents

elasticdump --input=<YourLocalFile> --output=http://"<UserName>:<YourPassword>"@<YourEsHost>/<YourEsIndex> --type=data

Parameter reference

Parameter	Description
`--input` / `--output`	Source and destination. Can be a cluster URL or a local file path. Important When exporting to a local file, elasticsearch-dump creates the file at the specified path. Ensure the destination filename is unique in the target directory.
<YourEsHost>/<OtherEsHost>	The endpoint (internal or public) of your Elasticsearch cluster (e.g., `es-cn-abc.public.elasticsearch.aliyuncs.com:9200`). View the basic information of a cluster.
<UserName>/<OtherName>	Cluster username (default is `elastic`).
<YourPassword>/<OtherPassword>	Cluster password.
`--type`	The type of migration: `settings`, `mapping`, or `data`.
`--searchBody`	Filter data using a Query DSL. Example: `{"query":{"term":{"user":"admin"}}}`

FAQ and troubleshooting

Q: Error: getaddrinfo ENOTFOUND elastic

Cause: This usually occurs when your password contains special characters (like #, !, or @) that break the URL structure.

Solutions: Use an authentication file

Create a file named auth.ini:
```
user=elastic
password="Your#Complex$Password"
```
Note
The password must be enclosed in double quotation marks (").

Run elasticdump using the --httpAuthFile flag:

elasticdump --input=http://es-*****.public.elasticsearch.aliyuncs.com:9200/customers --output=/root/customers.json --httpAuthFile=/root/auth.ini --type=settings

Q: URIError: URI malformed

Cause: elasticsearch-dump cannot parse usernames and passwords that contain special characters.

Solutions:

Method 1: Remove the special characters in the username or password.
Method 2: Log on to the Kibana console, create a user with the required permissions, and use it for the migration.