Feature overview

The procedure for data synchronization in the Cloud Backup console is as follows:

1. Activate Cloud Backup

   Activating Cloud Backup is free, and Cloud Backup does not charge any fees for this feature. Alibaba Cloud NAS charges for reading files over the Internet and for file storage. Alibaba Cloud OSS also charges for file storage. For more information about billing, see Billing overview of Alibaba Cloud NAS and Billing overview of Alibaba Cloud OSS.

2. Install the client

   Before you synchronize data from HDFS, on-premises NAS, S3-compatible storage, CPFS, OSS, or OSS protocol-compatible storage, you must install the client on the server that runs the sync task. The server must be able to communicate with the data source node over the network. The client is used to establish communication and management services with Cloud Backup.

3. Add data sources

   In the Cloud Backup console, add HDFS, on-premises NAS, S3-compatible storage, CPFS, OSS, or OSS protocol-compatible storage as a data source.

4. Create a sync task

   Create a data sync task. Cloud Backup starts the task and continuously synchronizes data based on the specified schedule. The data synchronization service is responsible only for data synchronization and cannot guarantee data consistency or integrity. After a sync task is complete, you must perform a full data consistency verification between the source and destination.

Limits

• The Cloud Backup client for Windows does not support mounting NAS using the NFS protocol. If the source or destination includes an NFS-based NAS, select the Cloud Backup client for Linux when you create a sync task.

• OSS and S3 do not support synchronizing objects of the Archive storage class. Only the object data is copied, including files pointed to by symbolic links (symlinks). Metadata, such as read and write permissions, is not copied.

Incremental file list (ChangeList)

Synchronizing data based on an incremental file list eliminates the need for a full scan of the file system. This reduces file scanning costs and improves synchronization efficiency. When you create a sync task, if you select OSS or S3-compatible storage as the data source, you can use the incremental file list feature.

1. An incremental file list is a CSV file that contains information about the files on the data source that need to be synchronized. Each row represents a file. The Name field specifies the relative path of the file from the root of the synchronization path. For example, the following figure shows a file named changeList1.csv. The Name field is a required column that lists the files to be synchronized. Other columns are not supported.

   

   Note

   • The values in the Name column must be unique. Otherwise, files may be overwritten during data synchronization.

   • The value in the Name column must be a file path, not a folder path.

2. When you create a data sync task, specify the path to the folder that contains the incremental file list. When the data sync task starts, it reads and parses all CSV files in that path on the source data source. Then, it transfers the specified files to the destination.

   Note

   For example, when you create an OSS-to-NAS sync task, you can use an incremental file list to specify the files to be synchronized. If the incremental file lists changeList1.csv and changeList2.csv are saved in the /changelist folder in the source, set the incremental file list path to /changelist.

3. After all files listed in a CSV file are processed, the CSV file is moved to a subfolder named /Completed to indicate that the processing is complete.

   CSV file folder before processing	CSV file folder after processing
   /changelist/changeList1.csv /changelist/changeList2.csv	/changelist/Completed/changeList1.csv /changelist/Completed/changeList2.csv