Migrate NAS SMB data using Robocopy-File Storage NAS(NAS)-阿里云帮助中心

Prerequisites

The source data must be in an SMB file system with a mount target in a VPC.

Billing

Data migration between NAS file systems may incur the following fees:

The transit ECS instance incurs charges based on its configuration. Billing overview.
Storage costs apply to both file systems. Purchase a resource plan to reduce costs. Billing overview.
Cloud Enterprise Network (CEN) charges apply for transit routers and inter-region connections. Billing.

Preparation

The transit ECS instance must access both file systems. Ensure both are reachable from the same VPC.

Check the source mount target information.

Before you begin the migration, record the details of the source file system's mount target and its associated VPC. For more information, see View mount targets.

Note
If your file system only has a mount target in the classic network, you must create a new mount target in a VPC. For instructions, see Create a mount target.
Configure the destination file system mount target.
- File systems are in the same region
  - If the mount targets of the source and destination file systems are in the same VPC, get the destination mount target details and then continue to the Procedure section to migrate the data.
  - If the mount targets of the source and destination file systems are not in the same VPC, use one of the following methods to prepare the mount target:
    - Create a file system in the destination region and zone. A mount target is created automatically. Create a General-purpose NAS file system in the console.
      
      Note
      When purchasing a pay-as-you-go General-purpose NAS (Capacity, Performance, or Advanced) SMB file system, select the same VPC and vSwitch as the source mount target. This auto-generates the destination mount target. You can then purchase a resource plan to reduce costs.
    - Create a mount target on an existing file system. Add a mount target.
    - Connect the VPCs by using CEN. Mount a NAS file system across VPCs in the same region by using CEN.
- File systems are in different accounts or regions
  
  If the source and destination file systems are in different accounts or regions, connect their VPCs by using CEN. Mount a NAS file system across accounts and regions by using CEN.

Procedure

After you prepare the mount targets, create a transit ECS instance, mount both SMB file systems, and use Robocopy to migrate the data.

Mount the source and destination file systems.
Important
Use a dedicated temporary ECS instance for migration. Sharing an existing instance causes CPU and network contention with running workloads.
Log on to the ECS console, click Create Instance, and configure the following key parameters.
- Region: Select the region where the source file system resides.
- Network and Zone: Select the source file system's VPC, zone, and vSwitch.
- Instance Type: The minimum instance type is typically sufficient.
- Image: Select a Windows Server version. We recommend that you select Windows Server 2019.
  Note
  If you use Windows Server 2025, you must disable SMB signing before you can mount a File Storage NAS file system. Run the following command in PowerShell as an administrator, and then restart the instance:
  reg add "HKLM\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters" /v RequireSecuritySignature /t REG_DWORD /d 0 /f
- Storage: In the Elastic Ephemeral Disk | File Storage NAS | Dedicated Block Storage Cluster (Optional) section, click Add File System and configure the settings. For an example, see the following figure.
  Note
  
  If the source and destination mount targets are in the same VPC, you can configure the NAS mount information on the ECS purchase page. After the ECS instance starts, the source and destination NAS file systems are automatically mounted.
  
  If the mount targets are not in the same VPC, region, or account, configure only the source file system on the ECS purchase page. After the instance starts, manually mount the destination file system. Mount an SMB file system.
After the instance starts and both file systems are mounted, verify the mounts:
```
net use
```
If successful, the output shows both mounts. In this example, the source is drive Z and the destination is drive Y.
```
Status     Local      Remote                       Network
------------------------------------------------------------------------------
OK        Y:      \\29e9c24****-eab13.cn-wulanchabu.nas.aliyuncs.com\myshare
                                             MicrosoftWindowgNetwork
OK        Z:       \\29fe7f4****-txr31.cn-wulanchabu.nas.aliyuncs.com\myshare
                                             MicrosoftWindowgNetwork
```

Migrate the data.

Run the following command to migrate data from the source file system (drive Z) to the destination file system (drive Y).

robocopy Z:\ Y:\ /e /w:5 /z /mt:32

Note

This command migrates only the data within the specified directory, not the directory itself.

The following table describes the key parameters. Replace them based on your actual requirements.

Parameter	Description
/mt	Specifies the number of concurrent threads. The default value is 8. The value must be an integer from 1 to 128. This example uses 32 threads for multithreaded copying.
/w	Sets the interval in seconds between retries after an error.
/z	Copies files in restartable mode.
/e	Copies subdirectories, including empty ones.
/copyall	Copies all file information, including: Data Attributes Timestamps Access control lists (ACLs) Owner information Auditing information

Note

To accelerate the migration of a massive amount of data, such as hundreds of millions of small files that exceed 10 TB, you can install the latest version of Python on a Windows ECS instance to perform the migration. For more information, see Accelerate data migration to an SMB file system.

Verify the migration.
After migration, run the following command to verify consistency between source and destination:
```
ROBOCOPY Z:\ Y:\ /e /l /ns /njs /njh /ndl /fp /log:reconcile.txt
```
Key parameters:
- /e: Lists all subdirectories, including empty ones.
- /l: Lists differences without modifying or copying files.
- /fp: Includes the full path of files in the log. This parameter is necessary only if you omit /ndl.
- /ns: Excludes file sizes from the log.
- /ndl: Excludes folder names from the log.
- /njs: Excludes the job summary.
- /njh: Excludes the job header.
- /log:reconcile.txt: Writes the results to the reconcile.txt log file. If the file already exists, it is overwritten.

Switch workloads to the new file system

After migration, unmount the original file system and mount the new one on all relevant ECS instances and containers.

Mount a NAS file system directly on an ECS instance
1. Run net use to record the current NAS mount information. Note the local drive letter to which the NAS file system is mounted.
2. Run the following command to unmount the original file system.
```
net use Z: /delete
```
  Replace Z: in the command with the actual drive letter.
  Note
  
  Run the net use * /delete command to manually unmount all mounted file systems in Windows.
  
  Run the net use * /delete /y command to automatically unmount all mounted file systems in Windows.
3. Mount the new file system to the original drive letter. Mount an SMB file system.
4. Start the processes that access the NAS file system and confirm that read and write operations are successful.
5. Modify the auto-mount information in the auto_mount.bat file by replacing the old mount target with the new one.
Mount a NAS file system in a Windows container
1. Modify the existing YAML configuration file to replace the original mount target with the new one.
2. Use the modified configuration file to create a new pod. Confirm that the new file system is mounted successfully and that read and write operations are normal.
3. Recycle all pods that use the original file system.

Important

After switching workloads, retain data on the original file system for at least one week to prevent data loss from accidental deletion or synchronization errors.

FAQ

How do I accelerate data migration to an SMB file system?

To accelerate large-scale migrations (for example, more than 10 TB with hundreds of millions of small files around 100 KB each) while workloads are writing to the same SMB file system, install Python on a Windows ECS instance and run the following migration script.

Download and install the latest version of Python.
Add the Python executable path to the PATH environment variable (for example, C:\Python27).
```
set PATH=%PATH%;C:\python27
```
You can also run the where python command to check the Python installation path, as shown in the following figure.

Copy the following migration.py script to a local directory on your ECS instance, such as C:\.

#!/usr/bin/python

import os
import random
import string
import sys, getopt
import datetime
import time

def execute_cmd(cmd):
   print('\tExecuting cmd: %s' % cmd)
   count = 0
   rc = 0
   while (count < 3*60):
       rc = os.system(cmd)
       if (rc != 0):
           count += 1
           time.sleep(1)
           continue
       else:
           break

   if rc != 0:
       print('\tFailed to execute cmd: %s. rc:%d' %(cmd, rc))
   return

def migrate_subdirs(srcPath, dstPath, rangeBegin, rangeEnd, ignoreFile):
   currTimeStr = datetime.datetime.now().strftime("%Y-%m-%d_%H:%M:%S")
   print('Start to migrate from %s to %s for subdir range[%s, %s] at %s.\n' %(srcPath, dstPath, rangeBegin, rangeEnd, currTimeStr))
   index = 0

   for entry in os.listdir(srcPath):
       if os.path.isdir(os.path.join(srcPath, entry)):
           if index >= rangeBegin and index <= rangeEnd:
               srcSubDir = srcPath + "\\" + entry
               dstSubDir = dstPath + "\\" + entry
               print('\tBegin of migrating from the %d th dir %s to %s.' %(index, srcSubDir, dstSubDir))
               cmd = "robocopy \"" + srcSubDir + "\" \"" + dstSubDir + "\" /e /w:5 /z /mt:32"
               if ignoreFile.strip():
                   cmd += " /XF \"" + ignoreFile + "\""
               cmd += " >> robocopy.log"
               execute_cmd(cmd)
               print('\tEnd of migrating from %s to %s.\n' %(srcSubDir, dstSubDir))
           index += 1
   print('Finish to migrate from %s to %s for subdir range[%s, %s] at %s.\n' %(srcPath, dstPath, rangeBegin, rangeEnd, datetime.datetime.now().strftime("%Y-%m-%d_%H:%M:%S")))

def migrate_regfiles(srcPath, dstPath, ignoreFile):
   currTimeStr = datetime.datetime.now().strftime("%Y-%m-%d_%H:%M:%S")
   print('Start to migrate from %s to %s for regular files at %s.\n' %(srcPath, dstPath, currTimeStr))
   for entry in os.listdir(srcPath):
       if os.path.isfile(os.path.join(srcPath, entry)):
           print('\tBegin of migrating %s from %s to %s.\n' %(entry, srcPath, dstPath))
           cmd = "attrib -R \"" + dstPath + "\\\\" + entry + "\""
           execute_cmd(cmd)
           cmd = "copy \"" + srcPath + "\\\\" + entry + "\" \"" + dstPath + "\" /Y"
           if ignoreFile.strip():
               cmd += " /XF \"" + ignoreFile + "\""
           cmd += " >> robocopy.log"
           execute_cmd(cmd)
           print('\tEnd of migrating %s from %s to %s' %(entry, srcPath, dstPath))
   print('Finish to migrate from %s to %s for regular files at %s.\n' %(srcPath, dstPath, datetime.datetime.now().strftime("%Y-%m-%d_%H:%M:%S")))


def main(argv):
   srcPath = ''
   dstPath = ''
   range = ''
   ignoreFile = ''
   try:
       opts, args = getopt.getopt(argv,"hs:d:r:i:f",["srcPath=","dstPath=","range=","ignore="])
   except getopt.GetoptError:
       print('migration.py -s <source path> -d <destination path> [-r BeginIndex:EndIndex | -f] [-i ignoredFile]')
       print('example: migration.py -s x:\pic -d z:\pic [-r 0:100]')
       sys.exit(2)

   subdironly = False
   fileonly = False
   for opt, arg in opts:
       if opt == '-h':
           print('migration.py -s <source path> -d <destination path> [-r BeginIndex:EndIndex | -f] [-i ignoredFile]')
           sys.exit()
       elif opt in ("-s", "--srcPath"):
           srcPath = arg
       elif opt in ("-d", "--dstPath"):
           dstPath = arg
       elif opt in ("-r", "--range"):
           range = arg
           subdironly = True
       elif opt in ("-f", "--file"):
           fileonly = True
       elif opt in ("-i", "--ignore"):
           ignoreFile = arg

   if not srcPath.strip() or not dstPath.strip():
        print('migration.py -s <source path> -d <destination path> [-r BeginIndex:EndIndex | -f] [-i ignoredFile]')
        sys.exit()

   if subdironly and fileonly:
        print('migration.py -s <source path> -d <destination path> [-r BeginIndex:EndIndex | -f] [-i ignoredFile]')

        sys.exit()

   if not fileonly:
       if not range.strip():
            rangeBegin = 0
            rangeEnd = sys.maxsize-1
       else:
            rangeBegin, rangeEnd = (int(x) for x in range.split(":"))
            if rangeBegin < 0 or rangeEnd >= sys.maxsize or rangeEnd < rangeBegin:
                print('migration.py -s <source path> -d <destination path> [-r BeginIndex:EndIndex | -f] [-i ignoredFile]')
                print('example: migration.py -s x:\pic -d z:\pic -r 0:99')
                sys.exit()
       migrate_subdirs(srcPath, dstPath, rangeBegin, rangeEnd, ignoreFile)

   if not subdironly:
        migrate_regfiles(srcPath, dstPath, ignoreFile)

if __name__ == "__main__":

  main(sys.argv[1:])

Migrate the data. The command uses the following format: python ./migration.py -s <source path> -d <destination path> [-r BeginIndex:EndIndex | -f] [-i ignoredFile].

The following table describes the main parameters in the command.

Parameter	Description
-s <source path>	Specifies the path to the source directory. For example, `-s Z:\` specifies the root directory of drive Z.
-d <destination path>	Specifies the path to the destination directory. For example, `-d Y:\` specifies the root directory of drive Y.
-r BeginIndex:EndIndex	Specifies the range of subdirectories to migrate, from `BeginIndex` to `EndIndex`. For example, `-r 1:100` migrates subdirectories 1 through 100.
-f	Migrates only the regular files in the top level of the source directory. Migrate all files in the source file system directory, ignoring scope limitations.
-i ignoredFile	Specifies files in the source directory to exclude from the migration. For example, `-i ignored.txt` excludes the file named `ignored.txt`.

Note

By default, the script runs Robocopy for all subdirectories in sequence, then copies root-level files. To accelerate migration, run the script in parallel on multiple clients with unique subdirectory ranges. For example, the first client uses -r 0:9999, the second uses -r 10000:19999, and so on.

For example, to migrate data from the source Z:\ to the destination Y:\, run the following command:

python ./migration.py -s Z:\ -d Y:\

Migrate data using Robocopy

Prerequisites

Background

Billing

Preparation

Procedure

Switch workloads to the new file system

FAQ

How do I accelerate data migration to an SMB file system?

Related topics