Implementation

Updated at:
Copy as MD

This topic describes the usage notes, migration limitations, and procedure for migrating data from OSS, OSS-HDFS, or a third-party source to Alibaba Cloud Object Storage Service (OSS).

Usage notes

  • The online migration service accesses source data using the standard public APIs of the source storage service provider. The migration behavior depends on the provider's service implementation.

  • Online migration consumes resources at both the source and the destination, potentially impacting your business operations. If your services are critical, evaluate the potential impact and configure a speed limit for the migration task, or run the task during off-peak hours.

  • The online migration service checks files at both the source and destination before a migration begins. However, if a file at the source has the same name as a file at the destination and the migration task is configured to overwrite existing files, the service overwrites the destination file. If the files contain different data, rename one of the files or back up the destination file before starting the migration.

  • The online migration service preserves the last modified time of source files. If a lifecycle rule is configured for the destination bucket, a migrated file whose last modified time matches the rule's conditions may be deleted or transitioned to the specified archive storage class.

Migration limitations

OSS/OSS Inventory

  • If static website hosting is enabled for the source address, the migration scans additional directories that do not actually exist. For example, after you upload the file myapp/resource/1.jpg and enable static website hosting, the migration service scans three objects: myapp/, myapp/resource/, and myapp/resource/1.jpg. Of these objects, myapp/ and myapp/resource/ fail to migrate, but this does not affect the successful migration of myapp/resource/1.jpg.

  • If the source address contains symbolic links, the symbolic links are migrated directly during data migration. For more information about symbolic links, see Symbolic links.

  • Data Online Migration can migrate only one bucket at a time. You cannot migrate the data of an entire account in a single migration.

  • Data migration is not supported in Finance Cloud or Government Cloud.

  • The following attributes are migrated for Alibaba OSS and Alibaba OSS Inventory data sources:

    • Attributes that can be migrated: x-oss-meta-*, LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Disposition, Content-Language, and Expires.

    • Attributes that cannot be migrated (including but not limited to): StorageClass, ACL, server-side encryption, tagging, and user-defined headers such as x-oss-persistent-header.

      Note

      Unsupported attributes include, but are not limited to, those listed above. The behavior of unlisted attributes is undefined., the actual migration result is authoritative.

S3/S3 Inventory

  • You can migrate data from only one bucket per migration task. Migrating data from an entire account at once is not supported.

  • The following attributes are migrated for AWS S3 and AWS S3 Inventory data sources:

    • Attributes that can be migrated: x-amz-meta-*, LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Disposition, Content-Language, and Expires.

    • Attributes that cannot be migrated (including but not limited to): StorageClass, ACL, server-side encryption, and tagging.

      Note

      Unsupported attributes include, but are not limited to, those listed above. The behavior of unlisted attributes is undefined., the actual migration result is authoritative.

  • For data sources that are not compatible with the S3 API, such as Volcengine TOS and Netease Cloud, migration may fail and cause unpredictable issues. We recommend that you migrate data over HTTP.

Compatible S3

  • You can migrate data from only one bucket per migration task. Migrating data from an entire account at once is not supported.

  • The following attributes are migrated for Compatible S3 data sources:

    • Attributes that can be migrated: LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Disposition, Content-Language, and Expires.

    • Attributes that cannot be migrated (including but not limited to): StorageClass, ACL, server-side encryption, and tagging.

      Note

      Unsupported attributes include, but are not limited to, those listed above. The behavior of unlisted attributes is undefined., the actual migration result is authoritative.

Inventory

You can migrate data from only one bucket per migration task. Migrating data from an entire account at once is not supported.

The following attributes are migrated for Inventory data sources:

  • The attributes that can be migrated depend on the data type, such as OSS or S3. For more information, see the migration tutorial for the corresponding data source.

  • Attributes that cannot be migrated: not yet determined. The actual migration result is authoritative.

HTTP/HTTPS

The following attributes are migrated for HTTP/HTTPS data sources:

  • Attributes that can be migrated: LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Disposition, Content-Language, and Expires.

  • Attributes that cannot be migrated: not yet determined. The actual migration result is authoritative.

Tencent COS

  • You can migrate data from only one bucket per migration task. Migrating data from an entire account at once is not supported.

  • The following attributes are migrated for Tencent COS data sources:

    • Attributes that can be migrated: x-cos-meta-*, LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Disposition, Content-Language, and Expires.

    • Attributes that cannot be migrated (including but not limited to): StorageClass, ACL, server-side encryption, and tagging.

      Note

      Unsupported attributes include, but are not limited to, those listed above. The behavior of unlisted attributes is undefined., the actual migration result is authoritative.

      Warning

      If the source data address uses a Tencent standard domain name, calling GetObject forcibly sets Content-Disposition to attachment, and this attribute is migrated to the destination, which causes an inconsistency with the source. When a browser accesses such files (such as HTML and PNG files), a forced download is triggered. To avoid this issue, bind a custom domain name on the Tencent COS website and use that domain name for the migration to preserve the original Content-Disposition setting.

  • Buckets for which metadata acceleration is enabled cannot be migrated.

Qiniu Kodo

  • You can migrate data from only one bucket per migration task. Migrating data from an entire account at once is not supported.

  • The following attributes are migrated for Qiniu Kodo data sources:

    • Attributes that can be migrated: x-qn-meta-*, LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Disposition, Content-Language, and Expires.

    • Attributes that cannot be migrated (including but not limited to): StorageClass, ACL, server-side encryption, and tagging.

      Note

      Unsupported attributes include, but are not limited to, those listed above. The behavior of unlisted attributes is undefined., the actual migration result is authoritative.

Huawei OBS/OBSFS

  • You can migrate data from only one bucket per migration task. Migrating data from an entire account at once is not supported.

  • The following attributes are migrated for Huawei OBS data sources:

    • Attributes that can be migrated: x-obs-meta-*, LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Language, and Expires.

    • Attributes that cannot be migrated (including but not limited to): StorageClass, Content-Disposition, ACL, server-side encryption, and tagging.

      Note

      Unsupported attributes include, but are not limited to, those listed above. The behavior of unlisted attributes is undefined., the actual migration result is authoritative.

    • Warning

      If the source data address uses a Huawei OBS standard domain name, calling GetObject forcibly sets Content-Disposition to attachment, and this attribute is migrated to the destination, which causes an inconsistency with the source. When a browser accesses such files (such as HTML and PNG files), a forced download is triggered.

Volcengine TOS

  • You can migrate data from only one bucket per migration task. Migrating data from an entire account at once is not supported.

  • The following attributes are migrated for Volcengine TOS data sources:

    • Attributes that can be migrated: x-tos-meta-*, LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Disposition, Content-Language, and Expires.

    • Attributes that cannot be migrated (including but not limited to): StorageClass, ACL, server-side encryption, and tagging.

      Note

      Unsupported attributes include, but are not limited to, those listed above. The behavior of unlisted attributes is undefined., the actual migration result is authoritative.

  • If Volcengine TOS is not compatible with the AWS S3 API, migration may fail and cause unpredictable issues. We recommend that you migrate data over HTTP.

  • Buckets for which hierarchical namespace (HNS) is enabled cannot be migrated.

UCloud

  • Data Online Migration can migrate only one bucket at a time. You cannot migrate the data of an entire account in a single migration.

  • The following attributes are migrated for UCloud US3 data sources:

    • Attributes that can be migrated: x-ufile-meta-*, LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Disposition, Content-Language, and Expires.

    • Attributes that cannot be migrated (including but not limited to): StorageClass, ACL, server-side encryption, and tagging.

      Note

      Unsupported attributes include, but are not limited to, those listed above. The behavior of unlisted attributes is undefined., the actual migration result is authoritative.

Baidu BOS

  • You can migrate data from only one bucket per migration task. Migrating data from an entire account at once is not supported.

  • The following attributes are migrated for Baidu BOS data sources:

    • Attributes that can be migrated: x-bce-meta-*, LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Disposition, and Content-Language.

    • Attributes that cannot be migrated (including but not limited to): Expires, StorageClass, ACL, server-side encryption, and tagging.

      Note

      Unsupported attributes include, but are not limited to, those listed above. The behavior of unlisted attributes is undefined., the actual migration result is authoritative.

      Warning

      If the source data address uses a Baidu BOS standard domain name, calling GetObject forcibly sets Content-Disposition to attachment, and this attribute is migrated to the destination, which causes an inconsistency with the source. When a browser accesses such files (such as HTML files), a forced download is triggered.

Kingsoft KS3

  • Data Online Migration can migrate only one bucket at a time. You cannot migrate the data of an entire account in a single migration.

    Important

    Kingsoft KS3 migrates data by using the AWS S3 protocol.

  • The following attributes are migrated for Kingsoft KS3 data sources:

    • Attributes that can be migrated: x-amz-meta-*, LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Disposition, Content-Language, and Expires.

    • Attributes that cannot be migrated (including but not limited to): StorageClass, ACL, server-side encryption, and tagging.

      Note

      Unsupported attributes include, but are not limited to, those listed above. The behavior of unlisted attributes is undefined., the actual migration result is authoritative.

Google Cloud Storage

  • You can migrate data from only one bucket per migration task. Migrating data from an entire account at once is not supported.

  • The following attributes are migrated for Google Cloud Storage data sources:

    • Attributes that can be migrated: x-amz-meta-*, LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Disposition, and Content-Language.

    • Attributes that cannot be migrated (including but not limited to): Expires, StorageClass, ACL, server-side encryption, and tagging.

      Note

      Unsupported attributes include, but are not limited to, those listed above. The behavior of unlisted attributes is undefined., the actual migration result is authoritative.

Youpai USS

  • You can migrate data from only one bucket per migration task. Migrating data from an entire account at once is not supported.

Important

Youpai USS migrates data by using the S3 protocol standard. For more information, see AWS S3 Compatibility

  • The following attributes are migrated for Youpai USS data sources:

    • Attributes that can be migrated: x-amz-meta-*, LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Disposition, Content-Language, and Expires.

    • Attributes that cannot be migrated (including but not limited to): StorageClass, ACL, server-side encryption, and tagging.

      Note

      Unsupported attributes include, but are not limited to, those listed above. The behavior of unlisted attributes is undefined., the actual migration result is authoritative.

Azure Blob

  • Data Online Migration can migrate only one bucket at a time. You cannot migrate the data of an entire account in a single migration.

  • The following attributes are migrated for Azure Blob data sources:

    • Attributes that can be migrated: x-ms-meta-*, LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Disposition, Content-Language, and Expires.

    • Attributes that cannot be migrated (including but not limited to): StorageClass, ACL, server-side encryption, and tagging.

      Note

      Unsupported attributes include, but are not limited to, those listed above. The behavior of unlisted attributes is undefined., the actual migration result is authoritative.

LocalFS

  • Empty directories, symbolic links to files or directories, character device files, block device files, socket files, and pipe files at the source data address are not migrated.

  • If hard links exist at the source data address, the linked files are migrated as regular files.

  • The attributes of the parent directories of files cannot be migrated.

  • File permissions such as SUID, SGID, and SBID cannot be migrated.

  • The attribute limits for migrating LocalFS data to Alibaba Cloud OSS are as follows:

    • Attributes that can be migrated: ModifyTime -> X-Oss-Meta-Mtime (OSS), Permissions -> X-Oss-Meta-Perms (OSS), and Uid:Gid -> X-Oss-Meta-Owner (OSS)

      Note
      • Permissions: the nine permissions, including read, write, and execute.

      • Uid:Gid: the user ID and the ID of the group to which the user belongs, separated by a colon (:).

    • Attributes that cannot be migrated (including but not limited to): AccessTime, ChangeTime, Attr, and ACL.

      Note

      Unsupported attributes include, but are not limited to, those listed above. The behavior of unlisted attributes is undefined., the actual migration result is authoritative.

OSSHDFS

  • Data Online Migration can migrate only one bucket at a time. You cannot migrate the data of an entire account in a single migration.

  • Only some attributes of OSS-HDFS can be migrated to the custom meta of the destination OSS address. The following attributes are supported:

    • Mtime (last modified time).

    • Owner and Group.

    • Permission (read, write, and execute).

Note

For attributes and behaviors that are not described in this topic, the actual migration result is authoritative.

FTP

  • Data Online Migration can migrate only the data in a single FTP directory at a time. You cannot migrate the data of an entire account in a single migration.

  • Empty directories, hidden files, symbolic links to files or directories, character device files, block device files, socket files, and pipe files at the source data address are not migrated.

  • Files whose names contain special characters at the source address may fail to migrate.

  • If a single file exceeds 5 GB and the destination data address type is Alibaba Cloud OSS, the file fails to migrate.

Step 1: Select a region

  1. Log on to the Data Online Migration console.

  2. Region selection: the region in the upper-left corner of the console indicates the region in which the migration service is deployed. For best performance, select the region where the data source resides or the region that is closest to the data source.

  3. If your data source resides outside the Chinese mainland, you must select a migration service that is deployed in a region outside the Chinese mainland and is geographically close to the data source.

Step 2: Create a source address

Note

For LocalFS and FTP sources, you must create a tunnel and an agent before you create the source address. For more information, see Manage tunnels and Agent management.

OSS

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. In the Create Address panel, configure the following parameters, and then click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select Alibaba OSS.

    Custom Domain Name

    No

    Specifies whether to use a custom domain name.

    Region

    Yes

    Select the region where the source address resides. For example, select China (Hangzhou).

    Authorize Role

    Yes

    Bucket

    Yes

    Enter the name of the bucket that stores the data to be migrated in the current account.

    Prefix

    No

    You can specify a data path prefix to migrate part of the data. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example: data/to/oss/.

    • With a prefix specified: For example, assume that the prefix of the source data address is set to example/src/ and this path contains the file example.jpg. If the prefix of the destination data address is set to example/dest/, the full path of the file example.jpg after migration is example/dest/example.jpg.

    • Without a prefix specified: For example, assume that no prefix is set for the source data address and the path of the file to migrate is srcbucket/example.jpg. If the prefix of the destination data address is set to destbucket/, the full path of the file after migration is destbucket/srcbucket/example.jpg.

    Tunnel

    No

    Select the name of the tunnel that you want to use.

    Important
    • This parameter is required only when you migrate data to the cloud over an Express Connect circuit or a VPN, or when you migrate data from self-managed storage to the cloud.

    • You must associate an agent when the destination data address is LocalFS, or when the migration must go over an Express Connect circuit (such as in Finance Cloud or Apsara Stack).

    Agent

    No

    Select the name of the agent that you want to use.

    Important
    • This parameter is required only when you migrate data to the cloud over an Express Connect circuit or a VPN, or when you migrate data from self-managed storage to the cloud.

    • You can select up to 200 agents for a specified tunnel.

OSS Inventory

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. In the Create Address panel, configure the following parameters, and click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select Alibaba OSS Inventory.

    Region

    Yes

    Select the region where the source address resides. For example, select China (Hangzhou).

    Authorize Role

    Yes

    Bucket

    Yes

    Enter the name of the bucket that stores the data to be migrated in the current account.

    Region

    Yes

    When Inventory Location is set to Alibaba OSS, enter the region where the OSS inventory resides.

    Inventory Authorize Role

    Yes

    Bucket

    Yes

    Enter the name of the bucket where the inventory to be migrated resides. The bucket belongs to the account that is used to log on to the current console.

    Inventory Path

    Yes

    Enter the path of the manifest.json file, such as dir/manifest.json.

    Tunnel

    No

    Select the name of the tunnel that you want to use.

    Important
    • This parameter is required only when you migrate data to the cloud over an Express Connect circuit or a VPN, or when you migrate data from self-managed storage to the cloud.

    • You must associate an agent when the destination data address is LocalFS, or when the migration must go over an Express Connect circuit (such as in Finance Cloud or Apsara Stack).

    Agent

    No

    Select the name of the agent that you want to use.

    Important
    • This parameter is required only when you migrate data to the cloud over an Express Connect circuit or a VPN, or when you migrate data from self-managed storage to the cloud.

    • You can select up to 200 agents for a specified tunnel.

S3

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. On the Create Data Address page, configure the following parameters, and then click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select AWS S3.

    Domain Name

    Yes

    Enter the endpoint domain name of AWS S3.For more information, see Amazon S3 endpoints.

    AccessKeyId

    Yes

    Enter the AccessKey pair of the IAM user that you created. AWS S3 uses the AccessKey pair to identify the user and verify whether the user has the permissions to read the source data.

    SecretAccessKey

    Yes

    Bucket

    Yes

    Enter the name of the AWS S3 bucket that stores the data to be migrated.

    Note

    The bucket name cannot start or end with invalid characters such as spaces, line breaks, or tabs.

    Prefix

    No

    You can specify a data path prefix to migrate part of the data.

    • Specify a prefix: migrates the data in the specified directory (prefix). The prefix cannot start with a forward slash (/) and must end with a forward slash (/).

    • Do not specify a prefix: migrates all data in the bucket.

    Tunnel

    No

    Select the name of the tunnel that you want to use.

    Important
    • This parameter is required only when you migrate data to the cloud over an Express Connect circuit or a VPN, or when you migrate data from self-managed storage to the cloud.

    • You must associate an agent when the destination data address is LocalFS, or when the migration must go over an Express Connect circuit (such as in Finance Cloud or Apsara Stack).

    Agent

    No

    Select the name of the agent that you want to use.

    Important
    • This parameter is required only when you migrate data to the cloud over an Express Connect circuit or a VPN, or when you migrate data from self-managed storage to the cloud.

    • You can select up to 200 agents for a specified tunnel.

S3 Inventory

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. In the Create Data Address panel, configure the following parameters, and then click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select AWS S3 Inventory.

    Domain Name

    Yes

    Enter the AWS S3 service endpoint. Endpoints are listed in Amazon S3 endpoints.

    AccessKeyId

    Yes

    Enter the IAM user access key. This key authenticates read access to the source data.

    SecretAccessKey

    Yes

    Bucket

    Yes

    Enter the name of the source AWS S3 bucket.

    Note

    The bucket name cannot start or end with spaces, line breaks, or tabs.

    Inventory Location

    Yes

    Select the storage where the inventory resides, and then select AWS S3.

    Inventory Domain Name

    Yes

    When Inventory Location is set to AWS S3, enter the endpoint domain name of the AWS S3 inventory. Endpoints are listed in Amazon S3 endpoints.

    Inventory Bucket

    Yes

    Enter the name of the bucket where the inventory list of the data to be migrated resides.

    Inventory Path

    Yes

    When Inventory Location is set to AWS S3, enter the path of the AWS S3 inventory manifest.json file, such as dir/manifest.json.

    Inventory AccessKeyId

    Yes

    When Inventory Location is set to AWS S3, enter the AccessKey pair that is used to access the AWS S3 inventory. The AccessKey pair is deleted after the migration is complete.

    Inventory SecretAccessKey

    Yes

    Tunnel

    No

    Select the name of the tunnel that you want to use.

    Important
    • This parameter is required only when you migrate data to the cloud over an Express Connect circuit or a VPN, or when you migrate data from self-managed storage to the cloud.

    • You must associate an agent when the destination data address is LocalFS, or when the migration must go over an Express Connect circuit (such as in Finance Cloud or Apsara Stack).

    Agent

    No

    Select the name of the tunnel that you want to use.

    Important
    • This parameter is required only when you migrate data to the cloud over an Express Connect circuit or a VPN, or when you migrate data from self-managed storage to the cloud.

    • You must associate an agent when the destination data address is LocalFS, or when the migration must go over an Express Connect circuit (such as in Finance Cloud or Apsara Stack).

Compatible S3

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. On the Create Data Address page, configure the following parameters, and then click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select Compatible S3.

    Domain Name

    Yes

    Enter the endpoint domain name of the data source that is compatible with the S3 protocol.

    Region

    Yes

    If you use a non-standard S3 domain name, enter the region that corresponds to the actual domain name of the source.

    AccessKeyId

    Yes

    Enter the AccessKey pair. The data source uses the AccessKey pair to identify the user and verify whether the user has the permissions to read the source data.

    SecretAccessKey

    Yes

    Bucket

    Yes

    Enter the name of the bucket that stores the data to be migrated.

    Note

    The bucket name cannot start or end with invalid characters such as spaces, line breaks, or tabs.

    Prefix

    No

    You can specify a data path prefix to migrate part of the data.

    • Specify a prefix: migrates the data in the specified directory (prefix). The prefix cannot start with a forward slash (/) and must end with a forward slash (/).

    • Do not specify a prefix: migrates all data in the bucket.

    Tunnel

    No

    Select the name of the tunnel that you want to use.

    Important
    • This parameter is required only when you migrate data to the cloud over an Express Connect circuit or a VPN, or when you migrate data from self-managed storage to the cloud.

    • You must associate an agent when the destination data address is LocalFS, or when the migration must go over an Express Connect circuit (such as in Finance Cloud or Apsara Stack).

    Agent

    No

    Select the name of the agent that you want to use.

    Important
    • This parameter is required only when you migrate data to the cloud over an Express Connect circuit or a VPN, or when you migrate data from self-managed storage to the cloud.

    • You can select up to 200 agents for a specified tunnel.

Inventory

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. In the Create Address panel, configure the following parameters, and then click OK.

  3. Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select Inventory.

    Data Type

    Yes

    Select the source that you want to migrate based on your business requirements.

    Domain Name

    Depends on the Data Type

    The endpoint address of the source storage service (for example, the endpoint domain name of AWS S3).

    Region

    Yes (Data Type is set to Alibaba OSS)

    Select the region where the source address resides. For example, select China (Hangzhou).

    Authorize Role

    Yes (Data Type is set to Alibaba OSS)

    AccessKey pair

    Yes

    Enter the AccessKey pair of the account to which the source data belongs (including AccessKeyId and SecretAccessKey), which is used to identify the user and verify whether the user has the permissions to read the source data.

    Storage

    Yes

    Enter the name of the Bucket.

    Prefix

    No

    You can specify a data path prefix to migrate the source data to a specified directory. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, data/to/oss/.

    • Specify a prefix: for example, if the prefix of the source data address is example/src/ and this path contains the file example.jpg. If the prefix of the destination data address is set to example/dest/, the full path of the file example.jpg after migration is example/dest/example.jpg.

    • Do not specify a prefix: if you do not set a data path prefix, the source data is migrated to the root directory of the destination bucket.

    Inventory Location

    Yes

    The storage where the inventory file resides. You can select Alibaba OSS or Third-party source.

    Inventory Path

    Yes

    Enter the path of the manifest.json file, such as dir/manifest.json.

    Inventory Domain Name

    Yes (Inventory Location is set to a value other than Alibaba OSS)

    When Inventory Location is set to Third-party source, enter the endpoint domain name of the inventory.

    InventoryRegion

    Depends on the Data Type

    If you set the Inventory Location parameter to Alibaba OSS, specify the region in which the OSS inventory list resides.

    Authorize Role

    Yes (Inventory Location is set to Alibaba OSS)

    Inventory Bucket

    Yes

    Enter the name of the Bucket where the inventory to be migrated resides. The bucket belongs to the account that is used to log on to the current console.

    Inventory AccessKey pair

    Yes (Inventory Location is set to a value other than Alibaba OSS)

    When Inventory Location is set to a value other than Alibaba OSS , enter the AccessKey pair that is used to access the source inventory (including AccessKeyId and SecretAccessKey), which is used to identify the user and verify whether the user has the permissions to read the inventory files.

    Tunnel

    No

    Select the name of the channel to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You must associate an agent if the destination is a local file system or if the migration uses a dedicated connection, for example, to a financial cloud or Apsara Stack environment.

    Agent

    No

    Select the name of the agent to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You can select a maximum of 200 agents for a specified channel.

HTTP/HTTPS

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. In the Create Address panel, configure the following parameters, and then click OK.

  3. Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select HTTP/HTTPS.

    Protocol

    Yes

    Select HTTP or HTTPS.

    Port

    Yes

    Enter the port number. The default port is 80 for HTTP and 443 for HTTPS. Valid values: 0 to 65535.

    Inventory Location

    Yes

    The storage where the inventory file resides. You can select Alibaba OSS or AWS S3.

    Inventory Path

    Yes

    Enter the path of the manifest.json file, such as dir/manifest.json.

    Inventory Domain Name or Region

    Yes

    • If you set the Inventory Location parameter to Alibaba OSS, specify the region in which the OSS inventory list resides.

    • When Inventory Location is set to AWS S3, enter the endpoint domain name of the AWS S3 inventory. Endpoints are listed in Amazon S3 endpoints.

    Authorize Role

    Yes (Inventory Location is set to Alibaba OSS)

    Bucket

    Yes

    Enter the name of the bucket where the inventory to be migrated resides. The bucket belongs to the account that is used to log on to the current console.

    Inventory AccessKeyId

    Yes (Inventory Location is set to AW3 S3)

    If Inventory Location is set to AWS S3, enter the access key used to access the AWS S3 inventory list. For security, delete this key after the migration is complete.

    Inventory SecretAccessKey

    Tunnel

    No

    Select the name of the channel to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You must associate an agent if the destination is a local file system or if the migration uses a dedicated connection, for example, to a financial cloud or Apsara Stack environment.

    Agent

    No

    Select the name of the agent to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You can select a maximum of 200 agents for a specified channel.

Tencent COS

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. On the Create Address page, configure the following parameters, and click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select Tencent COS.

    Domain Name

    Yes

    Enter the endpoint of the bucket, such as cos.ap-nanjing.myqcloud.com. For more information about Tencent COS domain names, see Regions and Endpoints.

    AccessKeyId

    Yes

    Enter the cloud API key that you created. Tencent COS uses the key to identify the user and verify whether the user has the permissions to read the source data.

    SecretAccessKey

    Yes

    Bucket

    Yes

    Enter the bucket name of Tencent COS. A COS bucket name consists of Custom name-APPID. For example, example-1234567890.

    Note

    In the Tencent Cloud console, you can view the APPID on the Account Information page.

    Prefix

    Yes

    Specify a prefix to migrate a subset of the data.

    • If specified, only data under that prefix is migrated. The prefix must not start with a forward slash (/) but must end with one.

    • If you omit this parameter, all data in the bucket is migrated.

    Tunnel

    No

    Select the name of the channel to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You must associate an agent if the destination is a local file system or if the migration uses a dedicated connection, for example, to a financial cloud or Apsara Stack environment.

    Agent

    No

    Select the name of the agent to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You can select a maximum of 200 agents for a specified channel.

Qiniu Kodo

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. In the Create Address panel, configure the following parameters, and then click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select Qiniu Kodo.

    Domain Name

    Yes

    Enter the domain name that corresponds to the region where the bucket resides.

    You can view the domain name in the Qiniu Kodo console. For more information, see Domain Name Management.

    Important
    • The Qiniu Kodo test domain has limits on the number of accesses from a single IP address and on the bandwidth. We recommend that you use a custom origin domain name to create the migration task.

    • For more information about domain names, see Test Domain Usage Guidelines (The linked document is for reference only and may be outdated due to changes on the source website).

    AccessKeyId

    Yes

    Enter the Qiniu Kodo access key. Qiniu Kodo uses the access key to identify the user and verify whether the user has the permissions to read the source data.

    SecretAccessKey

    Yes

    Bucket

    Yes

    For a Qiniu object storage bucket, enter only the custom name. For example, if the bucket name is example-1234567890, enter example.

    Prefix

    Yes

    Specify a prefix to migrate a subset of the data.

    • If specified, only data under that prefix is migrated. The prefix must not start with a forward slash (/) but must end with one.

    • If you omit this parameter, all data in the bucket is migrated.

    Tunnel

    No

    Select the name of the channel to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You must associate an agent if the destination is a local file system or if the migration uses a dedicated connection, for example, to a financial cloud or Apsara Stack environment.

    Agent

    No

    Select the name of the agent to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You can select a maximum of 200 agents for a specified channel.

Huawei OBS/OBSFS

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. On the Create Data Address page, configure the following parameters, and then click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    • For standard object storage, select Huawei OBS.

    • For a parallel file system, select Huawei OBSFS.

    Domain Name

    Yes

    Enter the endpoint domain name of Huawei OBS.The endpoint is in the following format: obs.<Region where the bucket resides>.myhuaweicloud.com. For example, if the bucket resides in the CN North-Beijing1 region, the endpoint is obs.cn-north-1.myhuaweicloud.com. For more information about the regions that Huawei OBS supports, see Huawei OBS.

    AccessKeyId

    Yes

    Enter the AccessKey pair of the IAM user that you created. Huawei OBS uses the AccessKey pair to identify the user and verify whether the user has the permissions to read the source data.

    SecretAccessKey

    Yes

    Bucket

    Yes

    Enter the name of the Huawei OBS object storage or parallel file system bucket where the data to be migrated resides.

    Note

    The bucket name cannot start or end with invalid characters such as spaces, line breaks, or tabs.

    Prefix

    No

    You can specify a data path prefix to migrate part of the data.

    • Specify a prefix: migrates the data in the specified directory (prefix). The prefix cannot start with a forward slash (/) and must end with a forward slash (/).

    • Do not specify a prefix: migrates all data in the bucket.

    Tunnel

    No

    Select the name of the channel to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You must associate an agent if the destination is a local file system or if the migration uses a dedicated connection, for example, to a financial cloud or Apsara Stack environment.

    Agent

    No

    Select the name of the agent to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You can select a maximum of 200 agents for a specified channel.

Volcengine TOS

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. On the Create Data Address page, configure the following parameters, and then click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select Volcengine TOS.

    Domain Name

    Yes

    Enter the endpoint domain name of Volcengine TOS.For more information, see Regions and endpoints supported by Volcengine TOS.

    Note

    For the region and domain name, select the corresponding value from the Volcengine TOS table: S3 Endpoint.

    AccessKeyId

    Yes

    Enter the AccessKey pair of the IAM user that you created. Volcengine TOS uses the AccessKey pair to identify the user and verify whether the user has the permissions to read the source data.

    SecretAccessKey

    Yes

    Bucket

    Yes

    Enter the name of the Volcengine TOS bucket where the data to be migrated resides.

    Note

    The bucket name cannot start or end with invalid characters such as spaces, line breaks, or tabs.

    Prefix

    No

    You can specify a data path prefix to migrate part of the data.

    • Specify a prefix: migrates the data in the specified directory (prefix). The prefix cannot start with a forward slash (/) and must end with a forward slash (/).

    • Do not specify a prefix: migrates all data in the bucket.

    Tunnel

    No

    Select the name of the channel to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You must associate an agent if the destination is a local file system or if the migration uses a dedicated connection, for example, to a financial cloud or Apsara Stack environment.

    Agent

    No

    Select the name of the agent to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You can select a maximum of 200 agents for a specified channel.

UCloud

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. On the Create Data Address page, configure the following parameters, and then click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select UCloud US3.

    Domain Name

    Yes

    The endpoint domain name of the bucket. Custom domain names and CDNs are not supported. For more information, see Endpoint list.

    You can log on to the UCloud console, click the name of the bucket that you want to migrate, and choose Overview > Basic Information to view the region where the bucket resides.

    Important

    The preceding steps are for reference only and may be outdated due to changes on the source website.

    Public Key

    Yes

    Enter the public key of the API key that is used for the migration. The key is deleted after the migration is complete.

    Private Key

    Yes

    Enter the private key of the API key that is used for the migration. The key is deleted after the migration is complete.

    Bucket

    Yes

    Select the bucket where the data to be migrated resides.

    Note

    The bucket name cannot start or end with invalid characters such as spaces, line breaks, or tabs.

    Prefix

    No

    You can specify a data path prefix to migrate part of the data.

    • Specify a prefix: migrates the data in the specified directory (prefix). The prefix cannot start with a forward slash (/) and must end with a forward slash (/).

    • Do not specify a prefix: migrates all data in the bucket.

    Tunnel

    No

    Select the name of the channel to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You must associate an agent if the destination is a local file system or if the migration uses a dedicated connection, for example, to a financial cloud or Apsara Stack environment.

    Agent

    No

    Select the name of the agent to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You can select a maximum of 200 agents for a specified channel.

Baidu BOS

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. On the Create Data Address page, configure the following parameters, and click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select Baidu BOS.

    Domain Name

    Yes

    Enter the region where the source address resides. For more information, see Regions and Endpoints.

    AccessKeyId

    Yes

    Enter the access key that is used for the migration. The key is deleted after the migration is complete.

    SecretAccessKey

    Yes

    Bucket

    Yes

    The bucket name of Baidu Cloud BOS object storage.

    The bucket name cannot start or end with invalid characters such as spaces, line breaks, or tabs.

    Prefix

    No

    You can specify a data path prefix to migrate part of the data.

    • Specify a prefix: migrates the data in the specified directory (prefix). The prefix cannot start with a forward slash (/) and must end with a forward slash (/).

    • Do not specify a prefix: migrates all data in the bucket.

    Tunnel

    No

    Select the name of the channel to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You must associate an agent if the destination is a local file system or if the migration uses a dedicated connection, for example, to a financial cloud or Apsara Stack environment.

    Agent

    No

    Select the name of the agent to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You can select a maximum of 200 agents for a specified channel.

Kingsoft KS3

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. On the Create Data Address page, configure the following parameters, and then click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select Kingsoft KS3.

    Domain Name

    Yes

    Enter the domain name that corresponds to the region where the bucket resides. Custom domain names and CDNs are not supported. For more information, see Mapping between endpoints and regions.

    AccessKeyID

    Yes

    Enter the access key that is used for the migration. We recommend that you create a key for the migration service and delete the key after the migration is complete. For more information, see Documentation.

    SecretAccessKey

    Yes

    Bucket

    Yes

    For a Kingsoft Cloud object storage bucket, enter only the custom name. For example, if the bucket name is exampletest-1234567890, enter exampletest.

    Prefix

    No

    You can specify a data path prefix to migrate part of the data.

    • Specify a prefix: migrates the data in the specified directory (prefix). The prefix cannot start with a forward slash (/) and must end with a forward slash (/).

    • Do not specify a prefix: migrates all data in the bucket.

    Tunnel

    No

    Select the name of the channel to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You must associate an agent if the destination is a local file system or if the migration uses a dedicated connection, for example, to a financial cloud or Apsara Stack environment.

    Agent

    No

    Select the name of the agent to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You can select a maximum of 200 agents for a specified channel.

Google Cloud Storage

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. On the Create Data Address page, configure the following parameters, and then click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select Google Cloud Storage.

    Request endpoint (Cloud Storage URI)

    Yes

    The request endpoint (Cloud Storage URI) of the bucket. Custom domain names and CDNs are not supported. You can query the request endpoint on the Interoperability page of the Settings panel in the Cloud Storage console.

    AccessKey ID

    Yes

    To read the data to be migrated, you must provide the AccessKey pair of the source data account. You can query the AccessKey pair on the Interoperability page of the Settings panel in the Cloud Storage console.

    Note

    Security tip: To ensure data security, we strongly recommend that you create a separate key with [Read-only] permissions for the migration service and delete the key after the migration task is complete.

    AccessKey Secret

    Yes

    To read the data to be migrated, you must provide the access key of the source data. You can query the secret of the user account on the Interoperability page of the Settings panel in the Cloud Storage console.

    Note

    Security tip: To ensure data security, we strongly recommend that you create a separate key with [Read-only] permissions for the migration service and delete the key after the migration task is complete.

    Bucket

    Yes

    Enter the name of the Google Cloud Storage bucket where the data to be migrated resides.

    Note

    The bucket name cannot start or end with invalid characters such as spaces, line breaks, or tabs.

    Prefix

    No

    You can specify a data path prefix to migrate part of the data.

    • Specify a prefix: migrates the data in the specified directory (prefix). The prefix cannot start with a forward slash (/) and must end with a forward slash (/).

    • Do not specify a prefix: migrates all data in the bucket.

    Tunnel

    No

    Select the name of the channel to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You must associate an agent if the destination is a local file system or if the migration uses a dedicated connection, for example, to a financial cloud or Apsara Stack environment.

    Agent

    No

    Select the name of the agent to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You can select a maximum of 200 agents for a specified channel.

Youpai USS

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. On the Create Data Address page, configure the following parameters, and then click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select Youpai USS.

    Domain Name

    Yes

    The endpoint domain name of the bucket. For more information, see Service domain names.

    AccessKey

    Yes

    The AWS S3 access credential. Authentication is performed based on the S3 protocol standard to verify whether the credential has the permissions to read data from the bucket. The AccessKey and SecretAccessKey that are required to calculate the signature can be obtained in the console from "Operator - Edit" or "Cloud Storage - Configuration - Storage Management - Operator Authorization - S3 Access Credential". For more information, see AWS S3 Compatibility.

    SecretAccessKey

    Yes

    Prefix

    Yes

    Specify a prefix to migrate a subset of the data.

    • If specified, only data under that prefix is migrated. The prefix must not start with a forward slash (/) but must end with one.

    • If you omit this parameter, all data in the bucket is migrated.

    Tunnel

    No

    Select the name of the channel to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You must associate an agent if the destination is a local file system or if the migration uses a dedicated connection, for example, to a financial cloud or Apsara Stack environment.

    Agent

    No

    Select the name of the agent to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You can select a maximum of 200 agents for a specified channel.

Azure Blob

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. On the Create Data Address page, configure the following parameters, and then click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select Azure Blob.

    Domain Name

    No

    The endpoint of Azure Blob. Custom domain names and CDN are not supported. Example: Storage Account.blob.core.windows.net.

    SAS Token

    Yes

    To read the data to be migrated, you must provide a SAS token. View the SAS token on the shared access signature page of the storage account in the console.

    Note

    Security tip: To ensure data security, we strongly recommend that you create a separate SAS token with [Read-only] permissions for the migration service and revoke the token after the migration task is complete.

    Container

    Yes

    Enter the name of the Azure Blob container where the data to be migrated resides.

    Note

    The container name cannot start or end with invalid characters such as spaces, line breaks, or tabs.

    Prefix

    No

    You can specify a data path prefix to migrate part of the data.

    • Specify a prefix: migrates the data in the specified directory (prefix). The prefix cannot start with a forward slash (/) and must end with a forward slash (/).

    • Do not specify a prefix: migrates all data in the bucket.

    Tunnel

    No

    Select the name of the channel to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You must associate an agent if the destination is a local file system or if the migration uses a dedicated connection, for example, to a financial cloud or Apsara Stack environment.

    Agent

    No

    Select the name of the agent to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You can select a maximum of 200 agents for a specified channel.

LocalFS

Important
  • If the LocalFS source data is stored in local files on the machine, you can deploy only one agent.

  • If the LocalFS source data is stored in local files mounted from Apsara File Storage NAS and is mounted on multiple machines, make sure that the NAS file system is mounted to directories with the same name. When you create the data address, set the directory to be migrated to the directory to which the NAS file system is mounted.

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. In the Create Address panel, configure the following parameters, and then click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select LocalFS.

    Directory To Be Migrated

    Yes

    You can specify the path of the directory to be migrated.This path must begin and end with a forward slash (/) and cannot contain environment variables or special characters.

    For example, if you set the prefix of the source data address to /example/src/ and this path contains the file example.jpg. If the prefix of the destination data address is set to example/dest/, the full path of the file example.jpg after migration is example/dest/example.jpg.

    Important
    • If the data address is associated with multiple agents, make sure that each agent can access the directory. Otherwise, some data is not migrated.

    • If the LocalFS source data is stored in local files mounted from Apsara File Storage NAS and is mounted on multiple machines, make sure that the NAS file system is mounted to local directories with the same name. When you create the data address, set the directory to be migrated to the name of the local directory to which the NAS file system is mounted.

    Tunnel

    Yes

    Select the name of the channel to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You must associate an agent if the destination is a local file system or if the migration uses a dedicated connection, for example, to a financial cloud or Apsara Stack environment.

    Agent

    Yes

    Select the name of the agent to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You can select a maximum of 200 agents for a specified channel.

OSSHDFS

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. In the Create Address panel, configure the following parameters, and then click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select OSSHDFS.

    Domain Name

    Yes

    Enter the endpoint that is used to access OSS-HDFS.

    • Format: <region-id>.oss-dls.aliyuncs.com. For example, the endpoint of the China (Hangzhou) region is cn-hangzhou.oss-dls.aliyuncs.com.

    AccessKeyId

    Yes

    Enter the access key of the IAM user that you created. The access key is used to identify OSS-HDFS. Make sure that the user has write permissions on the destination data.

    SecretAccessKey

    Yes

    Bucket

    Yes

    Enter the name of the bucket that stores the data to be migrated in the current account.

    Directory To Be Migrated

    Yes

    To migrate part of the data, specify a data path that starts and ends with a forward slash (/), such as /data/to/osshdfs/. To migrate all files in the bucket, set the value to /.

    Tunnel

    No

    Select the name of the tunnel that you want to use.

    Important
    • This parameter is required only when you migrate data to the cloud over an Express Connect circuit or a VPN, or when you migrate data from FTP, HDFS, or self-managed storage to the cloud.

    • You must associate an agent when the destination data address is of the LOCALFS type, or when the migration must go through an Express Connect circuit, such as in Finance Cloud or Apsara Stack scenarios.

    Agent

    No

    Select the name of the agent that you want to use.

    Important
    • This parameter is required only when you migrate data to the cloud over an Express Connect circuit or a VPN, or when you migrate data from FTP, HDFS, or self-managed storage to the cloud.

    • You can select up to 30 agents at a time for a specified tunnel.

FTP

  1. In the left-side navigation pane, choose Data Online Migration > Address Management, and then click Create Address.

  2. In the Create Data Address panel, configure the following parameters, and click OK.

    Parameter

    Required

    Description

    Name

    Yes

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select FTP.

    Connection Address

    Yes

    The IP address and port number of the FTP server. The address of the FTP server is a concatenation of the IP address and the port number. That is, the value of this parameter is in the {hostname or IP address}:{port} format, for example, 192.168.16.13:21.

    Directory To Be Migrated

    Yes

    • You can specify the path of the directory to be migrated.This path must begin and end with a forward slash (/) and cannot contain environment variables or special characters.

      For example, if you set the prefix of the source data address to /example/src/ and this path contains the file example.jpg. If the prefix of the destination data address is set to example/dest/, the full path of the file example.jpg after migration is example/dest/example.jpg.

    Username

    Yes

    Enter the FTP username and password.

    Password

    Yes

    Tunnel

    Yes

    Select the name of the tunnel that you want to use.

    Important
    • This parameter is required only when you migrate data to the cloud over an Express Connect circuit or a VPN, or when you migrate data from FTP or self-managed storage to the cloud.

    • You must associate an agent when the destination data address is of the LOCALFS type, or when the migration must go through an Express Connect circuit, such as in Finance Cloud or Apsara Stack scenarios.

    Agent

    Yes

    Select the name of the agent that you want to use.

    Important
    • This parameter is required only when you migrate data to the cloud over an Express Connect circuit or a VPN, or when you migrate data from FTP or self-managed storage to the cloud.

    • You can select up to 30 agents at a time for a specified tunnel.

Step 3: Create a destination address

  1. In the navigation pane on the left, choose Data Online Migration > Address Management, and then click Create Address.

  2. In the Create Address panel, configure the following parameters, and then click OK.

  3. Parameter

    Required

    Description

    Name

    Yes

    Enter a name for the destination data address. The naming rules are as follows:

    • The name must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name cannot start with a hyphen (-) or an underscore (_).

    Type

    Yes

    Select Alibaba OSS.

    Custom Domain Name

    No

    Custom domain names are supported.

    Region

    Yes

    Select the region where the destination address resides, for example, China (Hangzhou).

    Authorize Role

    Yes

    Bucket

    Yes

    Enter the name of the bucket that stores the migrated data in the current account.

    Prefix

    No

    You can specify a data path prefix to migrate the source data to a specified directory. The prefix cannot start with a forward slash (/) and must end with a forward slash (/). For example, data/to/oss/.

    • Specify a prefix: for example, if the prefix of the source data address is example/src/ and this path contains the file example.jpg. If the prefix of the destination data address is set to example/dest/, the full path of the file example.jpg after migration is example/dest/example.jpg.

    • Do not specify a prefix: if you do not set a data path prefix, the source data is migrated to the root directory of the destination bucket.

    Tunnel

    No

    Select the name of the channel to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You must associate an agent if the destination is a local file system or if the migration uses a dedicated connection, for example, to a financial cloud or Apsara Stack environment.

    Agent

    No

    Select the name of the agent to use.

    Important
    • This parameter is required only when migrating data over an Express Connect or VPN connection, or when migrating data from a self-managed storage system.

    • You can select a maximum of 200 agents for a specified channel.

Step 4: Create a migration task

  1. In the left-side navigation pane, choose Data Online Migration > Migration Tasks, and then click Create Task.

  2. On the Select Address page, configure the following parameters, and then click Next.

    Parameter

    Required

    Description

    Name

    Yes

    Enter a name for the migration task. The naming rules are as follows:

    • The name cannot be empty and must be 3 to 63 characters in length.

    • The name can contain lowercase letters, digits, hyphens (-), and underscores (_). The name is case-sensitive.

    • The name is encoded in UTF-8 and cannot start with a hyphen (-) or an underscore (_).

    Source Address

    Yes

    Select the source address that you created.

    Destination Address

    Yes

    Select the destination address that you created.

  3. On the Task Configurations page, configure the following parameters.

Parameter

Required

Description

Basic Settings

Migration Bandwidth

No

Select the migration bandwidth.

  • Default: the maximum bandwidth. The actual speed depends on the file size and the number of files.

  • Specify an upper limit: specify a bandwidth upper limit as prompted in the console.

Important
  • The actual migration bandwidth depends on factors such as the data source, network, destination throttling, and file size, and may not reach the specified upper limit.

  • Evaluate your data source, migration destination, business conditions, and network bandwidth, and then select an appropriate value based on your actual situation. Improper throttling may affect your workloads.

Files Migrated Per Second

No

Select the number of files migrated per second.

  • Default: the default number of files migrated per second.

  • Specify an upper limit: specify the number of files migrated per second as prompted in the console.

Important
  • The actual migration bandwidth depends on factors such as the data source, network, destination throttling, and file size, and may not reach the specified upper limit.

  • Evaluate your data source, migration destination, business conditions, and network bandwidth, and then select an appropriate value based on your actual situation. Improper throttling may affect your workloads.

Overwrite Mode

Yes

Select the overwrite mode for files with the same name.

  • Do not overwrite: the file is not migrated.

  • Overwrite All: files at the source address overwrite files at the destination address.

  • Overwrite based on the last modification time:

    • If the last modified time of the file at the source address is later than that of the file at the destination address, the file at the destination address is overwritten.

    • If the last modified time of the file at the source address is the same as that of the file at the destination address, and the Size or Content-Type of the two files differs, the file at the destination address is overwritten.

  • Warning
    • Overwrite based on the last modification time cannot strictly guarantee that newer files are never overwritten. A risk exists that an older file overwrites a newer one.

    • If you select the Overwrite based on the last modification time overwrite mode, make sure that the source file can return information such as Last Modified Time, Size, and Content-Type. Otherwise, the overwrite policy may fail and produce unexpected migration results.

    • If you select Do not overwrite or Overwrite based on the last modification time, the system requests the metadata of the source and the destination once each to perform the subsequent overwrite evaluation. As a result, corresponding request fees are incurred at both the source and the destination.

Audit Mode

Migration Report

Yes

The method that is used to push migration reports.

  • Do not push (default): Migration reports are not pushed to the destination bucket.

  • Push: Migration reports are pushed to the destination bucket. For the detailed path, see What to do next.

Important
  • Pushing migration reports occupies storage space at the destination.

  • Migration reports may be pushed with a delay. Wait for the reports to be generated.

  • Each task execution record has a unique ID. Note that a migration report is pushed only once. Delete reports with caution.

Migration Logs

Yes

The method that is used to push migration logs.

  • Do not push (default): Migration logs are not pushed.

  • Push: Migration logs are pushed to Log Service (SLS). You can view the migration logs in SLS.

  • Push only file error logs.: Only error migration logs are pushed to Log Service (SLS). You can view the error migration logs in SLS.

When you select Push or Push only file error logs., Data Online Migration creates a project in Log Service named aliyun-oss-import-log-{Alibaba Cloud account ID}-{current region where the migration service is deployed}. Example: aliyun-oss-import-log-137918634953****-cn-hangzhou.

Important

Make sure that you have completed the following operations before you select Push or Push only file error logs.. Otherwise, the migration task may run abnormally.

  • Log Service is activated.

  • You have granted authorization on the authorization page of the Authorize item.

Authorize

No

This option is displayed when Migration Logs is set to Push or Push only file error logs..

Click Authorize to go to the Cloud Resource Access Authorization page. The system creates the AliyunOSSImportSlsAuditRole role on the page and grants the required permissions to the role. Click Agree to Authorize to complete the authorization.

Filter

File Name

No

The file name filter.

Two filter rules are supported: Include and Exclude. For more information about the regular expression syntax, see the RE2 library. Only part of the syntax is supported. For example:

  • .*\.jpg$ indicates all files whose names end with .jpg.

  • ^file.* indicates, by default, all files in the root directory whose names start with file.

    If a prefix is set for the source data address, for example, data/to/oss/, you must use ^data/to/oss/file.* to match all files under the specified prefix whose names start with file.

  • .*/picture/.* indicates subdirectories at a level named picture.

Important
  • When the filter rule is set to Include, all files that match the rule are migrated. If multiple rules are configured, a file that matches any of the rules is migrated.

    For example, given two files picture.jpg and picture.png, if you set an Include filter rule to .*\.jpg$, only picture.jpg is migrated. If you also set an Include filter rule to .*\.png$, both files are migrated.

  • When the filter rule is set to Exclude, no files that match the rule are migrated. If multiple rules are configured, a file that matches any of the rules is not migrated.

    For example, given two files picture.jpg and picture.png, if you set an Exclude filter rule to .*\.jpg$, only picture.png is migrated. If you also set an Exclude filter rule to .*\.png$, neither file is migrated.

  • Exclude rules take precedence. If a file matches both an Exclude rule and an Include rule, the file is not migrated.

    For example, for the file file.txt, if you set an Exclude filter rule to .*\.txt$ and an Include filter rule to file.*, file.txt is not migrated.

File Modification Time

No

The filter based on the last modified time of files.

You can specify the last modified time of files as a filter rule. If you specify a time range, only files whose last modified time falls within the specified range are migrated. The rules are as follows:

  • If you specify only the start time as January 1, 2019 and do not specify an end time, only files whose last modified time is later than or equal to January 1, 2019 are migrated.

  • If you specify only the end time as January 1, 2022 and do not specify a start time, only files whose last modified time is earlier than or equal to January 1, 2022 are migrated.

  • If you specify the start time as January 1, 2019 and the end time as January 1, 2022, only files whose last modified time is later than or equal to January 1, 2019 and earlier than or equal to January 1, 2022 are migrated.

Whether to Migrate

No

Important

This option is supported only when the source is LocalFS.

Specifies whether to migrate entities of special types. If you select the check box, the migration is enabled. If you clear the check box, the migration is disabled.

Directory:

  • When this option is enabled, all directories that are scanned at the source address are also added to the migration queue and are counted in the Files and Storage fields of the migration task. An empty object whose name ends with '/' is created at the destination address, and the attributes of the source directory (if the directory can be migrated) are written to the UserMeta of the destination object.

  • When this option is disabled, all directories that are scanned at the source address are ignored and are not counted in the Files and Storage fields of the migration task. No empty object whose name ends with '/' is created at the destination address.

Symbolic Link:

  • When this option is enabled, all symbolic link files at the source address are also added to the migration queue and are counted in the Files and Storage fields of the migration task. A corresponding Symlink object is created at the destination address, and the attributes of the source symbolic link file (if the file can be migrated) are written to the UserMeta of the Symlink object. The value of the Target attribute of the Symlink object depends on the Convert Destination Path Or Not option.

  • When this option is disabled, all symbolic link files at the source address are ignored and are not counted in the Files and Storage fields of the migration task.

Important

In all cases, the target files or directories to which the symbolic link files at the source address point are not additionally migrated, unless the target files or directories are also within the scope of this migration.

Migration Configurations

Convert Destination Path Or Not

No

Important

This option is supported only when the source is LocalFS.

Converts the Target attribute of the symbolic link files at the source address so that the migrated Symlink objects can still point to their target objects wherever possible. If you select the check box, the conversion is enabled. If you clear the check box, the conversion is disabled.

Important

  • This option takes effect only when symbolic link migration is enabled.

  • Regardless of whether the Target attribute is converted, the migration never checks whether the target object exists, whether the type of the target object is valid, or whether you have the permissions to access the target object.

When this option is enabled, the Target attribute is first resolved into the shortest absolute path, that is, the equivalent path with the fewest characters (AbsTarget), relative to the directory where the symbolic link resides. A string replacement is then performed to replace SrcPrefix in AbsTarget with DestPrefix if SrcPrefix is matched. The replaced value is set as the Target attribute of the destination Symlink object.

Description

Example: assume that SrcPrefix is set to "/mnt/nas1/" and DestPrefix is set to "cloud_base/" for the migration task, and that /mnt/nas1/links/a.lnk is a symbolic link file at the source address. Assume that the Target attribute of the symbolic link file:

  • is "../data/./a.txt". The shortest absolute path after resolution is "/mnt/nas1/data/a.txt", and the final Target value after replacement is "cloud_base/data/a.txt".

  • is "/mnt/nas1/verbose/../data/./a.txt". The shortest absolute path after resolution is "/mnt/nas1/data/a.txt", and the final Target value after replacement is "cloud_base/data/a.txt".

  • is "/root/outer/../data/./a.txt". The shortest absolute path after resolution is "/root/data/a.txt". The replacement does not match, so the final Target value remains "/root/data/a.txt".

When this option is disabled, no conversion is performed, and the original value of the Target attribute of the source symbolic link is set as the Target attribute of the destination Symlink object.

Retain Last Modified Time

Yes

Specifies whether to retain the last modified time of source files.

  • Yes (default): The last modified time of the source file is applied to the destination.

  • No: The last modified time is not applied.

Convert from Appendable to Normal or Multipart

No

Specifies whether to convert files of the Appendable type to the Normal or Multipart type.

  • Yes: files of the Appendable type at the source are converted to the Normal or Multipart type at the destination.

  • No: no conversion is performed. Files of the Appendable type at the source remain the Appendable type at the destination.

Important
  • This option is displayed on the page and can be configured only if the whitelist access permission is enabled.

  • Currently, this option is supported only for tasks that migrate data from OSS to OSS.

Add Target Storage Class

No

Specifies whether to specify the storage class of the destination files.

  • Specify: the files that are migrated to the destination are set to the storage class that you specify. The following storage classes can be specified:

    • Standard

    • Infrequent Access (IA)

    • Archive

    • Cold Archive

    • Deep Cold Archive

  • Do Not Specify (default): the storage class is not set. The files that are migrated to the destination use the default storage class of the destination.

Important
  • This option is displayed on the page and can be configured only if the whitelist access permission is enabled.

  • Currently, this option is supported only for tasks whose destination is OSS.

Task Scheduling

Execution Time

No

Important
  1. If a running task has not finished the current round before the next scheduled time, the task automatically continues at the next scheduled time after the current round ends, until the specified number of executions is complete.

  2. Limit on concurrent migration tasks: A maximum of 10 tasks can run concurrently when the migration service is deployed in China (Hong Kong) or a Chinese mainland region, and a maximum of 5 tasks can run concurrently in regions outside the Chinese mainland. If the limit is exceeded, scheduled tasks may not run as expected.

Specifies the execution time of the migration task.

  • Immediately: The task is executed immediately.

  • At the Specified Time: Specify a daily execution window for the task. By default, the task starts at the specified start time and pauses at the specified stop time.

  • Periodic Scheduling Start the task by configuring the execution frequency and the number of executions.

    • Execution Frequency: Five frequencies are supported: Every Hour, Every Day, Every Week, Certain Days of the Week, and Custom. For more information, see Execution frequency.

    • Number of Executions: Specify the number of times that the task is executed. If you do not set this parameter, the task is executed once by default. For the maximum number of executions, see the prompt in the console.

Important

You can manually start and pause the task at any time, regardless of the custom execution time.

  1. Read the Online Migration Service Agreement, select I have read and agree to the Compliance Commitment and that when the migration task is completed, I have the obligation and responsibility to confirm the consistency of the migrated data, and then click Next.

  2. Confirm that the configuration is correct, and then click OK. Wait for the migration task to run.

Execution frequency

Execution frequency

Description

Example

Hourly

Run the task once every hour. You can use this option with the maximum number of runs.

The current time is 8:05. The frequency is set to hourly with a maximum of 3 runs. The first run starts at the next hour, 9:00.

  • If a run finishes before the next hour, the second run starts at 10:00. This pattern continues until the specified number of runs is complete.

  • If a run has not finished by the next hour and ends at 12:30, the second run starts at the next hour, 13:00. This pattern continues until the specified number of runs is complete.

Daily

Run the task once a day. You must specify an hour (0-23) for the task to start. You can use this option with the maximum number of runs.

The current time is 8:05. The task is scheduled to run daily at 10:00, with a maximum of 5 runs. The first run starts at 10:00 today.

  • If a run finishes before 10:00 the next day, the second run starts at 10:00 the next day. This pattern continues until the specified number of runs is complete.

  • If a run has not finished by 10:00 the next day and ends at 12:05 the next day, the second run starts at 10:00 on the third day. This pattern continues until the specified number of runs is complete.

Weekly

Run the task once a week. You must specify a day of the week and an hour (0-23) for the task to start. You can use this option with the maximum number of runs.

The current time is Monday, 8:05. The task is scheduled to run every Monday at 10:00, with a maximum of 10 runs. The first run starts at 10:00 today.

  • If a run finishes before 10:00 next Monday, the second run starts at 10:00 next Monday. This pattern continues until the specified number of runs is complete.

  • If a run has not finished by 10:00 next Monday and ends at 12:05 next Monday, the second run starts at 10:00 on the following Monday. This pattern continues until the specified number of runs is complete.

Specific days of the week

Run the task on selected days of the week. You must specify the days and an hour (0-23) for the task to start.

The current time is Wednesday, 8:05. The task is scheduled to run on Mondays, Wednesdays, and Fridays at 10:00. The first run starts at 10:00 today.

  • If a run finishes before 10:00 on Friday, the second run starts at 10:00 on Friday. This pattern continues until the specified number of runs is complete.

  • If a run has not finished by 10:00 on Friday and ends at 12:05 next Monday, the second run starts at 10:00 next Wednesday. This pattern continues until the specified number of runs is complete.

Custom

Use a cron expression to define a custom schedule for the task start time.

Note

A cron expression consists of six space-separated fields that define the execution schedule: second, minute, hour, day of the month, month, and day of the week. The minimum interval is 1 hour.

The following cron expression examples are for reference only. For more options, use a cron expression generator.

  • 0 0 * * * *: Runs the task at the beginning of every hour (0 minutes, 0 seconds).

  • 0 30 0/3 * * ?: Runs the task every 3 hours at 30 minutes past the hour (for example, at 0:30, 3:30, 6:30, 9:30, 12:30, 15:30, 18:30, and 21:30).

  • 0 0 12 * * MON-FRI: Runs the task at 12:00 PM every weekday from Monday to Friday.

  • 0 0 12 1-15 * SAT,SUN: Runs the task at 12:00 PM on weekends (Saturday and Sunday) that fall between the 1st and 15th of the month.

  • 0 30 8 1,15 * *: Runs the task at 8:30 AM on the 1st and 15th of each month.

Step 5: Verify data

The migration service only transfers your data and does not guarantee data consistency or integrity. After the migration task completes, you must verify data consistency between the source and the destination.

Warning

After the migration task completes, you must verify the data at the destination. Verify that all of your data has been completely migrated to the destination before deleting the source data. If you delete the source data without verifying it, you are responsible for any resulting data loss.