Migrate data from an HTTP or HTTPS server to OSS

Precautions

When you use Data Online Migration:

Online Migration Service accesses source data through the source provider's public API. Service behavior depends on the provider's API implementation.
Migration consumes resources at both source and destination, which may affect your workloads. For mission-critical services, set a rate limit or run the task during off-peak hours.
The service checks files at source and destination before migration. If a same-name file exists at both locations and the task is configured to overwrite, the destination file is overwritten directly. Back up or rename files with different content to prevent data loss.
Online migration preserves the last modified time of source files. If a lifecycle rule is configured on the destination bucket, it may delete or transition migrated files based on their last modified time.

Migration limitations

Object attributes during HTTP/HTTPS migration:

These attributes are preserved: LastModifyTime, Content-Type, Cache-Control, Content-Encoding, Content-Disposition, Content-Language, and Expires.
Other attributes may not be migrated. Actual values are determined at migration time.

Step 1: Select a region

Log in to the Data Online Migration console as the RAM user you created.
In the top navigation bar, use the region selector in the upper-left corner to select the migration service deployment region. Choose the data source region or the geographically closest region.

Available deployment regions: China (Beijing), China (Shanghai), China (Hangzhou), China (Shenzhen), and China (Ulanqab) in mainland China, and China (Hong Kong), Singapore (Singapore), Germany (Frankfurt), and US (Virginia).
Important
- Data source addresses and migration tasks are specific to each region. Choose your region carefully.
- Select the region of your data source. If that region is unavailable, create the migration task in the geographically closest region.
- For cross-border migration, enable Transfer Acceleration to improve speed. Buckets with Transfer Acceleration enabled incur transfer acceleration fees. Access OSS by using Transfer Acceleration.

Step 2: Create a source address

In the left-side navigation pane, go to Data Online Migration > Address Management and click Create Address.
In the Create Address panel, set the following parameters and click OK.

Parameter	Required	Description
Name	Yes	Enter a name for the source. The name must meet the following requirements: The name must be 3 to 63 characters in length. The name is case-sensitive and can contain only lowercase letters, digits, hyphens (-), and underscores (_). The name cannot start with a hyphen (-) or an underscore (_).
Type	Yes	Select HTTP/HTTPS.
Protocol	Yes	Select HTTP or HTTPS.
Port	Yes	Enter the port number. Default: 80 (HTTP) or 443 (HTTPS). Valid values: 0 to 65535.
Inventory Location	Yes	Select the service that stores the inventory file: Alibaba Cloud OSS or AWS S3.
Inventory Path	Yes	Enter the path to the manifest.json file.
Inventory Domain Name or Region	Yes	If you set the Inventory Location parameter to Alibaba OSS, specify the region in which the OSS inventory list resides. If Inventory Location is set to AWS S3, enter the domain name used to access the AWS S3 inventory. For more information, see Amazon S3 endpoint.
Role	Yes (if Inventory Location is Alibaba Cloud OSS)	If the inventory bucket is in your current account: Authorize a role in the migration console. Manually authorize a role in the migration console (alternative). If the inventory bucket is in a different account: Authorize a role in the OSS console.
Bucket	Yes	Enter the inventory bucket name. Must be in your current Alibaba Cloud account.
Inventory AccessKeyId	Yes (if Inventory Location is AWS S3)	If Inventory Location is set to AWS S3, enter the access key used to access the AWS S3 inventory list. For security, delete this key after the migration is complete.
Inventory SecretAccessKey	Yes (if Inventory Location is AWS S3)
Tunnel	No	Select the channel that you want to use. Important This parameter is required only when you migrate data from self-managed storage to the cloud, or when you migrate data over a dedicated connection or VPN. An agent is required when the destination is a local file system (LocalFs) or when migrating over a dedicated connection for services like Finance Cloud or Apsara Stack.
Agent	No	Select one or more agents. Important This parameter is required only when you migrate data from self-managed storage to the cloud, or when you migrate data over a dedicated connection or VPN. You can select up to 200 agents for a specified channel.

Step 3: Create a destination address

In the left-side navigation pane, go to Data Online Migration > Address Management and click Create Address.
In the Create Address panel, set the following parameters and click OK.

Parameter	Required	Description
Name	Yes	Enter a destination address name. The name must meet these requirements: The name must be 3 to 63 characters in length. The name is case-sensitive and can contain only lowercase letters, digits, hyphens (-), and underscores (_). The name cannot start with a hyphen (-) or an underscore (_).
Type	Yes	Select Alibaba OSS.
Custom domain name	No	Optional custom domain name for accessing the destination OSS bucket.
Region	Yes	Select the destination bucket region, such as China (Hangzhou).
Authorize Role	Yes	If the destination bucket is in your current account: Authorize a role from the migration console (Recommended). Manually authorize a role from the migration console (Alternative). If the destination bucket is in a different account: Authorize a role from the OSS console.
Bucket	Yes	Enter the destination bucket name. Must be in your current Alibaba Cloud account.
Prefix	No	Specify a prefix to migrate data to a specific directory. The prefix cannot start with a forward slash (/) but must end with one. Example: `data/to/oss/`. With prefix: If the source prefix is `example/src/` and contains example.jpg, setting the destination prefix to `example/dest/` migrates the file to `example/dest/example.jpg`. Without prefix: Data is migrated to the bucket root directory.
Tunnel	No	Select the channel that you want to use. Important This parameter is required only when you migrate data from self-managed storage to the cloud, or when you migrate data over a dedicated connection or VPN. An agent is required when the destination is a local file system (LocalFs) or when migrating over a dedicated connection for services like Finance Cloud or Apsara Stack.
Agent	No	Select one or more agents. Important This parameter is required only when you migrate data from self-managed storage to the cloud, or when you migrate data over a dedicated connection or VPN. You can select up to 200 agents for a specified channel.

Step 4: Create a migration task

In the navigation pane on the left, choose Data Online Migration > Migration Tasks, and then click Create Task.

On the Select Address page, configure the following parameters, and then click Next.

Parameter	Required	Description
Name	Yes	Enter a name for the migration task. The name must meet the following requirements: The name must be 3 to 63 characters in length. The name is case-sensitive and can contain only lowercase letters, digits, hyphens (-), and underscores (_). The name cannot start with a hyphen (-) or an underscore (_).
Source Address	Yes	Select a previously created source address.
Destination Address	Yes	Select a previously created destination address.

On the Task Configurations page, set the following parameters.

Parameter	Required	Description
Basic settings
Migration Bandwidth	No	Migration bandwidth setting. Default: Uses the maximum available bandwidth. The actual migration speed depends on the file size and the number of files. Specify an upper limit: Specify a bandwidth cap as prompted on the console. Important Actual migration bandwidth depends on the data source, file sizes, network conditions, and destination-side throttling. The bandwidth may not reach the specified upper limit. Evaluate your data source, destination, workloads, and network bandwidth before setting this value. Improper throttling may affect your business.
Files Migrated Per Second	No	Files migrated per second. Default: The default number of files migrated per second. Specify an upper limit: Specify an upper limit as prompted on the console. Important Actual migration rate depends on the data source, file sizes, network conditions, and destination-side throttling. The rate may not reach the specified upper limit. Evaluate your data source, destination, workloads, and network bandwidth before setting this value. Improper throttling may affect your business.
Overwrite Mode	No	How to handle files with the same name at the destination. Do not overwrite: Skips migrating the file. Overwrite All: The source file overwrites the destination file. Overwrite based on the last modification time: The destination file is overwritten if the source file's last modified time is later. If the last modified times are the same, the destination file is overwritten if its Size or Content-Type differs. Warning The Overwrite based on the last modification time policy does not guarantee that an older file will not overwrite a newer one. If you select Overwrite based on the last modification time, ensure your source data can return metadata such as last modified time, Size, and Content-Type. Otherwise, the overwrite policy may not work as expected and can lead to unintended migration results. If you select Do not overwrite or Overwrite based on last modified time, the service requests object metadata from both the source and destination to perform the comparison. This incurs request fees on both the source and destination.
Auditing
Migration Report	Yes	Delivery method for the migration report. Do not push (Default): The report is not delivered to the destination bucket. Push: The report is delivered to the destination bucket. Next steps. Important Reports consume storage space at the destination. Report delivery may be delayed. Each execution has a unique ID. A report is pushed once per execution. Use caution when deleting reports.
Migration Logs	Yes	Migration log delivery method. Do not push (Default): The migration log is not pushed. Push: Pushes the migration log to Log Service. You can view the migration log in Log Service. Push only file error logs.: Pushes only logs for file migration errors to Log Service. You can view these error logs in Log Service. If you select Push or Push only file error logs., Online Migration Service creates a project in Log Service named aliyun-oss-import-log-Alibaba Cloud account ID-current region. For example: aliyun-oss-import-log-137918634953**-cn-hangzhou. Important Ensure that you complete the following actions before selecting Push or Push only file error logs.. Otherwise, the migration task may fail. You have activated Log Service. You have granted the required permissions on the Authorize** page.
Authorize	No	This option appears only when Migration Logs is set to Push or Push only file error logs.. Click Authorize to go to the Cloud Resource Access Authorization page. The system creates a role named AliyunOSSImportSlsAuditRole and grants permissions to the role. Click Agree to Authorization to complete the authorization.
Migration configuration
Preserve file last modified time	Yes	Preserve the last modified time of source files. Preserve (Default): Applies the source file's last modified time to the destination object. Do not preserve: The last modified time is not set.
Specify storage class for destination objects	No	Set a storage class for destination objects. Specify: Assign a storage class to migrated objects: Standard Infrequent Access Archive Cold Archive Deep Cold Archive Do not specify (Default): Objects use the default storage class of the destination bucket. Important Available only after your account is added to the allowlist. Supported only for migration tasks where the destination is OSS.
Task scheduling
Execution time	No	Important If a task is still running when its next execution is scheduled, the current run will complete, the scheduled run is skipped, and the task will execute at the next interval. Concurrent migration task limit: Up to 10 in Chinese mainland and China (Hong Kong) regions, and up to 5 in other regions. Specify when to run the migration task. Immediately: Runs the task immediately. At the Specified Time: Sets a daily time window for the task to run. By default, the task starts at the specified start time and pauses at the specified stop time. Periodic Scheduling: Runs the task based on a specified frequency and number of executions. Execution Frequency: Supported frequencies are Hourly, Daily, Weekly, Specific days of the week, and Custom. Execution frequency. Number of Executions: Specifies the number of times the task runs. If not set, the task runs once by default. For the maximum number of executions, refer to the prompt on the console. Important You can manually start and pause the task at any time, regardless of the scheduled execution time.

Read the Online Migration Service Agreement, select the checkbox for I have understood and confirmed the compliance commitment statement, and I acknowledge my obligation and responsibility to verify the consistency of migrated data after the migration task is completed, and then click Next.
Review the configuration information. If it is correct, click OK and wait for the migration task to run.

Execution frequency

Execution frequency	Description	Example
Hourly	Run the task once every hour. You can use this option with the maximum number of runs.	The current time is 8:05. The frequency is set to hourly with a maximum of 3 runs. The first run starts at the next hour, 9:00. If a run finishes before the next hour, the second run starts at 10:00. This pattern continues until the specified number of runs is complete. If a run has not finished by the next hour and ends at 12:30, the second run starts at the next hour, 13:00. This pattern continues until the specified number of runs is complete.
Daily	Run the task once a day. You must specify an hour (0-23) for the task to start. You can use this option with the maximum number of runs.	The current time is 8:05. The task is scheduled to run daily at 10:00, with a maximum of 5 runs. The first run starts at 10:00 today. If a run finishes before 10:00 the next day, the second run starts at 10:00 the next day. This pattern continues until the specified number of runs is complete. If a run has not finished by 10:00 the next day and ends at 12:05 the next day, the second run starts at 10:00 on the third day. This pattern continues until the specified number of runs is complete.
Weekly	Run the task once a week. You must specify a day of the week and an hour (0-23) for the task to start. You can use this option with the maximum number of runs.	The current time is Monday, 8:05. The task is scheduled to run every Monday at 10:00, with a maximum of 10 runs. The first run starts at 10:00 today. If a run finishes before 10:00 next Monday, the second run starts at 10:00 next Monday. This pattern continues until the specified number of runs is complete. If a run has not finished by 10:00 next Monday and ends at 12:05 next Monday, the second run starts at 10:00 on the following Monday. This pattern continues until the specified number of runs is complete.
Specific days of the week	Run the task on selected days of the week. You must specify the days and an hour (0-23) for the task to start.	The current time is Wednesday, 8:05. The task is scheduled to run on Mondays, Wednesdays, and Fridays at 10:00. The first run starts at 10:00 today. If a run finishes before 10:00 on Friday, the second run starts at 10:00 on Friday. This pattern continues until the specified number of runs is complete. If a run has not finished by 10:00 on Friday and ends at 12:05 next Monday, the second run starts at 10:00 next Wednesday. This pattern continues until the specified number of runs is complete.
Custom	Use a cron expression to define a custom schedule for the task start time.	Note A cron expression consists of six space-separated fields that define the execution schedule: second, minute, hour, day of the month, month, and day of the week. The minimum interval is 1 hour. The following cron expression examples are for reference only. For more options, use a cron expression generator. `0 0 * * * `: Runs the task at the beginning of every hour (0 minutes, 0 seconds). `0 30 0/3 * ?`: Runs the task every 3 hours at 30 minutes past the hour (for example, at 0:30, 3:30, 6:30, 9:30, 12:30, 15:30, 18:30, and 21:30). `0 0 12 * * MON-FRI`: Runs the task at 12:00 PM every weekday from Monday to Friday. `0 0 12 1-15 * SAT,SUN`: Runs the task at 12:00 PM on weekends (Saturday and Sunday) that fall between the 1st and 15th of the month. `0 30 8 1,15 * *`: Runs the task at 8:30 AM on the 1st and 15th of each month.

Step 5: Validate data

Migration Service transfers data but does not guarantee consistency or integrity. After migration completes, validate all migrated data to ensure consistency between source and destination.

Warning

After the migration task is complete, you must verify the migrated data at the destination. You are solely responsible for any data loss and all associated consequences if you delete the source data before confirming the integrity of the destination data.