Download a large file to a local hard disk
This topic describes how to use ossutil to download a large object to a hard disk.
Scenarios
Unreliable network connections may interrupt large object downloads and cause inconvenience. In scenarios in which an external hard disk is used to store the object, the data transmission is easily interrupted. ossutil supports resumable download, which you can use to resume interrupted download tasks and ensure complete and efficient downloads.
By default, resumable download is disabled in ossutil 2.0. You must enable resumable download when you download large objects.
Prerequisites
-
Object Storage Service (OSS) is activated.
-
A bucket is created.
-
ossutil 2.0 is installed.
Resumable download mechanism
To download a large object, ossutil splits the object into multiple small parts and downloads the parts. If you enable resumable download, ossutil saves key information such as the name, size, and download ID of the object to a checkpoint file whose name contains the .dcp extension in a specific directory named checkpoint dir. When the download task is resumed, ossutil reads the download ID from the checkpoint file, uses the download ID to query parts that are not downloaded, and downloads these parts.
Parameter description
|
Parameter |
Type |
Description |
|
--checkpoint-dir |
string |
The directory in which the checkpoint file is stored. |
|
--part-size |
SizeSuffix |
The part size. |
|
-u |
/ |
Specifies that ossutil downloads an object only when the last modified time of the object is later than the last modified time of the local file. |
Example
The following example describes how to run the cp command of ossutil to download the download.rar object from a bucket to the local_directory directory in your computer. During the download process, the system automatically creates a file whose name contains the .dcp extension to record the download progress information in the specified your-checkpoint-dir directory.
If the download task is interrupted due to unstable network connections or disk read and write exceptions, you do not need to restart a download task. You need to only run the cp command again. Then, ossutil automatically detects the generated .dcp file to resume the download task. This greatly improves the efficiency and reliability of the data transmission, especially in the poor network environment. To optimize the download experience, we recommend that you set the part-size parameter to a fixed value when you perform the download operation. In this example, --part-size 64m indicates that each part is 64 MB in size.
ossutil 2.0 supports resumable download for parts. The part size has little impact on the overall transmission effect. You can flexibly specify the part size based on your actual scenario.
ossutil cp oss://bucket/download.rar local_directory -u --checkpoint-dir D:\\your-checkpoint-dir\ --part-size 64m