cp (Download objects)

Updated at:

Use the cp command to download resources such as files, images, and videos from Object Storage Service (OSS) to your local computer. This command lets you download multiple objects, limit the download speed, or download specific versions of objects from a versioning-enabled bucket.

Usage notes

  • Starting from ossutil 1.6.16, you can use ossutil as the binary name in the command line, regardless of your operating system. If you are using a version earlier than 1.6.16, you must change the binary name to match your operating system. For more information, see ossutil command reference.

  • When you use this command with OSS on CloudBox:

    1. Replace the Endpoint in the configuration file with the CloudBox Endpoint. For more information, see CloudBox Endpoints.

    2. Add the --sign-version, --region, and --cloudbox-id options to the examples in this topic. For more information about these three options, see Common options.

Permissions

By default, an Alibaba Cloud account has full permissions on its resources. A Resource Access Management (RAM) user or RAM role does not have any permissions by default. You must grant permissions to a RAM user or RAM role using a RAM policy or a bucket policy.

API Action

Description

oss:GetObject

Required to download an object.

oss:GetObjectVersion

Optional. Required to download a specific version of a source object using the --version-id option.

kms:GenerateDataKey

Optional. Required if the downloaded object is encrypted on the server side using KMS.

kms:Decrypt

oss:ListObjects

Optional. Required for batch downloads (recursive operations).

Syntax

ossutil cp cloud_url file_url [options]

Parameter

Description

cloud_url

The source path of the OSS object. The format is oss://bucket[/prefix]. For example, oss://examplebucket/examplefile.txt.

file_url

The destination path for the local file. If the destination is a directory, the path must end with a separator (/ or \). For example, a Linux path is /localfolder/examplefile.txt and a Windows path is D:\localfolder\examplefile.txt.

-r, --recursive

Recursively applies the operation to all matching objects under the specified prefix. Otherwise, the operation applies only to the single specified object.

-f --force

Forces the operation without prompting for confirmation.

-u, --update

Performs the download only if the destination file does not exist or if the source object's last modified time is later than the destination file's.

--maxdownspeed

The maximum download speed in KB/s. The default value is 0, which means no speed limit.

--disable-ignore-error

Disables ignoring errors during batch operations.

--only-current-dir

Downloads only the objects in the current directory and ignores subdirectories and the objects within them.

--bigfile-threshold

The size threshold in bytes for a resumable download.

Default: 100 MB

Value range: 0 to 9223372036854775807

--part-size

The part size in bytes. By default, ossutil automatically calculates an appropriate part size based on the object size.

Value range: 1 to 9223372036854775807

--checkpoint-dir

The directory where ossutil stores checkpoint information for a resumable download. If a resumable download fails, ossutil creates a .ossutil_checkpoint directory to record checkpoint information. ossutil deletes this directory after the download succeeds. Ensure that the specified directory can be deleted.

--range

Downloads a specified byte range of the object and saves it as a new file. Byte numbering is zero-based.

  • Specify a range

    For example, 3-9 downloads bytes from the 3rd to the 9th byte, inclusive.

  • Specify a start position

    For example, 3- downloads from the 3rd byte to the end of the object, inclusive.

  • Specify an end position

    For example, -9 downloads from the 0th byte to the 9th byte, inclusive.

--encoding-type

The encoding type of the object name. Set this option to url. If this option is not specified, the object name is not encoded.

--include

Includes all objects that match the specified pattern.

For more information, see include and exclude options.

--exclude

Excludes all objects that match the specified pattern.

For more information, see include and exclude options.

--meta

Sets the metadata for the object. The format is header:value#header:value. Example: Cache-Control:no-cache#Content-Encoding:gzip. For more information about metadata, see set-meta (Manage object metadata).

--acl

The access control list (ACL) of the object. Valid values:

  • default (default): The object inherits the ACL of the bucket.

  • private: Only the bucket owner has read and write permissions. Other users cannot access the objects in the bucket.

  • public-read: Only the bucket owner has write permissions. All other users, including anonymous users, have read-only permissions on the objects in the bucket. This can lead to data leaks and increased fees. Use this ACL with caution.

  • public-read-write: Anyone, including anonymous users, can read and write objects in the bucket. This can lead to data leaks, increased fees, and potential legal liabilities if malicious content is uploaded. Except for specific use cases, do not configure public read-write permissions.

--snapshot-path

The directory where snapshot information is saved during the download. For the next download, ossutil reads the snapshot information from this directory to perform an incremental download.

--disable-crc64

Disables the 64-bit cyclic redundancy check (CRC-64). By default, ossutil enables CRC-64 for all data transfers.

--payer

The payment method for the request. To have the requester pay for traffic and requests generated from accessing the specified path, set this option to requester.

--partition-download

Specifies a partition of the object to download. The value is in the format Partition number:Total partitions. For example, 1:5 indicates that this ossutil command downloads partition 1 out of 5 total partitions. Partition numbers are 1-based. The tool's internal algorithm determines how ossutil partitions the object. This option lets you split an object into multiple partitions and download them in parallel using multiple ossutil commands. Each command downloads its assigned partition.

-j, --job

The number of concurrent tasks for multi-object operations. Default: 3. Value range: 1 to 10,000.

--parallel

The number of concurrent tasks for single-object operations. Value range: 1 to 10,000. If this option is not set, ossutil determines the value based on the operation type and object size.

--version-id

Downloads a specific version of an object. Use the --version-id option only with a versioning-enabled bucket. To enable versioning for a bucket, see bucket-versioning (Versioning).

--start-time

A UNIX timestamp. If you specify this option, objects with a last modified time earlier than this timestamp are ignored.

Note

Only ossutil 1.7.18 and later support this parameter. For more information about upgrading, see update (Upgrade ossutil).

--end-time

A UNIX timestamp. If you specify this option, objects with a last modified time later than this timestamp are ignored.

Note
  • If you specify both `start-time` and `end-time`, the copy command is executed only for files that were last modified between the specified start and end times.

  • Only ossutil 1.7.18 and later support this parameter. For more information about how to upgrade the version, see update (Upgrade ossutil).

For more information about other common options for this command, see Common options.

If the default concurrency does not meet your performance needs, you can adjust the -j, --jobs and --parallel options to tune performance. By default, ossutil calculates the number of parallels based on the object size. When you transfer large objects in batches, the actual concurrency is the number of jobs multiplied by the number of parallels.

  • If the ECS instance or server that runs the command has limited resources, such as network, memory, or CPU, we recommend that you set the concurrency to a value below 100. If these resources are not fully used, you can increase the concurrency.

  • Setting the concurrency too high can degrade performance or cause EOF errors because of thread switching overhead and resource competition. You can adjust the -j, --jobs and --parallel options based on your machine's resources. When you perform stress testing, start with low concurrency values and gradually increase them to find the optimal setting.

Examples

The following examples are for a Linux system. You must adjust the parameters to match your operating system and environment. The examples assume the following:

  • Bucket name: examplebucket

  • OSS directory: destfolder/

  • Local directory: localfolder/

  • Local file: examplefile.txt

Download a single object

  • Download an object to a specified directory and keep its original name:

    ossutil cp oss://examplebucket/destfolder/examplefile.txt localfolder/
  • Download an object to a specified directory and rename it to example.txt:

    ossutil cp oss://examplebucket/destfolder/examplefile.txt localfolder/example.txt

Batch download

  • Use the -r option to download the destfolder directory, including all its subdirectories and objects.

    ossutil cp -r oss://examplebucket/destfolder/ localfolder/
  • Use the --only-current-dir option to download only the objects in the current directory and ignore any subdirectories.

    ossutil cp oss://examplebucket/destfolder/ localfolder/ --only-current-dir -r
  • Use the -u option to download only objects that do not exist locally or have been updated in OSS. This option skips downloading an object if a local file with the same name and last modified time already exists.

    ossutil cp -r -u oss://examplebucket/destfolder/ localfolder/

Download conditional objects

  • Download all objects that are not in the JPG format from the destfolder directory:

    ossutil cp -r oss://examplebucket/destfolder/ localfolder/ --exclude "*.jpg"
  • Download objects from the destfolder directory that contain abc in their names but are not in JPG or TXT format:

    ossutil cp -r oss://examplebucket/destfolder/ localfolder/ --include "*abc*" --exclude "*.jpg" --exclude "*.txt"

Limit the download speed

Use the --maxdownspeed option to limit the download speed to 1 MB/s (1,024 KB/s).

ossutil cp -r oss://examplebucket/destfolder/ localfolder/ --maxdownspeed 1024

Range download

Use the --range option to download bytes 10 to 20 of examplefile.txt to your local computer.

ossutil cp oss://examplebucket/destfolder/examplefile.txt localfolder/ --range 10-20

Download objects within a specified time range

Use the --start-time and --end-time options to download only the objects in the destfolder directory that were last modified between 10:09:18 on October 31, 2023 (UTC+8) and 12:55:58 on October 31, 2023 (UTC+8).

ossutil cp -r oss://examplebucket/destfolder/ localfolder/ --start-time 1698718158 --end-time 1698728158

Download a specific version of an object

First, use the ls --all-versions command to obtain all version IDs of the object. Then, use the --version-id option to download a specific version.

Note

The --version-id option can be used only for objects in a versioning-enabled bucket. To enable versioning for a bucket, see bucket-versioning (Versioning).

ossutil cp oss://examplebucket/test.jpg localfolder/ --version-id CAEQARiBgID8rumR2hYiIGUyOTAyZGY2MzU5MjQ5ZjlhYzQzZjNlYTAyZDE3MDRk

Download objects and generate snapshot information

Use the --snapshot-path option to generate snapshot information for the downloaded objects in a specified directory. When you run a subsequent download command with this option, ossutil reads the snapshot information from the directory to perform an incremental download. For more information, see Upload objects and generate snapshots.

ossutil cp -r oss://examplebucket/destfolder/ localfolder/ --start-time 1698718158 --end-time 1698728158

Cross-account or cross-region download

Use the -e, -i, and -k common options to download an object from a bucket that is owned by another account and is located in the China (Shanghai) region.

Note

Specify the Endpoint that corresponds to the region where the bucket is located. For more information, see Regions and Endpoints.

ossutil cp oss://examplebucket/exampleobject.txt  localfolder/  -e oss-cn-shanghai.aliyuncs.com -i yourAccessKeyID -k yourAccessKeySecret