Use ossutil to download and transfer authorized datasets

Updated at:
Copy as MD

After you subscribe to a dataset, you can query the authorized dataset version to obtain an OSS access point alias. You can use this alias to access the authorized data. This alias can be used in the same way as an OSS bucket name. This topic describes how to use the ossutil command line interface (CLI) to query and download data.

View dataset version information

The page displays the information required to transfer the data.

  • OSS access point alias: Equivalent to an OSS bucket name. This alias is used to access data.

  • region: The region where the authorized data is located.

  • path: The file path shared by the data provider.

  • pay by requester: Specifies whether the requester pays for data traffic and request fees.

Download and install ossutil

For information about how to download and install ossutil, see Install ossutil.

Configure ossutil

Use the `ossutil config` command to configure ossutil.

  • accessKeyID: Enter the AccessKey ID of the Alibaba Cloud account or the RAM user that was used to view the dataset version information.

  • accessKeySecret: Enter the AccessKey secret of the Alibaba Cloud account or the RAM user that was used to view the dataset version information.

  • endpoint: Enter the endpoint for the corresponding region. For example: https://oss-cn-hangzhou.aliyuncs.com

Transfer data to a destination bucket

The ossutil `cp` command does not support copying data directly to a destination bucket using an access point. You must first download the data and then upload it.

Download data to a local directory

ossutil cp -r oss://ap-a8j94b41sgvd61p-493a10b54d73b97e8ff2821ea5a0817f1a-ossalias/data-api-oss/ tempa/

An access point alias is equivalent to a bucket name. Use the access point alias and path to specify the files to download. The `-r` option recursively downloads all files in the folder. For more information about the command, see cp (Download files).

If the provider has enabled pay-by-requester, you must add the `--payer=requester` parameter when you use ossutil to transfer the data. If pay-by-requester is enabled, the authorized user is charged for the traffic and request fees that are generated by the data download. For more information, see Traffic fees.

Upload data from a local directory to a cross-region bucket

ossutil cp -e oss-cn-beijing.aliyuncs.com -r tempa/ oss://cri-3yyq14vsfaqsoaow-chart/data-api-oss/ 

For a cross-region transfer, you can use the `-e` option to specify the endpoint of the bucket. The `-r` option recursively uploads all files in the folder. For more information about the command, see cp (Upload files).

Notes

  1. For security reasons and due to access point authorization initialization, only the Alibaba Cloud account and RAM users that have viewed the dataset version information by calling the `GetSubscribedDataSetVersion` operation can use ossutil to transfer data.

  2. In addition to using ossutil for data transfers, you can also use an access point alias to access and download data using the OSS OpenAPI.

  3. Currently, temporary users who assume a role using Security Token Service (STS) cannot use access points.