How to perform resumable download using OSS SDK for Java-Object Storage Service(OSS)-阿里云帮助中心

You may fail to download a large object if the network is unstable or other exceptions occur. In some cases, you may fail to download the object even after multiple attempts. To resolve this issue, Object Storage Service (OSS) provides the resumable download feature. In resumable download, OSS splits an object into multiple parts and downloads each part separately. After all parts are downloaded, OSS combines the parts into a complete object.

Usage notes

In this topic, the public endpoint of the China (Hangzhou) region is used. To access OSS from other Alibaba Cloud services in the same region, use an internal endpoint. For details about supported regions and endpoints, see Regions and endpoints.
In this topic, access credentials are obtained from environment variables. For more information, see Configure access credentials.
This topic demonstrates creating an OSSClient instance with an OSS endpoint. For alternative configurations, such as using a custom domain or authenticating with credentials from Security Token Service (STS), see Client configuration.
To use resumable download, you must have the oss:GetObject permission. For more information, see Grant a custom policy.

Performance tuning

The speed of a resumable download depends on the part size, the number of concurrent tasks, and the connection pool size of the OSS client. Configure the following parameters based on the size of the object and the resources of the client:

Part size — Set PartSize to a value between 1 MB and 10 MB. The larger the object, the larger the part size that you can specify. Increasing the part size from 1 MB to 10 MB improves the download speed by approximately 80%, which makes the part size the most effective parameter to tune.
Number of concurrent tasks — Set TaskNum to a value between 10 and 50. If you specify a value greater than 50, the download speed is limited by the network bandwidth, CPU, and memory resources of the client, and by the internal network bandwidth quota of the VPC. In this case, a larger value provides only limited improvement.
Connection pool size — Call the setMaxConnectionsCount method of ClientBuilderConfiguration to set the connection pool size to a value greater than or equal to the number of concurrent tasks. Otherwise, concurrent tasks wait for available connections.

Examples

The following sample code provides an example on how to perform resumable download:

import com.aliyun.oss.ClientBuilderConfiguration;
import com.aliyun.oss.OSS;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.OSSClientBuilder;
import com.aliyun.oss.OSSException;
import com.aliyun.oss.common.comm.SignVersion;
import com.aliyun.oss.model.*;

public class Demo {
    public static void main(String[] args) throws Exception {
        // In this example, the endpoint of the China (Hangzhou) region is used. Specify your actual endpoint. 
        String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
        // Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
        EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
        // Specify the name of the bucket. Example: examplebucket. 
        String bucketName = "examplebucket";
        // Specify the full path of the object. Example: exampledir/exampleobject.txt. Do not include the bucket name in the full path. 
        String objectName = "exampledir/exampleobject.txt";
        // Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou.
        String region = "cn-hangzhou";

        // Create an OSSClient instance. 
        // Call the shutdown method to release associated resources when the OSSClient is no longer in use.
        ClientBuilderConfiguration clientBuilderConfiguration = new ClientBuilderConfiguration();
        clientBuilderConfiguration.setSignatureVersion(SignVersion.V4);        
        OSS ossClient = OSSClientBuilder.create()
        .endpoint(endpoint)
        .credentialsProvider(credentialsProvider)
        .clientConfiguration(clientBuilderConfiguration)
        .region(region)               
        .build();
        
      	try {
            // Perform resumable download in which 10 parts can be concurrently downloaded. 
            DownloadFileRequest downloadFileRequest = new DownloadFileRequest(bucketName, objectName);
            // Specify the full path to which you want to download the object. Example: D:\\localpath\\examplefile.txt. 
            downloadFileRequest.setDownloadFile("D:\\localpath\\examplefile.txt");
            // Specify the part size. Unit: bytes. Valid values: 100 KB to 5 GB. The default part size is 100 KB. 
            downloadFileRequest.setPartSize(1 * 1024 * 1024);
            // Specify the number of concurrent threads for the resumable download task. Default value: 1. 
            downloadFileRequest.setTaskNum(10);
            // Specify whether to enable resumable download. By default, resumable download is disabled. 
            downloadFileRequest.setEnableCheckpoint(true);
            // Specify the full path of the checkpoint file. Example: D:\\localpath\\examplefile.txt.dcp. 
            // The checkpoint file is generated when the download is interrupted. If you want to resume the download task, you must specify the full path of the checkpoint file. After the object is downloaded, the checkpoint file is deleted. 
            //downloadFileRequest.setCheckpointFile("D:\\localpath\\examplefile.txt.dcp");

            // Download the object. 
            DownloadFileResult downloadRes = ossClient.downloadFile(downloadFileRequest);
            // After the object is downloaded, the object metadata is returned. 
            ObjectMetadata objectMetadata = downloadRes.getObjectMetadata();
            System.out.println(objectMetadata.getETag());
            System.out.println(objectMetadata.getLastModified());
            System.out.println(objectMetadata.getUserMetadata().get("meta"));
        } catch (OSSException oe) {
            System.out.println("Caught an OSSException, which means your request made it to OSS, "
                    + "but was rejected with an error response for some reason.");
            System.out.println("Error Message:" + oe.getErrorMessage());
            System.out.println("Error Code:" + oe.getErrorCode());
            System.out.println("Request ID:" + oe.getRequestId());
            System.out.println("Host ID:" + oe.getHostId());
        } catch (Throwable ce) {
            System.out.println("Caught an ClientException, which means the client encountered "
                    + "a serious internal problem while trying to communicate with OSS, "
                    + "such as not being able to access the network.");
            System.out.println("Error Message:" + ce.getMessage());
        } finally {
            if (ossClient != null) {
                ossClient.shutdown();
            }
        }
    }
}

References

For the complete sample code that is used to perform resumable download, visit GitHub.
For more information about the API operation that you can call to perform resumable download, see GetObject.