使用场景
大文件加速上传
当文件大小超过5 GB时,使用分片上传可实现并行上传多个Part以加快上传速度。
网络环境较差
网络环境较差时,建议使用分片上传。当出现上传失败的时候,您仅需重传失败的Part。
文件大小不确定
可以在需要上传的文件大小还不确定的情况下开始上传,这种场景在视频监控等行业应用中比较常见。
使用限制
限制项 | 规格 |
文件大小 | 不超过48.8 TB |
Part数量 | 1~10, 000个 |
单个Part大小 | 最小值为100 KB,最大值为5 GB。最后一个Part的大小不要求大于100 KB。 |
单次ListParts请求返回的Part最大数量 | 1, 000个 |
单次ListMultipartUploads请求返回的Multipart Upload事件最大数量 | 1, 000个 |
注意事项
文件上传性能调优
如果您在上传大量文件时,在命名上使用了顺序前缀(如时间戳或字母顺序),可能会出现大量文件索引集中存储于存储空间中某个特定分区的情况。此时如果您的请求速率过大,会导致请求速率下降。建议您在上传大量文件时,不要使用顺序前缀的文件名。更多信息,请参见OSS性能与扩展性最佳实践。
文件覆盖
上传同名文件会覆盖OSS中已有文件。您可以通过以下方式防止文件被意外覆盖:
删除Part
分片上传过程被中断后,已上传的Part会一直保存在Bucket中。如果您不再需要这些Part,请通过生命周期规则自动删除。具体操作,请参见生命周期规则。
操作步骤
使用阿里云SDK
仅支持通过Java SDK进行分片上传,Java SDK要求3.15.0及以上版本。
import com.aliyun.oss.ClientException;
import com.aliyun.oss.OSS;
import com.aliyun.oss.OSSClientBuilder;
import com.aliyun.oss.OSSException;
import com.aliyun.oss.model.*;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import com.aliyun.oss.common.auth.DefaultCredentialProvider;
import com.aliyun.oss.common.comm.SignVersion;
import com.aliyun.oss.ClientBuilderConfiguration;
import com.aliyun.oss.common.auth.CredentialsProviderFactory;
import com.aliyun.oss.common.auth.EnvironmentVariableCredentialsProvider;
public class Demo {
public static void main(String[] args) throws Exception {
String endpoint = "https://cb-f8z7yvzgwfkl9q0h****.cn-hangzhou.oss-cloudbox.aliyuncs.com";
EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
String bucketName = "examplebucket";
String region = "cn-hangzhou";
String cloudBoxId = "cb-f8z7yvzgwfkl9q0h****";
String objectName = "exampledir/exampleobject.txt";
ClientBuilderConfiguration conf = new ClientBuilderConfiguration();
conf.setSignatureVersion(SignVersion.V4);
OSS ossClient = OSSClientBuilder.create()
.endpoint(endpoint)
.credentialsProvider(new DefaultCredentialProvider(credentialsProvider.getCredentials()))
.clientConfiguration(conf)
.region(region)
.cloudBoxId(cloudBoxId)
.build();
try {
InitiateMultipartUploadRequest request = new InitiateMultipartUploadRequest(bucketName, objectName);
InitiateMultipartUploadResult upresult = ossClient.initiateMultipartUpload(request);
String uploadId = upresult.getUploadId();
List<PartETag> partETags = new ArrayList<PartETag>();
final long partSize = 1 * 1024 * 1024L;
final File sampleFile = new File("D:\\localpath\\examplefile.txt");
long fileLength = sampleFile.length();
int partCount = (int) (fileLength / partSize);
if (fileLength % partSize != 0) {
partCount++;
}
for (int i = 0; i < partCount; i++) {
long startPos = i * partSize;
long curPartSize = (i + 1 == partCount) ? (fileLength - startPos) : partSize;
InputStream instream = new FileInputStream(sampleFile);
instream.skip(startPos);
UploadPartRequest uploadPartRequest = new UploadPartRequest();
uploadPartRequest.setBucketName(bucketName);
uploadPartRequest.setKey(objectName);
uploadPartRequest.setUploadId(uploadId);
uploadPartRequest.setInputStream(instream);
uploadPartRequest.setPartSize(curPartSize);
uploadPartRequest.setPartNumber( i + 1);
UploadPartResult uploadPartResult = ossClient.uploadPart(uploadPartRequest);
partETags.add(uploadPartResult.getPartETag());
}
CompleteMultipartUploadRequest completeMultipartUploadRequest =
new CompleteMultipartUploadRequest(bucketName, objectName, uploadId, partETags);
CompleteMultipartUploadResult completeMultipartUploadResult = ossClient.completeMultipartUpload(completeMultipartUploadRequest);
System.out.println(completeMultipartUploadResult.getETag());
} catch (OSSException oe) {
System.out.println("Caught an OSSException, which means your request made it to OSS, "
+ "but was rejected with an error response for some reason.");
System.out.println("Error Message:" + oe.getErrorMessage());
System.out.println("Error Code:" + oe.getErrorCode());
System.out.println("Request ID:" + oe.getRequestId());
System.out.println("Host ID:" + oe.getHostId());
} catch (ClientException ce) {
System.out.println("Caught an ClientException, which means the client encountered "
+ "a serious internal problem while trying to communicate with OSS, "
+ "such as not being able to access the network.");
System.out.println("Error Message:" + ce.getMessage());
} finally {
if (ossClient != null) {
ossClient.shutdown();
}
}
}
}
使用命令行工具ossutil
关于使用ossutil分片上传具体操作, 请参见cp(上传文件)。