Accessing OSS with the AWS SDK

更新时间:
复制 MD 格式

To access OSS with an AWS SDK, configure the OSS endpoint and your access credentials. No other code changes are required.

  • Endpoint: Use an S3-compatible endpoint format. Replace {region} with the region ID, such as cn-hangzhou. For a complete list of regions, see Regions and endpoints.

    Type

    Format

    Public endpoint

    https://s3.oss-{region}.aliyuncs.com

    Internal endpoint

    https://s3.oss-{region}-internal.aliyuncs.com

    Transfer acceleration endpoint

    https://s3.oss-accelerate.aliyuncs.com

  • Access credentials: Create an AccessKey with OSS permissions in Resource Access Management (RAM).

Java

SDK 2.x

import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.S3Configuration;
import java.net.URI;

S3Client s3Client = S3Client.builder()
    .endpointOverride(URI.create("https://s3.oss-cn-hangzhou.aliyuncs.com"))
    .region(Region.AWS_GLOBAL)
    .serviceConfiguration(
        S3Configuration.builder()
            .pathStyleAccessEnabled(false)
            .chunkedEncodingEnabled(false)
            .build()
    )
    .build();

SDK 1.x

import com.amazonaws.client.builder.AwsClientBuilder.EndpointConfiguration;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;

AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
    .withEndpointConfiguration(new EndpointConfiguration(
        "https://s3.oss-cn-hangzhou.aliyuncs.com", 
        "cn-hangzhou"))
    .withPathStyleAccessEnabled(false)
    .withChunkedEncodingDisabled(false)
    .build();

For SDK 1.x, the S3ObjectInputStream returned by getObject immediately discards unread data when you call close(). You must read the stream completely before closing it.

S3Object object = s3Client.getObject("my-bucket", "file.txt");
InputStream input = object.getObjectContent();

byte[ ] data = IOUtils.toByteArray(input);

input.close();

Python

import boto3
from botocore.config import Config

s3 = boto3.client(
    's3',
    endpoint_url='https://s3.oss-cn-hangzhou.aliyuncs.com',
    config=Config(
        signature_version='s3',
        s3={'addressing_style': 'virtual'}
    )
)

Node.js

SDK v3

import { S3Client } from '@aws-sdk/client-s3';

const client = new S3Client({
    endpoint: 'https://s3.oss-cn-hangzhou.aliyuncs.com',
    region: 'cn-hangzhou'
});

SDK v2

const AWS = require('aws-sdk');

const s3 = new AWS.S3({
    endpoint: 'https://s3.oss-cn-hangzhou.aliyuncs.com',
    region: 'cn-hangzhou'
});

Go

SDK v2

import (
    "context"
    "github.com/aws/aws-sdk-go-v2/aws"
    awsconfig "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/s3"
)

cfg, _ := awsconfig.LoadDefaultConfig(context.TODO(),
    awsconfig.WithEndpointResolverWithOptions(
        aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
            return aws.Endpoint{
                URL: "https://s3.oss-cn-hangzhou.aliyuncs.com",
            }, nil
        }),
    ),
)
client := s3.NewFromConfig(cfg)

SDK v1

import (
    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/s3"
)

sess := session.Must(session.NewSessionWithOptions(session.Options{
    Config: aws.Config{
        Endpoint: aws.String("https://s3.oss-cn-hangzhou.aliyuncs.com"),
        Region:   aws.String("cn-hangzhou"),
    },
    SharedConfigState: session.SharedConfigEnable,
}))
svc := s3.New(sess)

.NET

SDK 3.x

using Amazon.S3;

var config = new AmazonS3Config
{
    ServiceURL = "https://s3.oss-cn-hangzhou.aliyuncs.com"
};
var client = new AmazonS3Client(config);

SDK 2.x

using Amazon.S3;

var config = new AmazonS3Config
{
    ServiceURL = "https://s3.oss-cn-hangzhou.aliyuncs.com"
};
var client = new AmazonS3Client(config);

PHP

SDK 3.x

<?php
require_once __DIR__ . '/vendor/autoload.php';
use Aws\S3\S3Client;

$s3Client = new S3Client([
    'version' => '2006-03-01',
    'region'  => 'cn-hangzhou',
    'endpoint' => 'https://s3.oss-cn-hangzhou.aliyuncs.com'
]);

SDK 2.x

<?php
require_once __DIR__ . '/vendor/autoload.php';
use Aws\S3\S3Client;

$s3Client = S3Client::factory([
    'version' => '2006-03-01',
    'region'  => 'cn-hangzhou',
    'base_url' => 'https://s3.oss-cn-hangzhou.aliyuncs.com'
]);

Ruby

SDK 3.x

require 'aws-sdk-s3'

s3 = Aws::S3::Client.new(
  endpoint: 'https://s3.oss-cn-hangzhou.aliyuncs.com',
  region: 'cn-hangzhou'
)

SDK 2.x

require 'aws-sdk'

s3 = AWS::S3::Client.new(
  s3_endpoint: 's3.oss-cn-hangzhou.aliyuncs.com',
  region: 'cn-hangzhou',
  s3_force_path_style: false
)

C++

Requires SDK version 1.7.68 or later.

#include <aws/s3/S3Client.h>
#include <aws/core/client/ClientConfiguration.h>

Aws::Client::ClientConfiguration config;
config.endpointOverride = "s3.oss-cn-hangzhou.aliyuncs.com";
config.region = "cn-hangzhou";

Aws::S3::S3Client s3_client(config);

Browser

Frontend web applications must use an STS temporary credential. Never hard-code a permanent AccessKey in client-side code. Your server calls AssumeRole to obtain a temporary credential and returns it to the client. For a complete tutorial, see Use an STS temporary credential to access OSS.

import { S3Client } from '@aws-sdk/client-s3';

// Fetch an STS temporary credential from your server.
async function getSTSCredentials() {
    const response = await fetch('https://your-server.com/api/sts-token');
    return await response.json();
}

// Initialize the S3 client with the temporary credential.
const client = new S3Client({
    region: 'cn-hangzhou',
    endpoint: 'https://s3.oss-cn-hangzhou.aliyuncs.com',
    credentials: async () => {
        const creds = await getSTSCredentials();
        return {
            accessKeyId: creds.accessKeyId,
            secretAccessKey: creds.secretAccessKey,
            sessionToken: creds.securityToken,
            expiration: new Date(creds.expiration)
        };
    }
});

Android

Android applications must use an STS temporary credential. Never hard-code a permanent AccessKey in the client application. Your server calls AssumeRole to obtain a temporary credential and returns it to the client. For a complete tutorial, see Use an STS temporary credential to access OSS.

import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.auth.BasicSessionCredentials;
import com.amazonaws.client.builder.AwsClientBuilder.EndpointConfiguration;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;

// Implement a credentials provider that fetches an STS temporary credential from your server.
public class OSSCredentialsProvider implements AWSCredentialsProvider {
    @Override
    public AWSCredentials getCredentials() {
        // Fetch an STS temporary credential from your server,
        // for example, by sending a request to https://your-server.com/api/sts-token.
        String accessKeyId = fetchFromServer("accessKeyId");
        String secretKeyId = fetchFromServer("secretKeyId");
        String securityToken = fetchFromServer("securityToken");
        
        return new BasicSessionCredentials(accessKeyId, secretKeyId, securityToken);
    }
    
    @Override
    public void refresh() {
        // Refresh the credential.
    }
}

// Create the S3 client.
AmazonS3 s3Client = AmazonS3Client.builder()
    .withCredentials(new OSSCredentialsProvider())
    .withEndpointConfiguration(new EndpointConfiguration(
        "https://s3.oss-cn-hangzhou.aliyuncs.com", ""))
    .build();

// Application code
s3Client.putObject("my-bucket", "test.txt", "Hello OSS");

iOS

iOS applications must use an STS temporary credential. Never hard-code a permanent AccessKey in the client application. Your server calls AssumeRole to obtain a temporary credential and returns it to the client. For a complete tutorial, see Use an STS temporary credential to access OSS.

#import <AWSS3/AWSS3.h>

// Implement the credentials provider.
@interface OSSCredentialsProvider : NSObject <AWSCredentialsProvider>
@end

@implementation OSSCredentialsProvider

- (AWSTask<AWSCredentials *> *)credentials {
    return [[AWSTask taskWithResult:nil] continueWithBlock:^id(AWSTask *task) {
        // Fetch an STS temporary credential from your server.
        NSString *accessKey = [self fetchFromServer:@"accessKeyId"];
        NSString *secretKey = [self fetchFromServer:@"secretKeyId"];
        NSString *sessionToken = [self fetchFromServer:@"securityToken"];
        
        AWSCredentials *credentials = [[AWSCredentials alloc]
            initWithAccessKey:accessKey
            secretKey:secretKey
            sessionKey:sessionToken
            expiration:[NSDate dateWithTimeIntervalSinceNow:3600]];
        
        return [AWSTask taskWithResult:credentials];
    }];
}

@end

// Configure the S3 client.
AWSEndpoint *endpoint = [[AWSEndpoint alloc] initWithURLString:@"https://s3.oss-cn-hangzhou.aliyuncs.com"];
AWSServiceConfiguration *configuration = [[AWSServiceConfiguration alloc]
    initWithRegion:AWSRegionUnknown
    endpoint:endpoint
    credentialsProvider:[[OSSCredentialsProvider alloc] init]];

[AWSS3 registerS3WithConfiguration:configuration forKey:@"OSS"];
AWSS3 *s3 = [AWSS3 S3ForKey:@"OSS"];

// Application code
AWSS3PutObjectRequest *request = [AWSS3PutObjectRequest new];
request.bucket = @"my-bucket";
request.key = @"test.txt";
request.body = [@"Hello OSS" dataUsingEncoding:NSUTF8StringEncoding];

[[s3 putObject:request] continueWithBlock:^id(AWSTask *task) {
    if (task.error) {
        NSLog(@"Error: %@", task.error);
    } else {
        NSLog(@"Success");
    }
    return nil;
}];

FAQ

Upload failure: InvalidArgument: aws-chunked encoding is not supported

Symptom: When you upload a file, you receive the following error:

InvalidArgument: aws-chunked encoding is not supported with the specified x-amz-content-sha256 value

Root cause:

This is the most common issue when using an AWS SDK to access OSS. OSS supports the AWS Signature V4 algorithm but differs in transfer encoding:

  • AWS S3: Uses chunked encoding by default to transfer large files.

  • OSS: Does not support chunked encoding for transfers.

Cause analysis:

Some SDKs bind their Signature V4 implementation to chunked encoding:

  • Python (boto3): The Signature V4 implementation forces the use of chunked encoding and cannot be disabled. You must use Signature V2.

  • Java: You can disable chunked encoding through configuration.

  • Go/Node.js: Chunked encoding is not used by default, so no special handling is required.

Solutions (by SDK):

SDK

Solution

Reason

Python (boto3)

Use Signature V2: signature_version='s3'

The Signature V4 implementation in boto3 is bound to chunked encoding and cannot be disabled.

Java 1.x

Signature V4 + .withChunkedEncodingDisabled(true)

Chunked encoding can be disabled.

Java 2.x

Signature V4 + .chunkedEncodingEnabled(false)

Chunked encoding can be disabled.

Go v1

Signature V4

Does not use chunked encoding by default.

Go v2

Signature V4; however, the Manager API may use chunked encoding for large file uploads.

The Manager feature may use chunked encoding.

Node.js v3

Signature V4

Does not use chunked encoding by default.

Python example (before and after):

# Incorrect configuration (boto3 Signature V4 implementation uses chunked encoding)
s3 = boto3.client('s3',
    endpoint_url='https://oss-cn-hongkong.aliyuncs.com',
    config=Config(signature_version='v4'))

# Correct configuration (boto3 uses Signature V2)
s3 = boto3.client('s3',
    endpoint_url='https://oss-cn-hongkong.aliyuncs.com',
    config=Config(signature_version='s3'))  # Signature V2 is the stable solution for boto3.

Technical details:

OSS Signature V4 follows the AWS Signature Version 4 specification but with the following requirements:

  • The request header must include x-oss-content-sha256: UNSIGNED-PAYLOAD.

  • The Transfer-Encoding: chunked method must not be used.

Most SDKs can be configured for compatibility. However, because the Signature V4 implementation in boto3 is tightly coupled with chunked encoding, you must use Signature V2 with boto3.

SDK and signature version selection

Version selection guide:

Language

SDK version

Signature version

Key considerations

Python

Latest boto3

V2 (s3)

The boto3 V4 implementation is incompatible with OSS.

Java 1.x

Latest 1.x version

V4

Chunked encoding must be disabled.

Java 2.x

Latest 2.x version

V4

Chunked encoding must be disabled.

Node.js

v3

V4 (default)

-

Go v1

Latest v1 version

V4 (default)

-

Go v2

Latest v2 version

V4 (default)

The Manager API may use chunked encoding for large file uploads.

Signature version details:

  • OSS Signature V4: OSS fully supports the AWS Signature V4 algorithm.

  • Signature V2: This is a special case for boto3, which is required due to SDK implementation limitations.

  • Compatibility: Except for boto3, all other SDKs can use Signature V4 to access OSS.

Version selection guide for new projects:

Scenario

Recommended solution

Reason

New Python project

boto3 + Signature V2

boto3 does not support Signature V4 for OSS.

New Java project

Java 2.x + Signature V4

Better performance.

New Node.js project

v3 + Signature V4

-

New Go project

Go v1 + Signature V4

Recommended

Existing project migration

Keep the current SDK version.

Minimizes the risk of breaking changes.

Signature error: SignatureDoesNotMatch

You might encounter a SignatureDoesNotMatch error, which indicates that the signature calculated by the server does not match the client's signature.

The most common cause is using an AWS AccessKey instead of an OSS AccessKey. AWS access credentials and OSS access credentials are separate systems and cannot be used interchangeably. Check parameters such as aws_access_key_id and aws_secret_access_key in your code to ensure you are using the AccessKey ID and AccessKey Secret created in the OSS console.

The second most common cause is clock skew. The S3 signature algorithm includes a timestamp in the signature. OSS rejects requests if the timestamp differs from the server's time by more than 15 minutes. You can run the date -u command to check the server's UTC time. If the time is inaccurate, use ntpdate or your system's time synchronization service to correct it.

A third cause is an incorrect endpoint configuration. If the endpoint still points to an AWS domain, such as s3.amazonaws.com, or uses the wrong OSS region, the signature calculation fails. The standard format for an OSS endpoint is https://oss-{region}.aliyuncs.com, where {region} must match the bucket's region, such as oss-cn-hangzhou or oss-cn-beijing.

When using boto3, there is another specific cause: if signature_version='s3' is not configured, boto3 defaults to Signature V4, causing the signature to fail. A correct boto3 configuration includes the Config(signature_version='s3') parameter.

A simple way to verify your configuration is to use the ossutil command-line tool. Run ossutil ls oss://your-bucket --access-key-id <key> --access-key-secret <secret> --endpoint oss-cn-hangzhou.aliyuncs.com. If this command successfully lists the bucket's contents, your access credentials and endpoint are correct, indicating the problem is in your code configuration.

Bucket access errors

A NoSuchBucket or AccessDenied error indicates that the specified bucket cannot be accessed. The most common cause is a mismatch between the endpoint and the bucket's region.

Each OSS bucket belongs to a specific region, such as cn-hangzhou or cn-beijing. When you access a bucket, the endpoint must match the region where the bucket is located. For example, if your bucket is in the Hangzhou region, the endpoint must be oss-cn-hangzhou.aliyuncs.com. You cannot use the endpoint for the Beijing region, oss-cn-beijing.aliyuncs.com. Unlike AWS S3, OSS does not support cross-region access or automatic redirects. If you use the wrong endpoint, OSS returns a NoSuchBucket error.

A second cause is incorrect RAM permissions. Ensure the RAM user associated with your OSS AccessKey has permission to access the target bucket. In the RAM console, confirm that the user has been granted the necessary permissions, such as oss:ListObjects, oss:GetObject, and oss:PutObject.

A third cause is bucket naming conventions. OSS supports two URL styles: virtual-hosted style (bucket-name.oss-cn-hangzhou.aliyuncs.com) and path-style (oss-cn-hangzhou.aliyuncs.com/bucket-name). When using the virtual-hosted style, the bucket name must comply with DNS naming conventions and cannot contain underscores. If your bucket name contains an underscore, you must configure your SDK to use path-style access or create a new bucket with a compliant name.

Performance optimization

Uploading and downloading large files are common tasks in object storage. AWS SDKs provide several transfer acceleration features that also work with OSS.

When using Python boto3, you can configure multipart upload parameters with TransferConfig. If a file is larger than the configured threshold, boto3 automatically splits the file into parts and uploads them in parallel, which can significantly improve throughput. The multipart_threshold parameter controls the file size threshold for enabling multipart upload, max_concurrency controls the number of concurrent upload threads, and multipart_chunksize controls the size of each part. Properly configuring these parameters can increase upload speeds for files larger than 100 MB severalfold.

When using the Java SDK, the TransferManager class encapsulates features such as multipart upload, concurrent transfers, and automatic retries. The TransferManager automatically selects the optimal transfer strategy based on the file size, so you do not have to manage part logic manually.

When using the Go SDK, use s3manager.Uploader instead of PutObject directly. The Uploader provides concurrent multipart uploads, automatically splitting large files and retrying failed uploads.

When using the Node.js SDK, you can use the Upload class from the @aws-sdk/lib-storage package. This class supports streaming uploads, which allows uploading to start while the file is being read, reducing memory usage.

All these transfer acceleration features are based on the S3 Multipart Upload API, which OSS fully supports. You can therefore use them directly with OSS.