S3 and S3 Vectors APIs on OSS: supported operations and limitations

S3 API operations compatible with general-purpose buckets

The following table describes the bucket, object, and multipart API operations in S3 that are compatible with general-purpose buckets in OSS.

Differences between general-purpose buckets and S3

The following section describes the differences between general-purpose buckets in OSS and S3:

• Request style

  Amazon S3 supports two request addressing styles: virtual-hosted and path-style. Virtual-hosted style requests use the bucket name as a subdomain in the endpoint, while path-style requests place the bucket name in the URL path. For security reasons, OSS supports only the virtual-hosted style. Therefore, after being migrated from S3 to OSS, all client applications and S3-compatible tools must be configured to use virtual-hosted style requests. Failure to do so will result in access being denied.

• Definitions of access control lists (ACLs)

  The definition of ACL is different in OSS and S3. The following table describes the differences.

  Level	Permission	S3	OSS
  Bucket	READ	The permissions to list buckets.	If no object permissions are configured for an object in a bucket, you can perform only read operations on the object.
  	WRITE	The permissions to write or overwrite objects in a bucket.	If the object that you want to write does not exist in the specified bucket, the object is created in the bucket. If the object that you want to write exists in the specified bucket and no permissions are configured for the existing object, the existing object can be overwritten. Use InitiateMultipartUpload to upload objects.
  	READ_ACP	The permissions to read the ACL of a bucket.	Only the bucket owner and authorized RAM users have the permissions to read the ACL of a bucket.
  	WRITE_ACP	The permissions to configure the ACL of a bucket.	Only the bucket owner and authorized RAM users have the permissions to configure the ACL of a bucket.
  Object	READ	The permissions to read an object.	The permissions to read an object.
  	WRITE	N/A	The permissions to overwrite an object.
  	READ_ACP	The permissions to read the ACL of an object.	Only the bucket owner and authorized RAM users have the permissions to read the ACL of an object.
  	WRITE_ACP	The permissions to configure the ACL of an object.	Only the bucket owner and authorized RAM users have the permissions to configure the ACL of an object.

  Important

  OSS supports only the following ACL modes in S3: private, public-read, and public-read-write.

• Storage class

  The storage classes of OSS and S3 do not have a one-to-one correspondence (OSS provides 5 storage classes, whereas S3 provides 7 major storage classes), and they differ in capabilities such as minimum storage duration and access method. You can convert the storage class of OSS objects based on your requirements. The main mappings and differences are as follows:

  OSS storage class	Corresponding S3 storage class	Key differences
  Standard	STANDARD	Minimum storage duration: Neither OSS nor S3 has a requirement. Access method: Both support real-time access.
  Infrequent Access	STANDARD_IA ONEZONE_IA	Mapping: OSS zone-redundant IA corresponds to STANDARD_IA, and locally redundant IA corresponds to ONEZONE_IA. Minimum storage duration: 30 days for both. Access method: Both support real-time access and charge data retrieval fees.
  Archive	GLACIER_IR GLACIER (Flexible)	Minimum storage duration: 60 days for OSS and 90 days for S3. Access method: OSS requires a restore operation (about 1 minute), or you can enable real-time access to Archive objects; S3 GLACIER_IR provides millisecond-level real-time access, whereas GLACIER (Flexible) requires a Restore operation (Expedited: 1–5 minutes, Standard: 3–5 hours, Bulk: 5–12 hours).
  Cold Archive	DEEP_ARCHIVE	Minimum storage duration: 180 days for both. Access method: Both require a restore operation. OSS restore: high-priority about 1 hour, standard 2–5 hours, batch 5–12 hours; among S3 storage classes, only GLACIER (Flexible) reaches this restore duration, whereas DEEP_ARCHIVE requires Standard ≤12 hours (Batch 9–12 hours) and Bulk ≤48 hours.
  Deep Cold Archive	No corresponding storage class	A storage class unique to OSS for extremely cold data. AWS has no storage class colder than DEEP_ARCHIVE (no corresponding S3 storage class). Minimum storage duration: 180 days for OSS (no corresponding S3 storage class). Access method: OSS requires a restore operation, high-priority about 12 hours, standard about 48 hours (among S3 storage classes, only DEEP_ARCHIVE reaches this restore duration).

  Note

  In addition, S3 provides a dedicated INTELLIGENT_TIERING storage class that automatically moves objects between multiple access tiers based on access frequency. OSS does not provide a dedicated intelligent-tiering storage class; you can achieve similar automatic tiering by using lifecycle rules.

  If real-time access to Archive objects is not enabled for a bucket, you must first restore the Archive objects in the bucket before you can access them. OSS ignores the duration of the restored state configured for objects in S3 API operations. By default, the restored state lasts for one day and can be extended up to seven days. Then, the object enters the frozen state.

• ETag

  • If objects are uploaded by using the PUT method, the ETag of an OSS object and the ETag of an Amazon S3 object have different case sensitivities. The ETag of an OSS object is in uppercase letters, and the ETag of an Amazon S3 object is in lowercase letters. If your client uses ETag to validate content, configure your client to ignore the case sensitivity to prevent errors.

  • If objects are uploaded by using the multipart upload method, OSS calculates ETag values by using a different method from that of S3.

S3 Vectors API operations compatible with vector buckets

Vector buckets in OSS Vectors are compatible with the Amazon S3 Vectors protocol. You do not need to rewrite your application. Point the endpoint of the S3 Vectors client in the AWS SDK to the access domain name of your vector bucket, and you can manage vector buckets, vector indexes, and vector data, and run similarity queries. For more information about OSS Vectors, see OSS Vectors. For more information about the native OSS Vectors API operations, see OSS Vector operations.

To access a vector bucket by using the AWS SDK, configure the client as follows:

• Service name: s3vectors. Requests are signed by using AWS Signature Version 4. The SDK handles signing. No extra configuration is required.

• Public endpoint: https://$bucketname-$uid.$regionID.oss-vectors.aliyuncs.com

• Internal endpoint: https://$bucketname-$uid.$regionID-internal.oss-vectors.aliyuncs.com

• Region: the ID of the region in which the vector bucket resides, such as cn-hangzhou.

The following table describes the S3 Vectors API operations that vector buckets support and the corresponding OSS Vectors API operations.

The following S3 Vectors operations are not supported. A call to one of these operations returns the 405 MethodNotAllowed error: UpdateIndexMode, PutVectorBucketDefaultIndexMode, TagResource, UntagResource, and ListTagsForResource.

The following sample code shows how to use Python (Boto3) to create a vector index, write vector data, and query vectors over the S3 Vectors API. Before you run the code, install a Boto3 version that includes the S3 Vectors client (pip install -U boto3) and store your AccessKey pair in environment variables.

import os
import boto3

# The access domain name of the vector bucket, in the https://$bucketname-$uid.$regionID.oss-vectors.aliyuncs.com format.
client = boto3.client(
    "s3vectors",
    region_name="cn-hangzhou",
    endpoint_url="https://examplebucket-123***456.cn-hangzhou.oss-vectors.aliyuncs.com",
    aws_access_key_id=os.environ.get("OSS_ACCESS_KEY_ID"),
    aws_secret_access_key=os.environ.get("OSS_ACCESS_KEY_SECRET"),
)

# Create a vector index. The index name can contain only letters and digits and must start with a letter.
client.create_index(
    vectorBucketName="examplebucket",
    indexName="exampleindex",
    dataType="float32",
    dimension=4,
    distanceMetric="cosine",
)

# Write vector data.
client.put_vectors(
    vectorBucketName="examplebucket",
    indexName="exampleindex",
    vectors=[
        {
            "key": "key1",
            "data": {"float32": [0.1, 0.2, 0.3, 0.4]},
            "metadata": {"genre": "doc"},
        }
    ],
)

# Query vector data.
response = client.query_vectors(
    vectorBucketName="examplebucket",
    indexName="exampleindex",
    queryVector={"float32": [0.1, 0.2, 0.3, 0.4]},
    topK=3,
    returnDistance=True,
    returnMetadata=True,
)
print(response["vectors"])

Differences between vector buckets and S3 Vectors

When you switch from S3 Vectors to OSS Vectors, take note of the following differences:

• How resources are addressed

  S3 Vectors uses a fixed service endpoint and identifies the vector bucket by the vectorBucketName or vectorBucketArn parameter in the request body. OSS Vectors resolves the vector bucket from the third-level domain name and validates the request body against it. If the vectorBucketName or vectorBucketArn parameter in the request body does not match the vector bucket in the domain name, the 400 ValidationException error is returned.

• ARN format

  OSS Vectors uses the Alibaba Cloud resource name format. The ARN of a vector bucket is acs:ossvector:$regionID:$uid:$bucketname and the ARN of a vector index is acs:ossvector:$regionID:$uid:$bucketname/$indexname. The arn:aws:s3vectors:* format is not supported. ARN parameters are optional in all requests. We recommend that you address resources by name by using the vectorBucketName and indexName parameters.

• Naming conventions

  The name of a vector bucket must be 3 to 32 characters in length (3 to 63 characters in S3 Vectors) and can contain only lowercase letters, digits, and hyphens (-). It cannot start or end with a hyphen (-). The name of a vector index must be 1 to 63 characters in length, can contain only letters and digits, and must start with a letter. In S3 Vectors, an index name is 3 to 63 characters in length and can contain lowercase letters, digits, hyphens (-), and periods (.). Therefore, when you switch from S3 Vectors, you must rename vector indexes whose names contain hyphens (-), underscores (_), or periods (.).

• Server-side encryption

  OSS Vectors does not support server-side encryption. The encryptionConfiguration parameter in CreateVectorBucket and CreateIndex requests is parsed but ignored, and no error is returned. GetVectorBucket and GetIndex responses do not include this parameter.

• Resource tags

  OSS Vectors does not support resource tags. The tags parameter in CreateVectorBucket requests is parsed but ignored, and the TagResource, UntagResource, and ListTagsForResource operations return the 405 MethodNotAllowed error.

• Enhanced Mode (metadata pre-filtering) is not supported

  S3 Vectors added the ENHANCED index mode for metadata pre-filtering and renamed the original mode to CLASSIC. OSS Vectors does not support this mode. The related operations and parameters behave as follows:

  • The UpdateIndexMode and PutVectorBucketDefaultIndexMode operations are not supported and return the 405 MethodNotAllowed error.

  • QueryVectors does not support the queryMode parameter. This parameter is optional in S3 Vectors. If you set it to CLASSIC or ENHANCED, the 400 ValidationException error is returned with the message Unknown field: queryMode in request. Omit this parameter to run queries as usual.

  • The $startsWith filter operator, which is available only in ENHANCED mode, is not supported. It returns the 400 ValidationException error with the message Invalid filter operation: $startsWith.

  • GetIndex responses do not include indexMode, and GetVectorBucket responses do not include defaultIndexMode.

  If you need metadata pre-filtering or hybrid search, use Fusion Mode indexes in OSS Vectors. For more information, see OSS Vectors.

• Features with no S3 Vectors equivalent

  Access logging for vector buckets (PutVectorBucketLogging, GetVectorBucketLogging, and DeleteVectorBucketLogging) and Fusion Mode indexes with hybrid search (PutVectorIndexFusion and QueryVectorsFusion) have no equivalent operations in S3 Vectors. Call the native OSS Vectors API operations for these features. CreateIndex in the S3 Vectors API creates a Standard Mode index.

• Error responses

  An S3-compatible request returns error information in the JSON format and the exception type in the x-amzn-errortype response header. The following table describes common exception types.

  HTTP status code	Exception type	Common cause
  400	ValidationException	A request parameter is invalid. For example, the ARN format is invalid, the index name does not comply with the naming conventions, or the vector bucket in the request body does not match the one in the domain name.
  400	SerializationException	The request body does not conform to the specification or is not a JSON body, for example, the request body is not valid JSON. The error message is Failed to parse request.
  403	AccessDeniedException	The AccessKey ID does not exist, the signature verification failed, or you do not have the required permissions.
  404	NotFoundException	The specified vector bucket or vector index does not exist.
  405	MethodNotAllowed	The operation is not supported by OSS Vectors. For example, the operation is a resource tag operation.
  409	ConflictException	The name of the vector bucket or vector index already exists, or you attempted to delete a vector bucket that is not empty.