Log shipping

更新时间:
复制 MD 格式

Enable logging to generate hourly access log files and store them in a specified bucket. Analyze these logs with Log Service or a Spark cluster.

Usage notes

  • If the source bucket has a region attribute, the target bucket must belong to the same account and region. It can be the same as or different from the source bucket.

    When the source and target buckets are the same, the log push operation generates additional logs, which creates a loop. Use different source and target buckets to avoid this.

  • For a source bucket without a region attribute, the target bucket must be the same as the source bucket.

  • Log files can take up to 48 hours to generate. Requests may appear in adjacent time period files, so logs for a specific period are not guaranteed to be complete or timely.

  • OSS generates a log file every hour until you disable logging. Delete unneeded log files to reduce storage costs.

    Use Lifecycle rules based on the last modified time to periodically delete log files.

  • To prevent service disruption or data contamination when configuring log shipping for a bucket with OSS-HDFS enabled, do not set the Log Prefix to.dlsdata/.

  • OSS may add new fields to logs. Design your log processing tools to handle additions. For example, the Bucket ARN field will be added to logs starting September 17, 2025.

  • Avoid delivering logs to a bucket with ObjectWorm enabled. ObjectWorm prevents log deletion during the retention period, causing storage costs to increase steadily.

Configure logging

Configure logging for a bucket

Console

  1. Log on to the OSS console.

  2. In the left-side navigation pane, click Buckets. On the Buckets page, find and click the desired bucket.

  3. In the left-side navigation pane, choose Logging > Logging.

  4. On the Logging page, turn on Logging, and then configure the following parameters.

    Parameter

    Description

    Target bucket

    The bucket that stores logs. Default: the current bucket. Select a different bucket from the drop-down list if needed. The target bucket must belong to the same account and region.

    Log prefix

    The directory in the target bucket for log files. If unspecified, logs go to the root directory. Example: setting the prefix to log/ stores logs in the log/ directory.

    RAM role

    The role that OSS assumes to write logs.

    • Default service role (Recommended)

      Automatically creates the AliyunOSSLoggingDefaultRole role with required permissions for all buckets in your account. First-time users: click Authorize Now to create and authorize the role.

    • Custom role

      To limit logging permissions to a specific bucket, create a custom role:

      1. Create a RAM role for a trusted Alibaba Cloud service. When you create the role, set Trusted Service to Cloud Service and Select Trusted Service to OSS.

      2. Create a custom policy by using the script editor.

        KMS encryption
        {
          "Version": "1",
          "Statement": [
            {
              "Effect": "Allow",
              "Action": [
                "kms:List*",
                "kms:Describe*",
                "kms:GenerateDataKey",
                "kms:Decrypt"
              ],
              "Resource": "*"
            },
            {
              "Effect": "Allow",
              "Action": [
                "oss:PutObject",
                "oss:AbortMultipartUpload"
              ],
              "Resource": "acs:oss:*:*:examplebucket/*"
            }
          ]
        }

        Without KMS encryption

        Use this policy for buckets without encryption or with OSS-managed server-side encryption.

        {
          "Version": "1",
          "Statement": [  
            {
              "Effect": "Allow",
              "Action": [
                "oss:PutObject",
                "oss:AbortMultipartUpload"
              ],
              "Resource": "acs:oss:*:*:examplebucket/*"
            }
          ]
        }
      3. Grant permissions to the RAM role.

  5. Click Save.

Ossutil

Enable logging with the ossutil CLI. Install ossutil.

The following command enables logging for examplebucket, storing access logs in destBucket with the prefix MyLog-.

ossutil api put-bucket-logging --bucket examplebucket --bucket-logging-status "{\"LoggingEnabled\":{\"TargetBucket\":\"destBucket\",\"TargetPrefix\":\"MyLog-\"}}"

put-bucket-logging.

SDK

Configure logging with the following SDK examples. Other SDKs are listed in SDKs.

import com.aliyun.oss.*;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.common.comm.SignVersion;
import com.aliyun.oss.model.SetBucketLoggingRequest;

public class Demo {

    public static void main(String[] args) throws Exception {
        // In this example, the endpoint of the China (Hangzhou) region is used. Specify your actual endpoint. 
        String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
        // Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
        EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
        // Specify the name of the source bucket for which you want to enable logging. Example: examplebucket. 
        String bucketName = "examplebucket";
        // Specify the name of the destination bucket in which you want to store the log objects. The source bucket and the destination bucket can be the same bucket or different buckets. 
        String targetBucketName = "yourTargetBucketName";
        // Set the directory in which you want to store the log objects to log/. If you specify this parameter, the log objects are stored in the specified directory of the destination bucket. If you do not specify this parameter, the log objects are stored in the root directory of the destination bucket. 
        String targetPrefix = "log/";
        // Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou.
        String region = "cn-hangzhou";

        // Create an OSSClient instance.
        // Call the shutdown method to release resources when the OSSClient is no longer in use. 
        ClientBuilderConfiguration clientBuilderConfiguration = new ClientBuilderConfiguration();
        clientBuilderConfiguration.setSignatureVersion(SignVersion.V4);        
        OSS ossClient = OSSClientBuilder.create()
        .endpoint(endpoint)
        .credentialsProvider(credentialsProvider)
        .clientConfiguration(clientBuilderConfiguration)
        .region(region)               
        .build();

        try {
            SetBucketLoggingRequest request = new SetBucketLoggingRequest(bucketName);
            request.setTargetBucket(targetBucketName);
            request.setTargetPrefix(targetPrefix);
            ossClient.setBucketLogging(request);
        } catch (OSSException oe) {
            System.out.println("Caught an OSSException, which means your request made it to OSS, "
                    + "but was rejected with an error response for some reason.");
            System.out.println("Error Message:" + oe.getErrorMessage());
            System.out.println("Error Code:" + oe.getErrorCode());
            System.out.println("Request ID:" + oe.getRequestId());
            System.out.println("Host ID:" + oe.getHostId());
        } catch (ClientException ce) {
            System.out.println("Caught an ClientException, which means the client encountered "
                    + "a serious internal problem while trying to communicate with OSS, "
                    + "such as not being able to access the network.");
            System.out.println("Error Message:" + ce.getMessage());
        } finally {
            if (ossClient != null) {
                ossClient.shutdown();
            }
        }
    }
}
<?php
if (is_file(__DIR__ . '/../autoload.php')) {
    require_once __DIR__ . '/../autoload.php';
}
if (is_file(__DIR__ . '/../vendor/autoload.php')) {
    require_once __DIR__ . '/../vendor/autoload.php';
}

use OSS\Credentials\EnvironmentVariableCredentialsProvider;
use OSS\OssClient;
use OSS\CoreOssException;

// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.  
$provider = new EnvironmentVariableCredentialsProvider();
// Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
$endpoint = "yourEndpoint";
// Specify the name of the source bucket for which you want to enable logging. Example: examplebucket. 
$bucket= "examplebucket";

$option = array();
// Specify the name of the destination bucket in which the log objects are stored. 
$targetBucket = "destbucket";
// Specify the directory in which the log objects are stored. If you specify this parameter, the log objects are stored in the specified directory of the destination bucket. If you do not specify this parameter, the log objects are stored in the root directory of the destination bucket. 
$targetPrefix = "log/";

try {
    $config = array(
        "provider" => $provider,
        "endpoint" => $endpoint,
        "signatureVersion" => OssClient::OSS_SIGNATURE_VERSION_V4,
        "region"=> "cn-hangzhou"
    );
    $ossClient = new OssClient($config);

    // Enable logging for the source bucket. 
    $ossClient->putBucketLogging($bucket, $targetBucket, $targetPrefix, $option);
} catch (OssException $e) {
    printf(__FUNCTION__ . ": FAILED\n");
    printf($e->getMessage() . "\n");
    return;
}
print(__FUNCTION__ . ": OK" . "\n");            
const OSS = require('ali-oss')
const client = new OSS({
  // Set region to the region where the bucket is located. For example, for China (Hangzhou), set region to oss-cn-hangzhou.
  region: 'yourregion',
  // Obtain access credentials from environment variables. Before running this code sample, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set.
  accessKeyId: process.env.OSS_ACCESS_KEY_ID,
  accessKeySecret: process.env.OSS_ACCESS_KEY_SECRET,
  authorizationV4: true,
  // Set bucket to the name of your bucket.
  bucket: 'yourbucketname'
});
async function putBucketLogging () {
  try {
     const result = await client.putBucketLogging('bucket-name', 'logs/');
     console.log(result)
  } catch (e) {
    console.log(e)
  }
}
putBucketLogging();
# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import BucketLogging

# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. 
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Specify the ID of the region that maps to the endpoint. Example: cn-hangzhou. This parameter is required if you use the signature algorithm V4.
region = "cn-hangzhou"

# Specify the name of your bucket.
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)

# Specify that the generated log objects are stored in the current bucket. 
# Set the directory in which the log objects are stored to log/. If you specify this parameter, the log objects are stored in the specified directory of the bucket. If you do not specify this parameter, the log objects are stored in the root directory of the bucket. 
# Enable logging for the bucket. 
logging = bucket.put_bucket_logging(BucketLogging(bucket.bucket_name, 'log/'))
if logging.status == 200:
    print("Enable access logging")
else:
    print("request_id:", logging.request_id)
    print("resp : ", logging.resp.response)            
using Aliyun.OSS;
using Aliyun.OSS.Common;

// Set the Endpoint. This example uses https://oss-cn-hangzhou.aliyuncs.com, the public Endpoint for the China (Hangzhou) region. For other regions, set the Endpoint to the actual value.
var endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
// Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
var accessKeyId = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_ID");
var accessKeySecret = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_SECRET");
// Set the name of the bucket for which to enable log storage, for example, examplebucket.
var bucketName = "examplebucket";
// Set the destination bucket to store the log files. The destination bucket can be the same as or different from the source bucket.
var targetBucketName = "destbucket";
// Set the region where the bucket is located. This example uses cn-hangzhou, which is the ID of the China (Hangzhou) region.
const string region = "cn-hangzhou";

// Create a ClientConfiguration instance and modify the default parameters as needed.
var conf = new ClientConfiguration();

// Use Signature Version 4.
conf.SignatureVersion = SignatureVersion.V4;

// Create an OssClient instance.
var client = new OssClient(endpoint, accessKeyId, accessKeySecret, conf);
client.SetRegion(region);
try
{
    // Set the folder where the log files are stored to log/. If you specify this folder, the log files are saved to the specified folder in the destination bucket. If you do not specify this folder, the log files are saved to the root directory of the destination bucket.
    var request = new SetBucketLoggingRequest(bucketName, targetBucketName, "log/");
    // Enable the log storage feature.
    client.SetBucketLogging(request);
    Console.WriteLine("Set bucket:{0} Logging succeeded ", bucketName);
}
catch (OssException ex)
{
    Console.WriteLine("Failed with error code: {0}. Error message: {1}. \nRequestID:{2}\tHostID:{3}",
        ex.ErrorCode, ex.Message, ex.RequestId, ex.HostId);
}
catch (Exception ex)
{
    Console.WriteLine("Failed with error message: {0}", ex.Message);
}
PutBucketLoggingRequest request = new PutBucketLoggingRequest();
// Specify the source bucket for which to enable access logging.
request.setBucketName("yourSourceBucketName");
// Specify the destination bucket to store access logs.
// The destination bucket and the source bucket must be in the same region. The source and destination buckets can be the same or different.
request.setTargetBucketName("yourTargetBucketName");
// Set the folder where the log files are stored.
request.setTargetPrefix("<yourTargetPrefix>");

OSSAsyncTask task = oss.asyncPutBucketLogging(request, new OSSCompletedCallback<PutBucketLoggingRequest, PutBucketLoggingResult>() {
    @Override
    public void onSuccess(PutBucketLoggingRequest request, PutBucketLoggingResult result) {
        OSSLog.logInfo("code::"+result.getStatusCode());
    }

    @Override
    public void onFailure(PutBucketLoggingRequest request, ClientException clientException, ServiceException serviceException) {
         OSSLog.logError("error: "+serviceException.getRawMessage());
    }
});
task.waitUntilFinished();
package main

import (
	"fmt"
	"os"

	"github.com/aliyun/aliyun-oss-go-sdk/oss"
)

func main() {
	// Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured. 
	provider, err := oss.NewEnvironmentVariableCredentialsProvider()
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}

	// Create an OSSClient instance. 
        // Specify the endpoint of the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhou.aliyuncs.com. Specify your actual endpoint. 
	// Specify the region in which the bucket is located. For example, if the bucket is located in the China (Hangzhou) region, set the region to cn-hangzhou. Specify the actual region.
	clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
	clientOptions = append(clientOptions, oss.Region("yourRegion"))
	// Specify the version of the signature algorithm.
	clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
	client, err := oss.New("yourEndpoint", "", "", clientOptions...)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
	// Specify the name of the source bucket for which you want to enable logging. Example: examplebucket. 
	bucketName := "examplebucket"
	// Specify the name of the destination bucket in which you want to store the log objects. The source and destination buckets can be the same bucket or different buckets, but they must be located in the same region. 
	targetBucketName := "destbucket"
	// Set the directory in which you want to store the log objects to log/. If you specify this parameter, the log objects are stored in the specified directory of the destination bucket. If you do not specify this parameter, the log objects are stored in the root directory of the destination bucket. 
	targetPrefix := "log/"

	// Enable logging for the bucket. 
	err = client.SetBucketLogging(bucketName, targetBucketName, targetPrefix, true)
	if err != nil {
		fmt.Println("Error:", err)
		os.Exit(-1)
	}
}
#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;

int main(void)
{
    /* Initialize the OSS account information. */
            
    /* Set Endpoint to the Endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set Endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
    std::string Endpoint = "yourEndpoint";
    /* Set Region to the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set Region to cn-hangzhou. */
    std::string Region = "yourRegion";
    /* Enter the name of the bucket for which you want to enable log storage, for example, examplebucket. */
    std::string BucketName = "examplebucket";
    /* Enter the destination bucket to store the log files. The targetBucketName and bucketName can be the same or different. */
    std::string TargetBucketName = "destbucket";
    /* Set the folder where the log files are stored to log/. If you specify this parameter, the log files are saved to the specified folder in the destination bucket. If you do not specify this parameter, the log files are saved to the root directory of the destination bucket. */
    std::string TargetPrefix  ="log/";

    /* Initialize network resources. */
    InitializeSdk();

    ClientConfiguration conf;
    conf.signatureVersion = SignatureVersionType::V4;
    /* Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set. */
    auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
    OssClient client(Endpoint, credentialsProvider, conf);
    client.SetRegion(Region); 

    /* Enable the log storage feature. */
    SetBucketLoggingRequest request(BucketName, TargetBucketName, TargetPrefix);
    auto outcome = client.SetBucketLogging(request);

    if (!outcome.isSuccess()) {
        /* Handle exceptions. */
        std::cout << "SetBucketLogging fail" <<
        ",code:" << outcome.error().Code() <<
        ",message:" << outcome.error().Message() <<
        ",requestId:" << outcome.error().RequestId() << std::endl;
        return -1;
    }

    /* Release network resources. */
    ShutdownSdk();
    return 0;
}
#include "oss_api.h"
#include "aos_http_io.h"
/* Set yourEndpoint to the Endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the Endpoint to https://oss-cn-hangzhou.aliyuncs.com. */
const char *endpoint = "yourEndpoint";
/* Specify the bucket name. For example, examplebucket. */
const char *bucket_name = "examplebucket";
/* Specify the destination bucket to store the log files. The targetBucketName and bucketName can be the same or different. */
const char *target_bucket_name = "yourTargetBucketName";
/* Set the folder where the log files are stored. If you specify this parameter, the log files are saved to the specified folder in the destination bucket. If you do not specify this parameter, the log files are saved to the root directory of the destination bucket. */
const char *target_logging_prefix = "yourTargetPrefix";
/* Set yourRegion to the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the region to cn-hangzhou. */
const char *region = "yourRegion";
void init_options(oss_request_options_t *options)
{
    options->config = oss_config_create(options->pool);
    /* Initialize the aos_string_t type with a char* string. */
    aos_str_set(&options->config->endpoint, endpoint);
    /* Obtain access credentials from environment variables. Before you run this sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are set. */
    aos_str_set(&options->config->access_key_id, getenv("OSS_ACCESS_KEY_ID"));
    aos_str_set(&options->config->access_key_secret, getenv("OSS_ACCESS_KEY_SECRET"));
    // You must also configure the following two parameters.
    aos_str_set(&options->config->region, region);
    options->config->signature_version = 4;
    /* Specify whether a CNAME is used. A value of 0 indicates that no CNAME is used. */
    options->config->is_cname = 0;
    /* Set network parameters, such as the timeout period. */
    options->ctl = aos_http_controller_create(options->pool, 0);
}
int main(int argc, char *argv[])
{
    /* Call the aos_http_io_initialize method at the program entry to initialize global resources such as the network and memory. */
    if (aos_http_io_initialize(NULL, 0) != AOSE_OK) {
        exit(1);
    }
    /* The memory pool (pool) for memory management is equivalent to apr_pool_t. The implementation code is in the apr library. */
    aos_pool_t *pool;
    /* Create a memory pool. The second parameter is NULL, which indicates that the memory pool does not inherit from another memory pool. */
    aos_pool_create(&pool, NULL);
    /* Create and initialize options. This parameter includes global configuration information such as the endpoint, access_key_id, access_key_secret, is_cname, and curl. */
    oss_request_options_t *oss_client_options;
    /* Allocate memory to options in the memory pool. */
    oss_client_options = oss_request_options_create(pool);
    /* Initialize the client options oss_client_options. */
    init_options(oss_client_options);
    /* Initialize parameters. */
    aos_string_t bucket;
    oss_logging_config_content_t *content;
    aos_table_t *resp_headers = NULL; 
    aos_status_t *resp_status = NULL; 
    aos_str_set(&bucket, bucket_name);
    content = oss_create_logging_rule_content(pool);
    aos_str_set(&content->target_bucket, target_bucket_name);
    aos_str_set(&content->prefix, target_logging_prefix);
    /* Enable access logging for the bucket. */
    resp_status = oss_put_bucket_logging(oss_client_options, &bucket, content, &resp_headers);
    if (aos_status_is_ok(resp_status)) {
        printf("put bucket logging succeeded\n");
    } else {
        printf("put bucket logging failed, code:%d, error_code:%s, error_msg:%s, request_id:%s\n",
            resp_status->code, resp_status->error_code, resp_status->error_msg, resp_status->req_id); 
    }
    /* Release the memory pool. This releases the memory allocated to resources during the request. */
    aos_pool_destroy(pool);
    /* Release the previously allocated global resources. */
    aos_http_io_deinitialize();
    return 0;
}
require 'aliyun/oss'

client = Aliyun::OSS::Client.new(
  # The China (Hangzhou) region is used as an example for the endpoint. Specify a region as needed.
  endpoint: 'https://oss-cn-hangzhou.aliyuncs.com',
  # Obtain access credentials from environment variables. Before running this code, set the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables.
  access_key_id: ENV['OSS_ACCESS_KEY_ID'],
  access_key_secret: ENV['OSS_ACCESS_KEY_SECRET']
)

# Specify the bucket name, for example, examplebucket.
bucket = client.get_bucket('examplebucket')
# Set logging_bucket to the destination bucket for log files.
# Set my-log to the folder where log files are stored. If you specify this parameter, log files are saved in the specified folder. If you do not specify this parameter, log files are saved in the root directory of the destination bucket.
bucket.logging = Aliyun::OSS::BucketLogging.new(
  enable: true, target_bucket: 'logging_bucket', target_prefix: 'my-log')

API

Call the PutBucketLogging operation to enable logging for a bucket.

Configure logging for a vector bucket

Console

  1. On the Vector Buckets page, click the target bucket. In the left-side navigation pane, choose Log Management > Logging.

  2. Turn on the Logging switch and configure the following parameters:

    • Target storage location: Select a bucket to store log files. The bucket must be in the same region as the vector bucket.

    • Log prefix: Set the directory and prefix for log files, such as MyLog-.

    • RAM role: Use the default service role AliyunOSSLoggingDefaultRole for logging or select a custom role.

Ossutil

The following examples show how to enable log storage for a bucket named examplebucket. The log file prefix is MyLog-, and the access logs are stored in the examplebucket bucket.

  • You can use a JSON configuration file. The bucket-logging-status.json file contains the following content:

    {
      "BucketLoggingStatus": {
        "LoggingEnabled": {
          "TargetBucket": "examplebucket",
          "TargetPrefix": "MyLog-",
          "LoggingRole": "AliyunOSSLoggingDefaultRole"
        }
      }
    }

    Example command:

    ossutil vectors-api put-bucket-logging --bucket examplebucket --bucket-logging-status file://bucket-logging-status.json
  • You can use JSON configuration parameters. Example command:

    ossutil vectors-api put-bucket-logging --bucket examplebucket --bucket-logging-status "{\"BucketLoggingStatus\":{\"LoggingEnabled\":{\"TargetBucket\":\"examplebucket\",\"TargetPrefix\":\"MyLog-\",\"LoggingRole\":\"AliyunOSSLoggingDefaultRole\"}}}"

SDK

Python

import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.vectors as oss_vectors

parser = argparse.ArgumentParser(description="vector put bucket logging sample")
parser.add_argument('--region', help='The region in which the bucket is located.', required=True)
parser.add_argument('--bucket', help='The name of the bucket.', required=True)
parser.add_argument('--endpoint', help='The endpoint to access OSS')
parser.add_argument('--account_id', help='The account ID.', required=True)
parser.add_argument('--target_bucket', help='The name of the target bucket.', required=True)

def main():
    args = parser.parse_args()

    # Load credentials from environment variables
    credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()

    # Use the SDK's default configuration
    cfg = oss.config.load_default()
    cfg.credentials_provider = credentials_provider
    cfg.region = args.region
    cfg.account_id = args.account_id
    cfg.use_internal_endpoint = True  # Set to False to use the public network endpoint.
    if args.endpoint is not None:
        cfg.endpoint = args.endpoint

    vector_client = oss_vectors.Client(cfg)

    result = vector_client.put_bucket_logging(oss_vectors.models.PutBucketLoggingRequest(
        bucket=args.bucket,
        bucket_logging_status=oss_vectors.models.BucketLoggingStatus(
            logging_enabled=oss_vectors.models.LoggingEnabled(
                target_bucket=args.target_bucket,
                target_prefix='log-prefix',
                logging_role='AliyunOSSLoggingDefaultRole'
            )
        )
    ))

    print(f'status code: {result.status_code},'
          f' request id: {result.request_id},'
    )

if __name__ == "__main__":
    main()

Go

package main

import (
	"context"
	"flag"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/vectors"
	"log"

	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
)

var (
	region     string
	bucketName string
	accountId  string
)

func init() {
	flag.StringVar(&region, "region", "", "The region in which the vector bucket is located.")
	flag.StringVar(&bucketName, "bucket", "", "The name of the vector bucket.")
	flag.StringVar(&accountId, "account-id", "", "The id of vector account.")
}

func main() {
	flag.Parse()
	if len(bucketName) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, bucket name required")
	}

	if len(region) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, region required")
	}

	if len(accountId) == 0 {
		flag.PrintDefaults()
		log.Fatalf("invalid parameters, accountId required")
	}

	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region).WithAccountId(accountId).
		// To access OSS over the public internet, set this parameter to false or remove this line.
		WithUseInternalEndpoint(true)

	client := vectors.NewVectorsClient(cfg)

	request := &vectors.PutBucketLoggingRequest{
		Bucket: oss.Ptr(bucketName),
		BucketLoggingStatus: &vectors.BucketLoggingStatus{
			&vectors.LoggingEnabled{
				TargetBucket: oss.Ptr("TargetBucket"),
				TargetPrefix: oss.Ptr("TargetPrefix"),
				LoggingRole:  oss.Ptr("AliyunOSSLoggingDefaultRole"),
			},
		},
	}
	result, err := client.PutBucketLogging(context.TODO(), request)
	if err != nil {
		log.Fatalf("failed to put vector bucket logging %v", err)
	}

	log.Printf("put vector bucket logging result:%#v\n", result)
}

API

Call the PutBucketLogging operation to enable logging for a vector bucket.

Log file naming

Log files use the following naming convention:

<TargetPrefix><SourceBucket>YYYY-mm-DD-HH-MM-SS-UniqueString

Parameter

Description

TargetPrefix

The prefix for the log file name.

SourceBucket

The source bucket that generates access logs.

YYYY-mm-DD-HH-MM-SS

The timestamp in year, month, day, hour, minute, and second format. Logs use hourly granularity: HH=01 covers 01:00:00 to 01:59:59. MM and SS are always 00.

UniqueString

A system-generated unique identifier for the log file.

Log format and example

  • Log format

    OSS access logs contain information about the requester and the accessed resource. The format is as follows:

    RemoteIP Reserved Reserved Time "RequestURL" HTTPStatus SentBytes RequestTime "Referer" "UserAgent" "HostName" "RequestID" "LoggingFlag" "RequesterAliyunID" "Operation" "BucketName" "ObjectName" ObjectSize ServerCostTime "ErrorCode" RequestLength "UserID" DeltaDataSize "SyncRequest" "StorageClass" "TargetStorageClass" "TransmissionAccelerationAccessPoint" "AccessKeyID" "BucketARN"

    Field

    Example

    Description

    RemoteIP

    192.168.0.1

    The IP address of the requester.

    Reserved

    -

    A reserved field. The value is always a hyphen (-).

    Reserved

    -

    A reserved field. The value is always a hyphen (-).

    Time

    03/Jan/2021:14:59:49 +0800

    The time when OSS received the request.

    RequestURL

    GET /example.jpg HTTP/1.0

    The request URL that contains a query string.

    OSS ignores query string parameters starting with x- but records them in the access log. Use x- prefixed parameters to tag and locate specific requests.

    HTTPStatus

    200

    The HTTP status code returned by OSS.

    SentBytes

    999131

    The downstream traffic generated by the request, in bytes.

    RequestTime

    127

    The request completion time, in milliseconds.

    Referer

    http://www.aliyun.com/product/oss

    The HTTP Referer of the request.

    UserAgent

    curl/7.15.5

    The User-Agent header of the HTTP request.

    HostName

    examplebucket.oss-cn-hangzhou.aliyuncs.com

    The destination domain name.

    RequestID

    5FF16B65F05BC932307A3C3C

    The request ID.

    LoggingFlag

    true

    Indicates whether logging is enabled. Valid values:

    • true: Logging is enabled.

    • false: Logging is not enabled.

    RequesterAliyunID

    16571836914537****

    The user ID of the requester. A hyphen (-) indicates an anonymous request.

    Operation

    GetObject

    The operation performed.

    BucketName

    examplebucket

    The name of the destination bucket.

    ObjectName

    example.jpg

    The name of the destination object.

    ObjectSize

    999131

    The size of the destination object, in bytes.

    ServerCostTime

    88

    The time OSS took to process the request, in milliseconds.

    ErrorCode

    -

    The error code returned by OSS. A hyphen (-) indicates that no error was returned.

    RequestLength

    302

    The length of the request, in bytes.

    UserID

    16571836914537****

    The ID of the bucket owner.

    DeltaDataSize

    -

    The change in the object size. A hyphen (-) indicates the request did not involve a write operation.

    SyncRequest

    -

    The type of request. Valid values:

    • -: A general request.

    • cdn: A CDN origin request.

    • lifecycle: A request to transition or delete data, triggered by a lifecycle rule.

    StorageClass

    Standard

    The storage class of the destination object. Valid values:

    • Standard: Standard.

    • IA: Infrequent Access.

    • Archive: Archive.

    • Cold Archive: Cold Archive.

    • Deep Cold Archive: Deep Cold Archive.

    • -: The object's storage class could not be obtained.

    TargetStorageClass

    -

    The storage class after a lifecycle or CopyObject transition. Valid values:

    • Standard: Changed to Standard.

    • IA: Changed to Infrequent Access.

    • Archive: Changed to Archive.

    • Cold Archive: Changed to Cold Archive.

    • Deep Cold Archive: Changed to Deep Cold Archive.

    • -: No object storage class conversion operation is involved.

    TransmissionAccelerationAccessPoint

    -

    The transfer acceleration endpoint region used to access the bucket. Example: cn-hangzhou for the China (Hangzhou) region.

    A hyphen (-) indicates no transfer acceleration domain was used, or the endpoint is in the same region as the bucket.

    AccessKeyID

    LTAI****************

    The AccessKey ID of the requester.

    • For requests from the console, the log field shows a temporary AccessKey ID that starts with TMP.

    • For requests from a tool or an SDK using a long-term key, the log field shows a common AccessKey ID. Example: LTAI****************.

    • For requests using temporary access credentials from Security Token Service (STS), the log shows a temporary AccessKey ID that starts with STS.

    Note

    A hyphen (-) in this field indicates an anonymous request.

    BucketARN

    acs:oss***************

    The globally unique resource descriptor for the bucket.

  • Log example

    192.168.0.1 - - [03/Jan/2021:14:59:49 +0800] "GET /example.jpg HTTP/1.0" 200 999131 127 "http://www.aliyun.com/product/oss" "curl/7.15.5" "examplebucket.oss-cn-hangzhou.aliyuncs.com" "5FF16B65F05BC932307A3C3C" "true" "16571836914537****" "GetObject" "examplebucket" "example.jpg" 999131 88 "-" 302 "16571836914537****" - "cdn" "standard" "-" "-" "LTAI****************" "acs:oss***************"

    Import stored log files into Log Service for analysis. Import OSS data. Query and analysis overview.

FAQ

Querying interrupted requests

OSS access logs do not record interrupted requests. Check the SDK return value to determine the cause of interruption.