注意事项
如果生成日志的源Bucket有地域属性,则存储日志的目标Bucket与源Bucket可以相同也可以不同,但是必须属于同一账号下的相同地域。
如果生成日志的源Bucket无地域属性,则存储日志的目标Bucket必须与源Bucket为同一个Bucket。
日志文件预计会在48小时内生成。某个时段的日志文件可能不会记录该时段的所有请求,部分请求可能会出现在上一时段或下一时段的日志文件中。因此,某个时段的日志文件不能确保该时段日志记录的完整性和即时性。
在您关闭日志转存功能前,OSS的日志文件会每隔1小时生成一次。请及时清理不再需要的日志文件,以减少您的存储费用。
您可以通过生命周期规则定期删除日志文件。更多信息,请参见基于最后一次修改时间的生命周期规则。
为避免影响OSS-HDFS服务的正常使用或者引发数据污染的风险,在开通了OSS-HDFS服务的Bucket中设置日志转存规则时,禁止将日志前缀填写为.dlsdata/
。
OSS会根据需求在日志的尾部添加一些字段,请您在开发日志处理工具时考虑兼容性的问题。
日志文件命名规则
转存后的日志文件命名规则如下:
<TargetPrefix><SourceBucket>YYYY-mm-DD-HH-MM-SS-UniqueString
字段 | 说明 |
TargetPrefix | 日志文件的文件名前缀。 |
SourceBucket | 产生访问日志的源Bucket名称。 |
YYYY-mm-DD-HH-MM-SS | 日志文件被创建的时间。从左到右分别表示:年、月、日、小时、分钟和秒。 |
UniqueString | 系统生成的字符串,是日志文件的唯一标识。 |
操作方式
使用OSS控制台
登录OSS管理控制台。
单击Bucket 列表,然后单击目标Bucket名称。
在左侧导航栏,选择。
在日志转存页面,开启日志存储,并设置日志存储位置及日志前缀。
单击保存。
使用阿里云SDK
以下仅列举常见SDK的设置日志转存的代码示例。关于其他SDK的设置日志转存的代码示例,请参见SDK简介。
Java
PHP
Node.js
Python
C#
Android-Java
Go
C++
C
Ruby
import com.aliyun.oss.ClientException;
import com.aliyun.oss.OSS;
import com.aliyun.oss.common.auth.*;
import com.aliyun.oss.OSSClientBuilder;
import com.aliyun.oss.OSSException;
import com.aliyun.oss.model.SetBucketLoggingRequest;
public class Demo {
public static void main(String[] args) throws Exception {
String endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
EnvironmentVariableCredentialsProvider credentialsProvider = CredentialsProviderFactory.newEnvironmentVariableCredentialsProvider();
String bucketName = "examplebucket";
String targetBucketName = "yourTargetBucketName";
String targetPrefix = "log/";
String region = "cn-hangzhou";
ClientBuilderConfiguration clientBuilderConfiguration = new ClientBuilderConfiguration();
clientBuilderConfiguration.setSignatureVersion(SignVersion.V4);
OSS ossClient = OSSClientBuilder.create()
.endpoint(endpoint)
.credentialsProvider(credentialsProvider)
.clientConfiguration(clientBuilderConfiguration)
.region(region)
.build();
try {
SetBucketLoggingRequest request = new SetBucketLoggingRequest(bucketName);
request.setTargetBucket(targetBucketName);
request.setTargetPrefix(targetPrefix);
ossClient.setBucketLogging(request);
} catch (OSSException oe) {
System.out.println("Caught an OSSException, which means your request made it to OSS, "
+ "but was rejected with an error response for some reason.");
System.out.println("Error Message:" + oe.getErrorMessage());
System.out.println("Error Code:" + oe.getErrorCode());
System.out.println("Request ID:" + oe.getRequestId());
System.out.println("Host ID:" + oe.getHostId());
} catch (ClientException ce) {
System.out.println("Caught an ClientException, which means the client encountered "
+ "a serious internal problem while trying to communicate with OSS, "
+ "such as not being able to access the network.");
System.out.println("Error Message:" + ce.getMessage());
} finally {
if (ossClient != null) {
ossClient.shutdown();
}
}
}
}
<?php
if (is_file(__DIR__ . '/../autoload.php')) {
require_once __DIR__ . '/../autoload.php';
}
if (is_file(__DIR__ . '/../vendor/autoload.php')) {
require_once __DIR__ . '/../vendor/autoload.php';
}
use OSS\Credentials\EnvironmentVariableCredentialsProvider;
use OSS\OssClient;
use OSS\CoreOssException;
$provider = new EnvironmentVariableCredentialsProvider();
$endpoint = "yourEndpoint";
$bucket= "examplebucket";
$option = array();
$targetBucket = "destbucket";
$targetPrefix = "log/";
try {
$config = array(
"provider" => $provider,
"endpoint" => $endpoint,
"signatureVersion" => OssClient::OSS_SIGNATURE_VERSION_V4,
"region"=> "cn-hangzhou"
);
$ossClient = new OssClient($config);
$ossClient->putBucketLogging($bucket, $targetBucket, $targetPrefix, $option);
} catch (OssException $e) {
printf(__FUNCTION__ . ": FAILED\n");
printf($e->getMessage() . "\n");
return;
}
print(__FUNCTION__ . ": OK" . "\n");
const OSS = require('ali-oss')
const client = new OSS({
region: 'yourregion',
accessKeyId: process.env.OSS_ACCESS_KEY_ID,
accessKeySecret: process.env.OSS_ACCESS_KEY_SECRET,
authorizationV4: true,
bucket: 'yourbucketname'
});
async function putBucketLogging () {
try {
const result = await client.putBucketLogging('bucket-name', 'logs/');
console.log(result)
} catch (e) {
console.log(e)
}
}
putBucketLogging();
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
from oss2.models import BucketLogging
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
region = "cn-hangzhou"
bucket = oss2.Bucket(auth, endpoint, "examplebucket", region=region)
logging = bucket.put_bucket_logging(BucketLogging(bucket.bucket_name, 'log/'))
if logging.status == 200:
print("Enable access logging")
else:
print("request_id :", logging.request_id)
print("resp : ", logging.resp.response)
using Aliyun.OSS;
using Aliyun.OSS.Common;
var endpoint = "https://oss-cn-hangzhou.aliyuncs.com";
var accessKeyId = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_ID");
var accessKeySecret = Environment.GetEnvironmentVariable("OSS_ACCESS_KEY_SECRET");
var bucketName = "examplebucket";
var targetBucketName = "destbucket";
const string region = "cn-hangzhou";
var conf = new ClientConfiguration();
conf.SignatureVersion = SignatureVersion.V4;
var client = new OssClient(endpoint, accessKeyId, accessKeySecret, conf);
c.SetRegion(region);
try
{
var request = new SetBucketLoggingRequest(bucketName, targetBucketName, "log/");
client.SetBucketLogging(request);
Console.WriteLine("Set bucket:{0} Logging succeeded ", bucketName);
}
catch (OssException ex)
{
Console.WriteLine("Failed with error info: {0}; Error info: {1}. \nRequestID:{2}\tHostID:{3}",
ex.ErrorCode, ex.Message, ex.RequestId, ex.HostId);
}
catch (Exception ex)
{
Console.WriteLine("Failed with error info: {0}", ex.Message);
}
PutBucketLoggingRequest request = new PutBucketLoggingRequest();
request.setBucketName("yourSourceBucketName");
request.setTargetBucketName("yourTargetBucketName");
request.setTargetPrefix("<yourTargetPrefix>");
OSSAsyncTask task = oss.asyncPutBucketLogging(request, new OSSCompletedCallback<PutBucketLoggingRequest, PutBucketLoggingResult>() {
@Override
public void onSuccess(PutBucketLoggingRequest request, PutBucketLoggingResult result) {
OSSLog.logInfo("code::"+result.getStatusCode());
}
@Override
public void onFailure(PutBucketLoggingRequest request, ClientException clientException, ServiceException serviceException) {
OSSLog.logError("error: "+serviceException.getRawMessage());
}
});
task.waitUntilFinished();
package main
import (
"fmt"
"os"
"github.com/aliyun/aliyun-oss-go-sdk/oss"
)
func main() {
provider, err := oss.NewEnvironmentVariableCredentialsProvider()
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
clientOptions := []oss.ClientOption{oss.SetCredentialsProvider(&provider)}
clientOptions = append(clientOptions, oss.Region("yourRegion"))
clientOptions = append(clientOptions, oss.AuthVersion(oss.AuthV4))
client, err := oss.New("yourEndpoint", "", "", clientOptions...)
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
bucketName := "examplebucket"
targetBucketName := "destbucket"
targetPrefix := "log/"
err = client.SetBucketLogging(bucketName, targetBucketName, targetPrefix, true)
if err != nil {
fmt.Println("Error:", err)
os.Exit(-1)
}
}
#include <alibabacloud/oss/OssClient.h>
using namespace AlibabaCloud::OSS;
int main(void)
{
std::string Endpoint = "yourEndpoint";
std::string Region = "yourRegion";
std::string BucketName = "examplebucket";
std::string TargetBucketName = "destbucket";
std::string TargetPrefix ="log/";
InitializeSdk();
ClientConfiguration conf;
conf.signatureVersion = SignatureVersionType::V4;
auto credentialsProvider = std::make_shared<EnvironmentVariableCredentialsProvider>();
OssClient client(Endpoint, credentialsProvider, conf);
client.SetRegion(Region);
SetBucketLoggingRequest request(BucketName, TargetBucketName, TargetPrefix);
auto outcome = client.SetBucketLogging(request);
if (!outcome.isSuccess()) {
std::cout << "SetBucketLogging fail" <<
",code:" << outcome.error().Code() <<
",message:" << outcome.error().Message() <<
",requestId:" << outcome.error().RequestId() << std::endl;
return -1;
}
ShutdownSdk();
return 0;
}
#include "oss_api.h"
#include "aos_http_io.h"
const char *endpoint = "yourEndpoint";
const char *bucket_name = "examplebucket";
const char *target_bucket_name = "yourTargetBucketName";
const char *target_logging_prefix = "yourTargetPrefix";
const char *region = "yourRegion";
void init_options(oss_request_options_t *options)
{
options->config = oss_config_create(options->pool);
aos_str_set(&options->config->endpoint, endpoint);
aos_str_set(&options->config->access_key_id, getenv("OSS_ACCESS_KEY_ID"));
aos_str_set(&options->config->access_key_secret, getenv("OSS_ACCESS_KEY_SECRET"));
aos_str_set(&options->config->region, region);
options->config->signature_version = 4;
options->config->is_cname = 0;
options->ctl = aos_http_controller_create(options->pool, 0);
}
int main(int argc, char *argv[])
{
if (aos_http_io_initialize(NULL, 0) != AOSE_OK) {
exit(1);
}
aos_pool_t *pool;
aos_pool_create(&pool, NULL);
oss_request_options_t *oss_client_options;
oss_client_options = oss_request_options_create(pool);
init_options(oss_client_options);
aos_string_t bucket;
oss_logging_config_content_t *content;
aos_table_t *resp_headers = NULL;
aos_status_t *resp_status = NULL;
aos_str_set(&bucket, bucket_name);
content = oss_create_logging_rule_content(pool);
aos_str_set(&content->target_bucket, target_bucket_name);
aos_str_set(&content->prefix, target_logging_prefix);
resp_status = oss_put_bucket_logging(oss_client_options, &bucket, content, &resp_headers);
if (aos_status_is_ok(resp_status)) {
printf("put bucket logging succeeded\n");
} else {
printf("put bucket logging failed, code:%d, error_code:%s, error_msg:%s, request_id:%s\n",
resp_status->code, resp_status->error_code, resp_status->error_msg, resp_status->req_id);
}
aos_pool_destroy(pool);
aos_http_io_deinitialize();
return 0;
}
require 'aliyun/oss'
client = Aliyun::OSS::Client.new(
endpoint: 'https://oss-cn-hangzhou.aliyuncs.com',
access_key_id: ENV['OSS_ACCESS_KEY_ID'],
access_key_secret: ENV['OSS_ACCESS_KEY_SECRET']
)
bucket = client.get_bucket('examplebucket')
bucket.logging = Aliyun::OSS::BucketLogging.new(
enable: true, target_bucket: 'logging_bucket', target_prefix: 'my-log')
使用命令行工具ossutil
您可以使用命令行工具ossutil来开启日志转存功能,ossutil的安装请参见安装ossutil。
以下命令用于为存储空间examplebucket
开启日志转存功能,日志文件前缀为MyLog-
,存储访问日志的Bucket为dest-bucket
。
ossutil api put-bucket-logging --bucket examplebucket --bucket-logging-status "{\"LoggingEnabled\":{\"TargetBucket\":\"destBucket\",\"TargetPrefix\":\"MyLog-\"}}"
关于该命令的更多信息,请参见put-bucket-logging。
相关API
以上操作方式底层基于API实现,如果您对程序自定义要求较高,您可以直接发起REST API请求。直接发起REST API请求需要手动编写代码计算签名。更多信息,请参见PutBucketLogging。
常见问题
中断的请求能否在OSS访问日志中查询?
OSS访问日志目前不会记录中断的请求。如果您是通过SDK发起的请求,可以根据SDK的返回值判断请求中断的原因。