DocsICP FilingConsole
官方文档
首页

JSON format

更新时间：
复制 MD 格式
产品详情
我的收藏

Simple Log Service can ship logs to Object Storage Service (OSS) and store them in JSON format.

Parameters

When you create an OSS data shipping job (new version), if you set Storage Format to json, configure the following parameters.

Parameter

Description

Ship Tag

Specifies whether to ship log tag fields. Example: __tag__:abc.

URLs of OSS objects

The following table lists sample OSS object URLs after data shipping.

Note

  • If you specify an object suffix when you create a data shipping job, the objects use that suffix.

  • If you do not specify an object suffix when you create a data shipping job, the objects use a default suffix based on the compression type.

Compression type

Object suffix

Sample URL

Description

Not compressed

If you specify an object suffix, the specified suffix is used. Example: .json.

oss://oss-shipper-chengdu/ecs_test/2022/01/26/20/54_1453812893059571256_937.json

Download the JSON file and open it as a text file. Sample content: 

{"__time__":"1453809242","__topic__":"","__source__":"10.170.***.***","ip":"10.200.**.***","time":"26/Jan/2016:19:54:02 +0800","url":"POST
              /PutData?Category=YunOsAccountOpLog&AccessKeyId=<yourAccessKeyId>&Date=Fri%2C%2028%20Jun%202013%2006%3A53%3A30%20GMT&Topic=raw&Signature=<yourSignature>
              HTTP/1.1","status":"200","user-agent":"aliyun-sdk-java"}

If you do not specify an object suffix, the object does not have a suffix.

oss://oss-shipper-chengdu/ecs_test/2022/01/26/20/54_1453812893059571256_937

Snappy

If you specify an object suffix, the specified suffix is used. Example: .json.

oss://oss-shipper-chengdu/ecs_test/2022/01/26/20/54_1453812893059571256_937.json

For information about Snappy compression and decompression, see Decompression tools for Snappy-compressed files.

Problem:

If you use Snappy to compress objects and read them with Hive in an E-MapReduce (EMR) cluster, the following error may occur.

Failed with exception java.io.IOException:java.lang.ArrayIndexOutOfBoundsException

Solution:

Run the io.compression.codec.snappy.native=true; statement before running a query. For other Hive-related issues in E-MapReduce, submit a ticket.

If you do not specify an object suffix, the .snappy suffix is used.

oss://oss-shipper-chengdu/ecs_test/2022/01/26/20/54_1453812893059571256_937.snappy

gzip

If you specify an object suffix, the specified suffix is used. Example: .json.

oss://oss-shipper-chengdu/ecs_test/2022/01/26/20/54_1453812893059571256_937.json

To decompress gzip-compressed objects, install gzip and run the corresponding gzip commands.

If you do not specify an object suffix, the .gz suffix is used.

oss://oss-shipper-chengdu/ecs_test/2022/01/26/20/54_1453812893059571256_937.gz

Zstandard

If you specify an object suffix, the specified suffix is used. Example: .json.

oss://oss-shipper-chengdu/ecs_test/2022/01/26/20/54_1453812893059571256_937.json

To decompress Zstandard-compressed objects, install Zstandard and run the corresponding Zstandard commands.

If you do not specify an object suffix, the .zst suffix is used.

oss://oss-shipper-chengdu/ecs_test/2022/01/26/20/54_1453812893059571256_937.zst