Encoding and decoding syntax-Simple Log Service(SLS)-阿里云帮助中心

Functions

Type	Subcategory	Function	Description
Encoding and decoding	String	str_encode	Encodes data.
	String	str_decode	Decodes data.
	Base64	base64_encoding	Encodes data using Base64.
	Base64	base64_decoding	Decodes Base64-encoded data.
	HTML	html_encoding	Encodes data for use in HTML.
	HTML	html_decoding	Decodes HTML-encoded data.
	URL	url_encoding	Encodes data for use in a URL.
	URL	url_decoding	Decodes URL-encoded data.
	Protobuf	protobuf_decoding	Parses data into JSON format using a specified Protobuf template.
	JSON Web Token (JWT)	jwt_encoding	Encodes JSON data based on the JWT standard.
	JSON Web Token (JWT)	jwt_decoding	Decodes data into raw JSON based on the JWT standard.
	Hashids	hashids_encoding	Encodes data using the Hashids library.
	Hashids	hashids_decoding	Decodes data encoded with the Hashids library.
Compression and decompression	Gzip	gzip_compress	Compresses and encodes data.
	Gzip	gzip_decompress	Decompresses data.
	Zlib	zlib_compress	Compresses and encodes data.
	Zlib	zlib_decompress	Decompresses data.
Encryption and decryption	Advanced Encryption Standard (AES)	aes_encrypt	Encrypts data using AES.
Encryption and decryption	Advanced Encryption Standard (AES)	aes_decrypt	Decrypts AES-encrypted data.
Hash	MD5	md5_encoding	Calculates the MD5 hash of data.
	SHA1	sha1_encoding	Calculates the SHA1 hash of data.
	Cyclic redundancy check (CRC)	crc32_encoding	Calculates the CRC32 checksum for data.

str_encode

Encodes the string using the specified encoding format.

Syntax

str_encode(value, "utf8", errors="ignore")

Parameters

Parameter	Type	Required	Description
value	Arbitrary (automatically converted to the string type)	Yes	The value that you want to encode.
encoding	String	No	The encoding format. Default value: UTF-8. ASCII is supported.
errors	String	No	The method that is used to process characters if the characters cannot be recognized based on the encoding format. Valid values: ignore (default): The characters are not encoded. strict: An error is reported, and the log is discarded. replace: The unrecognizable characters are replaced with question marks (?). xmlcharrefreplace: The unrecognizable characters are replaced with XML characters.

Response
An encoded string is returned.

Examples

Example 1

Raw log
```
test: asewds
```

Transformation rule

e_set("f1", str_decode(str_encode("Hello", "utf8"), "utf8"))

Result
```
test: asewds
f1: Hello
```

Example 2

Raw log
```
f2: test Test data
```

Transformation rule

e_set("f1", str_encode(v("f2"), "ascii", errors="ignore"))

Result
```
f1:test 
f2:test Test data
```

Example 3
- Raw log
```
f2: test data
```
- Transformation rule
```
e_set("f1", str_encode(v("f2"), "ascii", errors="strict"))
```
- Result
  An error is reported during execution.

Example 4

Raw log
```
f2: test Test data
```

Transformation rule

e_set("f1", str_encode(v("f2"), "ascii", errors="replace"))

Result
```
f1:test ????
f2:test Test data
```

Example 5

Raw log
```
f2: test Test data
```

Transformation rule

e_set("f1", str_encode(v("f2"), "ascii", errors="xmlcharrefreplace"))

Result

f1:test &#27979;&#35797;&#25968;&#25454;
f2:test Test data

str_decode

Decodes the input value using the specified encoding format.

Syntax
```
str_decode(value, "utf8", errors="ignore")
```
Note
The str_decode function can process only the data of the byte data type.

Parameters

Parameter Name	Data Types	Required	Description
value	Arbitrary (automatically converted to the string type)	Yes	The value that you want to decode.
encoding	Arbitrary (automatically converted to the string type)	No	The encoding format. Default value: UTF-8. ASCII is supported.
errors	Arbitrary (automatically converted to the string type)	No	The method that is used to process characters if the characters cannot be recognized based on the encoding format. Valid values: ignore (default): The characters are not decoded. strict: An error is reported, and the log is discarded. replace: The unrecognizable characters are replaced with question marks (?). xmlcharrefreplace: The unrecognizable characters are replaced with XML characters.

Response
A decoded value is returned.

Examples

Raw log
```
test: asewds
```

Transformation rule

e_set("encoding", str_decode(b'\xe4\xbd\xa0\xe5\xa5\xbd', "utf8", 'strict'))

Result
```
test: asewds
encoding: Hello
```

base64_encoding

Encodes data using base64 encoding.

Syntax
```
base64_encoding(value, format=None)
```

Parameters

Parameter	Type	Required	Description
value	string	Yes	The string to encode.
format	string	No	The base64 encoding standard. Defaults to `format=RFC3548`. You can also use `format=RFC4648`.

Response

Returns the encoded string.

Examples

Raw log
```
str_en : data to be encoded
```

Transformation rule

e_set("str_base64",base64_encoding(v("str_en")))

Result

str_en : data to be encoded
str_base64 : ZGF0YSB0byBiZSBlbmNvZGVk

base64_decoding

Decodes a Base64-encoded string.

Syntax
```
base64_decoding(value, format=None)
```

Parameters

Parameter	Type	Required	Description
value	String	Yes	The Base64-encoded string to decode.
format	String	No	The Base64 decoding scheme. Defaults to `RFC3548`. You can also specify `RFC4648`. Note The RFC 4648 scheme pads the encoded string with the equal sign (`=`) to ensure that its length is a multiple of four characters.

Response

The decoded string.

Examples

Raw log
```
str_de: ZGF0YSB0byBiZSBlbmNvZGVk
```

Transformation rule

e_set("str_de_base64",base64_decoding(v("str_de")))

Result

str_de: ZGF0YSB0byBiZSBlbmNvZGVk
str_de_base64: data to be encoded

html_encoding

Encodes a string for use in HTML.

Syntax
```
html_encoding(value)
```

Parameters

Parameter	Type	Required	Description
value	String	Yes	The string to encode.

Response

The encoded string.

Examples

Raw log

str : <img class="size-medium wp-image-113" style="margin-left: 15px;" title="su1" src="http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg" alt="" width="300" height="194" />

Transformation rule

e_set("str_html_en",html_encoding(v("str")))

Transformation result

str : <img class="size-medium wp-image-113" style="margin-left: 15px;" title="su1" src="http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg" alt="" width="300" height="194" />
str_html_en : &lt;img class=&quot;size-medium wp-image-113&quot; style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; src=&quot;http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg&quot; alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;

html_decoding

Decodes an HTML-encoded string.

Syntax
```
html_decoding(value)
```

Parameters

Parameter	Type	Required	Description
value	String	Yes	The HTML-encoded string to decode.

Response

The decoded string.

Examples

Raw log

str : &lt;img class=&quot;size-medium wp-image-113&quot; style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; src=&quot;http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg&quot; alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;

Transformation rule

e_set("str_html_de",html_decoding(v("str")))

Result

str : &lt;img class=&quot;size-medium wp-image-113&quot; style=&quot;margin-left: 15px;&quot; title=&quot;su1&quot; src=&quot;http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg&quot; alt=&quot;&quot; width=&quot;300&quot; height=&quot;194&quot; /&gt;
str_html_de : <img class="size-medium wp-image-113" style="margin-left: 15px;" title="su1" src="http://aliyundoc.com/wp-content/uploads/2008/10/su1-300x194.jpg" alt="" width="300" height="194" />

url_encoding

Encodes a string for use in a URL.

Syntax
```
url_encoding(value, plus=False)
```

Parameters

Parameter

Type

Required

Description

value

String

Yes

The string to encode.

plus

Boolean

No

Specifies whether to convert spaces to plus signs (+). The default is False.

True: Converts spaces to plus signs (+).
False: Encodes spaces as %20.

Response

Returns the URL-encoded string.

Examples

Example 1: Encode a URL string.

Raw log

content : https://www.example.org/hello/asdah

Transformation rule
```
e_set("url",url_encoding(v("content")))
```

Result

content : https://www.example.org/hello/asdah
url: https%3A%2F%2Fwww.example.org%2Fhello%2Fasdah

Example 2: Encode a string that contains a space. Because the plus parameter is omitted, the space is encoded as %20.
- Raw log
```
content : 1 2+3:4
```
- Transformation rule
```
e_set("url",url_encoding(v("content")))
```
- Result
```
content : 1 2+3:4
url:1%202%2B3%3A4
```
Example 3: Set the plus parameter to True to convert spaces to a plus sign (+).
- Raw log
```
content : 1 2+3:4
```
- Transformation rule
```
e_set("url", url_encoding(v("content"), plus=True))
```
- Result
```
content : 1 2+3:4
url:1+2%2B3%3A4
```

URL decoding

Decodes a URL-encoded string.

Syntax
```
url_decoding(value, plus=False)
```

Parameters

Parameter

Type

Required

Description

value

String

Yes

The URL-encoded string to decode.

plus

Boolean

No

Specifies whether to convert plus signs (+) to spaces. Defaults to False.

True: Converts plus signs (+) to spaces.
False: Does not convert plus signs (+) to spaces.

Response

Returns the decoded string.

Examples

Example 1: Decode a URL-encoded string.

Raw log

content : https%3A%2F%www.example.org%2FHello%2Fasdah

Transformation rule
```
e_set("URL",url_decoding(v("content")))
```

Result

content : https%3A%2F%www.example.org%2FHello%2Fasdah
URL : https://www.example.org/Hello/asdah

Example 2: Decode a URL-encoded string. In this example, the plus parameter is omitted, so plus signs (+) in the string are not converted to spaces.

Raw log

content : /answer?event_date=2022-06-30+09%3A06%3A53%20123

Transformation rule
```
e_set("URL",url_decoding(v("content")))
```

Result

content:/answer?event_date=2022-06-30+09%3A06%3A53%20123
URL:/answer?event_date=2022-06-30+09:06:53 123

Example 3: Decode a URL-encoded string. In this example, setting the plus parameter to True converts plus signs (+) in the string to spaces.

Raw log

content : /answer?event_date=2022-06-30+09%3A06%3A53%20123

Transformation rule

e_set("URL",url_decoding(v("content"),plus=True))

Result

content:/answer?event_date=2022-06-30+09%3A06%3A53%20123
URL:/answer?event_date=2022-06-30 09:06:53 123

protobuf_decoding

Parses Protobuf-serialized data into JSON format using a Protobuf template.

Syntax

protobuf_decoding(data, protocol, input_format="bytes", message_name=None)

Parameters

Parameter	Type	Required	Description
data	String	Yes	The Protobuf-serialized data.
protocol	String	Yes	The Protobuf template.
input_format	String	No	The format of the input data. Valid values: `hex`: The data is hexadecimal. `bytes`: The data is a byte string. `raw`: Same as `bytes`. `base64`: The data is Base64-encoded.
message_name	String	No	The name of the message to retrieve.

Response

Returns a list of JSON objects, or a single JSON object if message_name is specified.

Examples

Example 1: Parse Base64-encoded Protobuf data into JSON format using a template.

Raw log

{
  "protocol": 'syntax = "proto3"; message Person {  string name = 1;  int32 id = 2; string email = 3;}  ',
  "data": "Cgl4aWFvIG1pbmcQARoOMTIzMTIzQDEyMy5jb20="
}

Transformation rule

e_set("data", protobuf_decoding(v("data"), v("protocol"), "base64", "Person"))

Result

data:{
  "name": "xiao ming",
  "id": 1,
  "email": "123123@123.com"
}
protocol:syntax = "proto3"; message Person {  string name = 1;  int32 id = 2; string email = 3;}

Example 2: Parse Protobuf data using a template from advanced parameters.
- Raw log
```
{
    "data": "Cgl4aWFvIG1pbmcQARoOMTIzMTIzQDEyMy5jb20="
}
```
- Transformation rule
```
e_set("data",
      protobuf_decoding(
          v("data"),
          res_local("protocol"),
          "base64",
          "Person"
      )
)
```
- Advanced parameter settings
  
  In the Create Data Transformation Job panel, configure advanced parameters. For this example, set key to protocol and value to syntax = "proto3"; message Person { string name = 1; int32 id = 2; string email = 3;}. For more information, see Create a data transformation job.
- Result
```
data:{
  "name": "xiao ming",
  "id": 1,
  "email": "123123@123.com"
}
```

Example 3: Parse Protobuf data using a template from an OSS file.

Raw log

{
    "data": "Cgl4aWFvIG1pbmcQARoOMTIzMTIzQDEyMy5jb20="
}

OSS file content

syntax = "proto3"; message Person {  string name = 1;  int32 id = 2; string email = 3;}

Transformation rule

e_set(
    "protocol",
    res_oss_file(
        endpoint="http://oss-cn-chengdu.aliyuncs.com",
        ak_id="your_ak_id",
        ak_key="your_ak_key",
        bucket="test-protobuf",
        file="test.json",
    ),
)
e_set("data", protobuf_decoding(v("data"), v("protocol"), "base64", "Person"))

Result

data:{
  "name": "xiao ming",
  "id": 1,
  "email": "123123@123.com"
}
protocol:syntax = "proto3"; message Person {  string name = 1;  int32 id = 2; string email = 3;}

jwt_encoding

Encodes JSON data as a JSON Web Token (JWT).

Note

JSON Web Token (JWT) is an open standard (RFC 7519) that provides a compact and self-contained method for securely transmitting information between parties as a JSON object. Its digital signature makes the information verifiable and trustworthy. A JWT can be signed with a secret (using an HMAC algorithm) or with a public and private key pair (using RSA or ECDSA). For more information, see JWT Introduction.

Syntax

jwt_encoding(payload, key, algorithms="HS256", headers=None)

Parameters

Parameter	Type	Required	Description
payload	JSON	Yes	The JSON object to use as the JWT payload. The JWT standard defines the following seven registered claims. You can also include custom claims: iss: The token issuer. exp: The token's expiration time. sub: The token subject. aud: The token audience. nbf: The time before which the token is invalid. iat: The time at which the token was issued. jti: A unique identifier for the token. The following is an example of a JSON payload: `{ "iss": "localhost", "sub": "name", "aud": "user", "address": { "street": "street number", "city": "hangzhou", "country": "china" } }`
key	String	Yes	The key for signing the JWT. The required key depends on the selected algorithm: asymmetric encryption algorithm: A PEM-encoded private key. symmetric encryption algorithm: A raw secret key.
algorithms	String	No	The signing algorithm. The default value is HS256. Supported algorithms include: `HS256,HS384,HS512,ES256,ES256K,ES384,ES512,RS256,RS384,RS512,PS256,PS384,PS512,EdDSA`.
headers	JSON	No	The JWT header fields. The default value is `{ "typ": "JWT", "alg": "HS256" }` .

Response

The encoded JWT as a string.

Examples

Example 1: Encode the data log field.

Raw log
```
data: {"some": "payload"}
```

Transformation rule

e_set("jwt_token", jwt_encoding(v("data"),"secret", algorithms="HS256"))

Result

data:{"some": "payload"}
jwt_token:eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Joh1R2dYzkRvDkqv3sygm5YyK8Gi4ShZqbhK2gxcs2U

Example 2: Encode the data log field using an asymmetric encryption algorithm.

Raw log
```
data: {"some": "payload"}
```

Transformation rule

e_set(
    "jwt_token",
    jwt_encoding(
        v("data"),
        "-----BEGIN RSA PRIVATE KEY-----\nMIICYQIBAAKBgQC1iaUr5cgShCn0127+w14XN297q/IviaewIeIJsKTZF1hBPsLn\nNIPsnqtQ9DFbjIyqyZvdmQFDJCSLpXaVc648yepnFDKbOfs3r+K4Crnpo2SuZmNV\nNDVEi4pECXlBz810zJY1wqVArM7qGAyCcRLBprwXB6wfEhk3CAP3c29+pwIDAQAB\nAoGAARo65I9arbIbxx7fz7BEDAQMK0YaDGvbltg91S07cw4PPSYybNEG1BMKm01A\nV3v9BrR+u9PIDC5WAnsYwiODqEoSyk8OwO1E2kWA6+MNclYYfVjaJeiRJ5PzCud/\niUObonptRzxuTng+u1oGuX7QwUhwGJdXVBUAtJFYwXR2qVECRQC5S+6vdFESRLSX\n7yBZVM6+49lZcdehMv0HwT17UseLvWcjeSbiogvv02HbYilrW9ZydKsixAWP5w/U\nS3L34CS811VcDQI9APrOiL7c1xg5fX8wAWv2d+e+MfZoB3ohb8671W3pmp3JVnjY\ntzhoYNNQmnmRQQWf7n3J63MQz4sYYNn0gwJEE/pl37Dw1MFnn0H/AOKt79LtKkGl\n+BFhSqbBFDzWmvBu4Fo9oQ3Lr63gzSCGSrb6JhkCIptz5hIJmOARozwdeebVozkC\nPQDwOqVmU3c/P8nB6oRiGditw0Jt1yTaSW6jkOyUc73iRngqFkIgqHGd1kWwDX4/\nWfoAyEhalY6Fh5s1COsCRQCy4b3hnQws5zz/gmGNnoyxn9N+A09ySaFtn2WkUlrR\nZ6DwoJz+n6EgjLY8z6ZQyv342iobO5zKkZHFvO9QYGqk7y8lJA==\n-----END RSA PRIVATE KEY-----\n",
        algorithms="RS256",
    ),
)

Result

data:{"some": "payload"}
jwt_token:eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Ewwls5YPuJCmAR3XR2tcptOLrH83wVCzmUaUpGMzMLcPknRrIvbDmFGlNlQha-PMx0jsxt3t1oxpz7P3z3SR9o4qyWusAb99UG_Jn8oP8W0a5GKSy4UEJB0xgpVvJl5F2JaIPeUSHpV0VeS2WAsGSBBSAaOMkrc-8uie-H4J9M0

Example 3: Encode the data log field with the HS256 algorithm and custom headers.

Raw log
```
data: {"some": "payload"}
```

Transformation rule

e_set(
    "jwt_token",
    jwt_encoding(
        v("data"),
        "secret",
        algorithms="HS256",
        headers={"kid": "230498151c214b788dd97f22b85410a5"},
    ),
)

Result

data:{"some": "payload"}
jwt_token:eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiIsImtpZCI6IjIzMDQ5ODE1MWMyMTRiNzg4ZGQ5N2YyMmI4NTQxMGE1In0.eyJzb21lIjoicGF5bG9hZCJ9.gdQ884yjlnLnIrYjfQaClE6rJC2x8v2OP2s_eXOLhZA

Example 4: Encode a payload that contains custom JWT fields.

Raw log

data: {"some": "payload", "iss": 9, "sub": "name", "nbf": 123, "iat": "22"}

Transformation rule

e_set("jwt_token", jwt_encoding(v("data"),"secret"))

Result

data:{"some": "payload", "iss": 9, "sub": "name", "nbf": 123, "iat": "22"}
jwt_token:eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCIsImlzcyI6OSwic3ViIjoibmFtZSIsIm5iZiI6MTIzLCJpYXQiOiIyMiJ9.c6MxZHdXsgASchQ1Mdqj208NO_3rNGjYnvo6c7HNAyk

jwt_decoding

Decodes a token into a JSON object according to the JSON Web Token (JWT) standard.

Note

JSON Web Token (JWT) is an open standard (RFC 7519) that provides a compact and self-contained method for securely transmitting information between parties as a JSON object. Its digital signature makes the information verifiable and trustworthy. A JWT can be signed with a secret (using an HMAC algorithm) or with a public and private key pair (using RSA or ECDSA). For more information, see JWT Introduction.

Syntax

jwt_decoding(jwt_payload, key="", algorithms=None, options=None, audience=None, issuer=None, leeway=0)

Parameters

Parameter	Type	Required	Description
jwt_payload	String	Yes	The token string to decode.
key	String	Yes	The key used to verify the token's signature. This key must match the one used for encoding. For symmetric algorithms, this is the secret. For asymmetric algorithms, this is the PEM-encoded public key.
algorithms	List	No	A list of allowed signature algorithms for verification, such as `HS256`.
options	JSON	No	A JSON object with advanced decoding and validation options. These options include: `verify_signature`: Specifies whether to verify the JWT signature. The default value is `True`. `require`: A list of claims required in the token's payload. By default, no claims are required. For example, `["exp", "iat", "nbf"]`. `verify_aud`: Specifies whether to validate the `aud` claim. The default value is `True`. `verify_iss`: Specifies whether to validate the `iss` claim. The default value is `True`. `verify_exp`: Specifies whether to validate the `exp` claim. The default value is `True`. `verify_iat`: Specifies whether to validate the `iat` claim. The default value is `True`. `verify_nbf`: Specifies whether to validate the `nbf` claim. The default value is `True`. For example, `{"require": ["aud"], "verify_aud": True}`.
audience	String/List	No	The expected audience, used to validate the `aud` claim in the token. This parameter is required if you set `"verify_aud": True` in the `options` parameter or if `"aud"` is included in the `require` list.
issuer	String	No	The expected issuer, used to validate the `iss` claim in the token. This parameter is required if you set `"verify_iss": True` in the `options` parameter or if `"iss"` is included in the `require` list.
leeway	float	No	A time margin in seconds to account for clock skew when validating time-based claims such as `exp`, `nbf`, and `iat`.

Response

Returns the decoded payload as a JSON object.

Examples

Example 1: Decode the value of the data field and use the HS256 signature algorithm for verification.

Raw log

data:eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Joh1R2dYzkRvDkqv3sygm5YyK8Gi4ShZqbhK2gxcs2U

Transformation rule

e_set("data_decoded", jwt_decoding(v("data"), "secret", algorithms="HS256"))

Result
```
data_decoded: {"some": "payload"}
```

Example 2: Use a public key to decode the value of the data field.

Raw log

data: "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Ewwls5YPuJCmAR3XR2tcptOLrH83wVCzmUaUpGMzMLcPknRrIvbDmFGlNlQha-PMx0jsxt3t1oxpz7P3z3SR9o4qyWusAb99UG_Jn8oP8W0a5GKSy4UEJB0xgpVvJl5F2JaIPeUSHpV0VeS2WAsGSBBSAaOMkrc-8uie-H4J9M0"

Transformation rule

e_set(
    "data_decoded",
    jwt_decoding(
        v("data"),
        "-----BEGIN RSA PUBLIC KEY-----\nMIGJAoGBALWJpSvlyBKEKfTXbv7DXhc3b3ur8i+Jp7Ah4gmwpNkXWEE+wuc0g+ye\nq1D0MVuMjKrJm92ZAUMkJIuldpVzrjzJ6mcUMps5+zev4rgKuemjZK5mY1U0NUSL\nikQJeUHPzXTMljXCpUCszuoYDIJxEsGmvBcHrB8SGTcIA/dzb36nAgMBAAE=\n-----END RSA PUBLIC KEY-----\n",
        algorithms="RS256",
    ),
)

Result

data_decoded: {"some": "payload"}
data:"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCJ9.Ewwls5YPuJCmAR3XR2tcptOLrH83wVCzmUaUpGMzMLcPknRrIvbDmFGlNlQha-PMx0jsxt3t1oxpz7P3z3SR9o4qyWusAb99UG_Jn8oP8W0a5GKSy4UEJB0xgpVvJl5F2JaIPeUSHpV0VeS2WAsGSBBSAaOMkrc-8uie-H4J9M0"

Example 3: Decode the value of the data field using the default decoding and validation options.

Raw log

data: "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCIsImlzcyI6OSwic3ViIjoibmFtZSIsIm5iZiI6MTIzLCJpYXQiOiIyMiJ9.DzvqhJd0PrTFk6eeASGZxOoDtrLBt_H3xC7CqOATRRw"

Transformation rule

e_set(
    "data_decoded", jwt_decoding(v("data"), "secret", algorithms="HS256", options=None)
)

Result

data_decoded:{
        "some": "payload",
        "iss": 9,
        "sub": "name",
        "nbf": 123,
        "iat": "22"
    }
data:"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCIsImlzcyI6OSwic3ViIjoibmFtZSIsIm5iZiI6MTIzLCJpYXQiOiIyMiJ9.DzvqhJd0PrTFk6eeASGZxOoDtrLBt_H3xC7CqOATRRw"

Example 4: Decode the value of the data field using custom decoding and validation options.

Raw log

data: "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCIsImlzcyI6Im5hbWUifQ.XwT9jqwofcdSP6olidbiYPC6CnZd36OEqCHZmGmooWM"

Transformation rule

e_set(
    "data_decoded",
    jwt_decoding(
        v("data"),
        "secret",
        algorithms="HS256",
        options={"require": ["iss"], "verify_iss": True},
        issuer="name",
    ),
)

Result

data_decoded:{"some": "payload", "iss": "name"}
data: "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzb21lIjoicGF5bG9hZCIsImlzcyI6Im5hbWUifQ.XwT9jqwofcdSP6olidbiYPC6CnZd36OEqCHZmGmooWM"

hashids_encoding

Encodes one or more integers into a short, unique, and non-sequential string using the Hashids library.

Syntax

hashids_encoding(value, salt="", min_length=0)

Parameters

Parameter	Type	Required	Description
value	integer, list, or tuple	Yes	The integer or collection of integers to encode. All integers must be non-negative.
salt	string	No	A string that randomizes the output. The default value is an empty string.
min_length	integer	No	The minimum length of the generated hash value. If the output is shorter, it is padded to this length. The default value is 0.

Response

Returns the encoded string.

Examples

Example 1: Encode an integer with default settings.
- Raw log
```
content:test
```
- Transformation rule
```
e_set("hashid", hashids_encoding(123))
```
- Result
```
content:test
hashid:Mj3
```

Example 2: Encode a list of integers with a custom salt.

Raw log
```
content:test
```

Transformation rule

e_set("hashid", hashids_encoding([123, 456], salt="test"))

Result
```
content:test
hashid:bpBHYO
```

Example 3: Encode a list of integers with a custom salt and minimum length.

Raw log
```
content:test
```

Transformation rule

e_set("hashid", hashids_encoding([123, 456], salt="test", min_length=32))

Result

content:test
hashid:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g

Example 4: Encode a tuple of integers with a custom salt and minimum length.

Raw log
```
content:test
```

Transformation rule

e_set("hashid", hashids_encoding((123, 456), salt="test", min_length=32))

Result

content:test
hashid:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g

hashids_decoding

Decodes data encoded with the Hashids library.

Syntax

hashids_decoding(hashid, salt="", min_length=0)

Parameters

Parameter	Type	Required	Description
hashid	String	Yes	The string to decode.
salt	String	No	The salt used for encoding. This value must match the salt used in the `hashids_encoding` function. The default value is an empty string.
min_length	Number	No	The minimum length used for encoding. This value must match the `min_length` used in the `hashids_encoding` function. The default value is 0.

Response

Returns an array of decoded integers.

Examples

Example 1: Decode a hash with default settings.
- raw log:
```
content:Mj3
```
- transformation rule:
```
e_set("value", hashids_decoding(v("content")))
```
- result:
```
content:Mj3
value:[123]
```

Example 2: Decode a hash with a custom salt.

raw log:
```
content:bpBHYO
```

transformation rule:

e_set("value", hashids_decoding(v("content"), salt="test"))

result:
```
content:bpBHYO
value:[123, 456]
```

Example 3: Decode a hash with a custom salt and minimum length.

raw log:

content:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g

transformation rule:

e_set("value", hashids_decoding(v("content"), salt="test", min_length=32))

result:

content:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g
value:[123, 456]

Example 4: Decode a hash with a custom salt and minimum length.

raw log:

content:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g

transformation rule:

e_set("value", hashids_decoding(v("content"), salt="test", min_length=32))

result:

content:O6jN0Z7VARqDzbpBHYOakeowE5Xnr41g
value:[123, 456]

gzip_compress

Compresses data using gzip and encodes the result.

Syntax

gzip_compress(data, compresslevel=6, to_format="base64", encoding="utf-8")

Parameters

Parameter	Type	Required	Description
data	String	Yes	The data to compress.
compresslevel	Int	No	The compression level, specified as an integer from 0 to 9. The default value is 6. 1: Fastest compression speed, lowest compression ratio. 9: Slowest compression speed, highest compression ratio. 0: No compression.
to_format	String	No	The encoding format for the compressed data. Valid values are `base64` and `hex`.
encoding	String	No	The encoding format of the raw data. The default value is `utf-8`. For other encoding formats, see Standard encoding formats.

Response

Returns the compressed and encoded data as a string.

Examples

Example 1: Compress a log field and encode it in Base64.

Raw log

content: I always look forward to my holidays whether I travel or stay at home.

Transformation rule

e_set("base64_encode_gzip_compress",gzip_compress(v("content"),to_format="base64"))

Result

content: I always look forward to my holidays whether I travel or stay at home.
base64_encode_gzip_compress: H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA

Example 2: Compress a log field and encode it in hex.

Raw log

content: H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA

Transformation rule

e_set("hex_encode_gzip_compress", gzip_compress(v("content"), to_format="hex"))

Result

content:H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA
hex_encode_gzip_compress:1f8b08004a478c6202ff0dc1dd0e43301800d047aa65156e3ff52f4a2ba17649c43255194d4a9f9e73527c64007e2e2426e81485c35628c1c42616535079bc6405e5d1e92ef009b59c906786a879efe1c50fb55d6c5de44cb717b2dae6d4f103f8feecbf4f88a2a441bae618c679575d9bc0e306907876806c000000

Gzip_decompress

Decompresses Gzip-compressed data.

Syntax

gzip_decompress(data, from_format="base64", encoding="utf-8")

Parameters

Parameter	Type	Required	Description
data	Any	Yes	The data to decompress.
from_format	String	No	The encoding of the input data. Supported values are `base64` and `hex`.
encoding	String	No	The character encoding of the original, uncompressed data. Defaults to `utf-8`. For other supported formats, see Standard encoding formats.

Response

The decompressed data.

Examples

Example 1: Decompress a Base64-encoded log field.

Raw log

content: H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA

Transformation rule

e_set("gzip_decompress",gzip_decompress(v("content"),from_format="base64"))

Result

content: H4sIAA8JXl4C/xXK0QmAMAwFwFXeBO7RMQKNREx5kAZDtle/7wbES3rDyRsnoyQmklgNo1/ztzJN08BAhjzqYGCnNCS/tPR4AcgrnWVGAAAA
gzip_decompress: I always look forward to my holidays whether I travel or stay at home.

Example 2: Decompress a hex-encoded log field.

Raw log

content:1f8b0800bff8856202ff0dc1dd0e43301800d047aa65156e3ff52f4a2ba17649c43255194d4a9f9e73527c64007e2e2426e81485c35628c1c42616535079bc6405e5d1e92ef009b59c906786a879efe1c50fb55d6c5de44cb717b2dae6d4f103f8feecbf4f88a2a441bae618c679575d9bc0e306907876806c000000

Transformation rule

e_set("gzip_decompress", gzip_decompress(v("content"), from_format="hex"))

Result

content:1f8b0800bff8856202ff0dc1dd0e43301800d047aa65156e3ff52f4a2ba17649c43255194d4a9f9e73527c64007e2e2426e81485c35628c1c42616535079bc6405e5d1e92ef009b59c906786a879efe1c50fb55d6c5de44cb717b2dae6d4f103f8feecbf4f88a2a441bae618c679575d9bc0e306907876806c000000
gzip_decompress: I always look forward to my holidays whether I travel or stay at home.

zlib_compress

Compresses and encodes data.

Syntax

zlib_compress(data, compresslevel=6, to_format="base64", encoding="utf-8")

Parameters

Parameter	Type	Required	Description
data	String	Yes	The data to compress.
compresslevel	Int	No	The compression level, an integer from 0 to 9. The default is 6. 1: Highest compression speed and lowest compression ratio. 9: Lowest compression speed and highest compression ratio. 0: No compression.
to_format	String	No	The encoding format for the compressed data. Currently, only `base64` is supported.
encoding	String	No	The character encoding of the original, uncompressed data. Default: `utf-8`. For a list of other supported formats, see Standard encoding formats.

Response

Returns the compressed and encoded string.

Examples

Raw log

content: I always look forward to my holidays whether I travel or stay at home.

Transformation rule

e_set("zlib_compress", zlib_compress(v("content"), to_format="base64"))

Transformation result

zlib_compress: "eJwVytEJgDAMBcBV3gTu0TECjURMeZAGQ7ZXv+8GxEt6w8kbJ6MkJpJYDaNf87cyTdPAQIY86mBgpzQkv7T0eAGNshln"
content: "I always look forward to my holidays whether I travel or stay at home."

zlib_decompress

Decompresses zlib-compressed data.

Syntax

zlib_decompress(data, from_format="base64", encoding="utf-8")

Parameters

Parameter	Type	Required	Description
data	String	Yes	The compressed data to decompress.
from_format	String	No	Specifies the encoding format of the input data. Currently, only `base64` is supported.
encoding	String	No	Specifies the character encoding for the original, uncompressed data. The default is `utf-8`. For a list of other supported formats, see Standard encoding formats.

Response

Returns the decompressed string.

Examples

Raw log

content: "eJwVytEJgDAMBcBV3gTu0TECjURMeZAGQ7ZXv+8GxEt6w8kbJ6MkJpJYDaNf87cyTdPAQIY86mBgpzQkv7T0eAGNshln"

Transformation rule

e_set("zlib_decompress", zlib_decompress(v("content"), from_format="base64"))

Result

content: "eJwVytEJgDAMBcBV3gTu0TECjURMeZAGQ7ZXv+8GxEt6w8kbJ6MkJpJYDaNf87cyTdPAQIY86mBgpzQkv7T0eAGNshln"
zlib_decompress: "I always look forward to my holidays whether I travel or stay at home."

aes_encrypt

Encrypts data using the Advanced Encryption Standard (AES), one of the most common symmetric encryption algorithms, to effectively improve data security.

Syntax

aes_encrypt(data, key, mode, pad_style, pad_block, input_format, input_encoding, output_format, iv)

Parameters

Parameter	Type	Required	Description
data	String	Yes	The data to encrypt.
key	String	Yes	The key for encrypting the data.
mode	String	No	The AES encryption mode. Valid values: CBC (default): Cipher Block Chaining ECB: Electronic Code Book CFB: Cipher Feedback OFB: Output Feedback CTR: Counter OPENPGP
pad_style	String	No	The padding mode. The default value is `pkcs7`. Valid values: `iso7816`, `x923`, and `pkcs7`.
input_format	String	No	The input character format. The default value is `raw`. Valid values: `raw`: Raw bytes `hex`: Hexadecimal format `base64`: Base64 encoding format
input_encoding	String	No	The character encoding of the input data. This parameter is required only if `input_format` is set to `raw`. The default value is `utf-8`.
output_format	String	No	The output character format. The default value is `hex`. Valid values: `raw`: Raw bytes `hex`: Hexadecimal format `base64`: Base64 encoding format
iv	Bytes	No	The initialization vector (IV) for encryption.

Response

Returns the encrypted string.

Examples

Example 1

Raw log
```
"test": "aliyuntest"
```

Transformation rule

e_set('result',aes_encrypt(v("test"), "qwertyuiopasdfgd", iv=b"xxywosjdapdiawdk", output_format="base64"))

Result
```
"result": "gXIqu0cBBtZHQxJBK8GLeA=="
```

Example 2

Raw log
```
"test": "aliyuntest"
```

Transformation rule

e_set('result',aes_encrypt(v("test"), "qwertyuiopasdfgh", iv=b"ywisnjaduaqibdqi", mode="OFB"))

Result
```
"result": "5cac3e9e1c42f713dc6d"
```

aes_decrypt

Decrypts AES-encrypted data.

Syntax

aes_decrypt(data, key, mode, pad_style, input_format, input_encoding, output_format, iv, output_encoding)

Parameters

Parameter	Type	Required	Description
data	String	Yes	The encrypted data.
key	String	Yes	The decryption key.
mode	String	No	The AES decryption mode. The supported values are: CBC (default): Cipher Block Chaining ECB: Electronic Code Book CFB: Cipher Feedback OFB: Output Feedback CTR: Counter OPENPGP
pad_style	String	No	The padding mode. The default value is `pkcs7`. Supported values include `iso7816`, `x923`, and `pkcs7`.
input_format	String	No	The input format. The default value is `hex`. Supported values include: `raw`: A raw byte string. `hex`: Hexadecimal format. `base64`: Base64 encoding format.
input_encoding	String	No	The character encoding format. This parameter is required only if `input_format` is set to `raw`. The default value is `utf-8`.
output_format	String	No	The output format. The default value is `raw`. Supported values include: `raw`: A raw byte string. `hex`: Hexadecimal format. `base64`: Base64 encoding format.
iv	Bytes	No	The initialization vector (IV) for decryption.
output_encoding	String	No	The character encoding for the output. The default value is `None`.

Response

The decrypted string.

Examples

Example 1

Raw log
```
"test": "gXIqu0cBBtZHQxJBK8GLeA=="
```

Transformation rule

e_set('result', aes_decrypt(v("test"), "qwertyuiopasdfgd", iv=b"xxywosjdapdiawdk", input_format="base64"))

Transformation result
```
"result": "aliyuntest"
```

Example 2

Raw log
```
"test": "5cac3e9e1c42f713dc6d"
```

Transformation rule

   e_set('result', aes_decrypt(v("test"), "qwertyuiopasdfgh", iv=b"ywisnjaduaqibdqi", mode="OFB"))

Transformation result
```
"result": "aliyuntest"
```

md5_encoding

Calculates the MD5 hash of a string.

Syntax
```
md5_encoding(value, format="hex")
```

Parameters

Parameter	Type	Required	Description
value	String	Yes	The string to hash.
format	String	No	Specifies the output format of the hash. Valid values are `binary` and `hex`. The default value is `hex`.

Response

Returns the MD5 hash as a string.

Examples

Example 1

Raw log
```
str : GeeksforGeeks
```

Transformation rule

e_set("str_md5_en",md5_encoding(v("str")))

Result

str : GeeksforGeeks
str_md5_en : f1e069787ece74531d112559945c6871

Example 2

Raw log
```
str : GeeksforGeeks
```

Transformation rule

e_set("str_md5_en",base64_encoding(md5_encoding(v("str"), format="binary")))

Result

str : GeeksforGeeks
str_md5_en : 8eBpeH7OdFMdESVZlFxocQ==

sha1_encoding

Calculates the SHA hash of a string. The default algorithm is SHA-1.

Syntax
```
sha1_encoding(value, format=None)
```

Parameters

Parameter	Type	Required	Description
value	String	Yes	The string to hash.
format	String	No	Specifies the hash algorithm. Supported values are `SHA1`, `SHA224`, `SHA256`, `SHA384`, and `SHA512`. The default value is `SHA1`.

Response

The resulting hash string.

Examples

Raw log
```
str : GeeksforGeeks
```

Transformation rule

e_set("str_sha1",sha1_encoding(v("str")))
e_set("str_sha512",sha1_encoding(v("str"),format='SHA512'))
e_set("str_sha224",sha1_encoding(v("str"),format='SHA224'))
e_set("str_sha384",sha1_encoding(v("str"),format='SHA384'))
e_set("str_sha256",sha1_encoding(v("str"),format='SHA256'))

Result

str : GeeksforGeeks
str_sha1 : 4175a37afd561152fb60c305d4fa6026b7e79856
str_sha512 : 0d8fb9370a5bf7b892be4865cdf8b658a82209624e33ed71cae353b0df254a75db63d1baa35ad99f26f1b399c31f3c666a7fc67ecef3bdcdb7d60e8ada90b722
str_sha224 : 173994f309f727ca939bb185086cd7b36e66141c9e52ba0bdcfd145d
str_sha384 : d1e67b8819b009ec7929933b6fc1928dd64b5df31bcde6381b9d3f90488d253240490460c0a5a1a873da8236c12ef9b3
str_sha256 : f6071725e7ddeb434fb6b32b8ec4a2b14dd7db0d785347b2fb48f9975126178f

Crc32_encoding

Calculates the CRC-32 value of the input data.

Syntax

crc32_encoding(data, input_format="raw", input_encoding="utf-8")

Parameters

Parameter	Type	Required	Description
data	String	Yes	The data for the CRC-32 calculation.
input_format	String	No	The format of the input string. The default value is `raw`. Valid values: `raw`: A raw byte string. `hex`: A hexadecimal string. `base64`: A Base64-encoded string.
input_encoding	String	No	The character encoding of the input string. This parameter is used only when `input_format` is set to `raw`. The default value is `utf-8`.

Response

Returns the CRC-32 value of the input data as an integer.

Examples

Example 1: Calculate the CRC-32 value for the test field.
- Raw Log
```
test: aliyuntest
```
- Transformation Rule
```
e_set("str_crc32", crc32_encoding(v("test")))
```
- Transformation Result
```
str_crc32:1434103726
test:aliyuntest
```

Example 2: Concatenate the test1 and test2 fields, compute the MD5 hash, and then calculate the CRC-32 value of the resulting hash.

Raw Log
```
test1: test1
test2: test2
```

Transformation Rule

e_set(
    "str_crc32",
    crc32_encoding(
        md5_encoding(str_join("+", v("test1"), v("test2")), format="binary")
    ),
)

Transformation Result

str_crc32:369733261
test1:test1
test2:test2

Example 3: Calculate the CRC-32 value for the test field, which contains a Base64-encoded string.

Raw Log
```
test: Taloz+e+PzP3NltrEXiCig==
```

Transformation Rule

e_set("str_crc32", crc32_encoding(v("test"), input_format="base64"))

Transformation Result

str_crc32:1093789404
test:Taloz+e+PzP3NltrEXiCig==

Functions

str_encode

Syntax

Parameters

Response

Examples

str_decode

Syntax

Parameters

Response

Examples

base64_encoding

Syntax

Parameters

Response

Examples

base64_decoding

Syntax

Parameters

Response

Examples

html_encoding

Syntax

Parameters

Response

Examples

html_decoding

Syntax

Parameters

Response

Examples

url_encoding

Syntax

Parameters

Response

Examples

URL decoding

Syntax

Parameters

Response

Examples

protobuf_decoding

Syntax

Parameters

Response

Examples

jwt_encoding

Syntax

Parameters

Response

Examples

jwt_decoding

Syntax

Parameters

Response

Examples

hashids_encoding

Syntax

Parameters

Response

Examples

hashids_decoding

Syntax

Parameters

Response

Examples

gzip_compress

Syntax

Parameters

Response

Examples

Gzip_decompress

Syntax

Parameters

Response

Examples

zlib_compress

Syntax

Parameters

Response