DocsICP FilingConsole
官方文档
首页

Event identification

更新时间：
复制 MD 格式
产品详情
我的收藏

Conditional event checks allow you to perform operations on data that meets specific criteria. This improves the reliability of your transformation logic. This topic describes common scenarios and provides examples of best practices for using functions to perform conditional event checks.

Scenario 1: Check if a field exists

  • Raw logs

    a: a_value
b:       // Empty string

  • SLS DSL orchestration

    • Solution 1 (Recommended): Use e_has or e_not_has.

      e_if(e_has("a"),e_set("has_a", true))
e_if(e_has("b"),e_set("has_b", true))
e_if(e_has("c"),e_set("has_c", true))
e_if(e_not_has("a"),e_set("not_has_a", true))
e_if(e_not_has("b"),e_set("not_has_b", true))
e_if(e_not_has("c"),e_set("not_has_c", true))

    • Solution 2: Use e_search.

      e_if(e_search('a: *'),e_set("has_a", true))
e_if(e_search('b: *'), e_set("has_b", true))
e_if(e_search('c: *'), e_set("has_c", true))
e_if(e_search('not a: *'), e_set("not_has_a", true))
e_if(e_search('not b: *'), e_set("not_has_b", true))
e_if(e_search('not c: *'), e_set("not_has_c", true))
      Note

      The e_if function in a transformation rule can be merged into a single statement with the format e_if(condition1, operation1, condition2, operation2). The functions are split into multiple statements in this topic for improved readability.

  • Results

    a:a_value
b:    // Empty string
has_a: true
has_b: true
not_has_c: true

Scenario 2: Check if a field exists and is not empty

  • Raw logs

    a: a_value
b:     // Empty string

  • SLS DSL orchestration

    • Solution 1 (Recommended): Use the field value function v

      e_if(v("a"), e_set("not_empty_a", true))
e_if(v("b"), e_set("not_empty_b", true))
e_if(v("c"), e_set("not_empty_c", true))
      Note

      The field value function v is automatically converted to a Bool value. It returns true if the field exists and its value is not empty. Otherwise, it returns false.

    • Solution 2: Use e_search

      # At least one character
e_if(e_search('a: "?"'), e_set("not_empty_a", true))
e_if(e_search('b: "?"'), e_set("not_empty_b", true))
e_if(e_search('c: "?"'), e_set("not_empty_c", true))

# Regular expression
e_if(e_search('a~=".+"'), e_set("not_empty_a", true))
e_if(e_search('b~=".+"'), e_set("not_empty_b", true))
e_if(e_search('c~=".+"'), e_set("not_empty_c", true))

# Exists and is not empty
e_if(e_search('a: * and not a==""'), e_set("not_empty_a", true))
e_if(e_search('b: * and not b==""'), e_set("not_empty_b", true))
e_if(e_search('c: * and not c==""'), e_set("not_empty_b", true))

  • Results

    a: a_value
b:     // Empty string
not_empty_a: true

Scenario 3: Check if a field exists and is empty

  • Raw logs

    a: a_value
b:       // Empty string

  • SLS DSL orchestration

    • Solution 1 (Recommended): Use the field value function v

      e_if(op_and(e_has("a"), op_not(v("a"))), e_set("empty_a", true))
e_if(op_and(e_has("b"), op_not(v("b"))), e_set("empty_b", true))
e_if(op_and(e_has("c"), op_not(v("c"))), e_set("empty_c", true))

# Incorrect solution
e_if(op_not(v("a")), e_set("empty_a", true))
e_if(op_not(v("b")), e_set("empty_b", true))
e_if(op_not(v("c")), e_set("empty_c", true))
      Note

      The field value function v is automatically converted to the Bool value true if the corresponding field exists and its value is not empty. Otherwise, it is converted to false. If a value does not exist, the return value is true, and op_not(None) also returns true.

    • Solution 2: Use e_search

      e_if(e_search('a==""'), e_set("empty_a", true))
e_if(e_search('b==""'), e_set("empty_b", true))
e_if(e_search('c==""'), e_set("empty_c", true))

# Incorrect call
e_if(e_search('a:""'), e_set("empty_a", true))
e_if(e_search('b:""'), e_set("empty_b", true))
      Note

      Because the function e_search performs a partial query, the empty string expression a: "" in the preceding faulty call evaluates to true as long as the field exists, regardless of its value.

  • Processing results

    a: a_value
b:       // Empty string
empty_b: true

Scenario 4: Perform logical queries based on field values

  • Raw logs

    "Log 1"
http_host: example.com
status: 200
request_method: GET
scheme: https
header_length: 700
body_length: 1200

"Log 2"
http_host: example.org
status: 200
request_method: POST
scheme: https
header_length: 100
body_length: 800

"Log 3"
http_host: example.net
status: 200
request_method: GET
scheme:  http
header_length: 700
body_length: 800

"Log 4"
http_host: aliyundoc.com
status: 404
request_method: GET
scheme: https
header_length: 100
body_length: 300

  • Requirement 1: Processing

    For all log events where the status field is 200, add a new field named type with the value normal.

    • SLS DSL orchestration

      e_if(e_match("status", "200"), e_set("type", "normal"))
or
e_if(e_search('status==200'), e_set("type", "normal"))
      Note

      • For simple scenarios, you can use either orchestration.

      • You can use status:200 to check if the `status` field contains `200`. However, using status==200 provides a more precise match.

    • Transformation result

      "Log 1"
type: normal
http_host: example.com
status: 200
request_method: GET
scheme: https
header_length: 700
body_length: 1200

"Log 2"
type: normal
http_host: example.org
status: 200
request_method: POST
scheme: https
header_length: 100
body_length: 800

"Log 3"
type: normal
http_host: example.net
status: 200
request_method: GET
scheme: http
header_length: 700
body_length: 800

"Log 4"
http_host: aliyundoc.com
status: 404
request_method: GET
scheme: https
header_length: 100
body_length: 300

  • Processing Requirement 2

    For all log events where the status field is 200, the request_method field is GET, and the scheme field is https, add a new field named type with the value normal.

    • SLS DSL orchestration

      e_if(e_search('status==200 and request_method==GET and scheme==https'), e_set("type", "normal"))
or
e_if(e_match_all("status", "200", "request_method", "GET", "scheme", "https"), e_set("type", "normal"))
      Note

      • To match conditions for multiple fields, use e_search or e_match_all. The e_search function is simpler to use.

      • The query status:200 checks if the status field contains 200. For a more precise match, use status==200.

    • Processing results

      "Log 1"
type: normal
http_host: example.com
status: 200
request_method: GET
scheme: https
header_length: 700
body_length: 1200

"Log 2"
http_host: example.org
status: 200
request_method: POST
scheme: https
header_length: 100
body_length: 800

"Log 3"
http_host: example.net
status: 200
request_method: GET
scheme: http
header_length: 700
body_length: 800

"Log 4"
http_host: aliyundoc.com
status: 404
request_method: GET
scheme: https
header_length: 100
body_length: 300

  • Requirement 3: Processing

    For all log events where the status field is 200, or the request_method field is GET, or the scheme field is https, add a new field named type with the value normal.

    • SLS DSL orchestration

      e_if(e_search('status==200 or request_method==GET or scheme==https'), e_set("type", "normal"))
or
e_if(e_match_any("status", "200", "request_method", "GET", "scheme", "https"), e_set("type", "normal"))

    • Transformation result

      "Log 1"
type: normal
http_host: example.com
status: 200
request_method: GET
scheme: https
header_length: 700
body_length: 100

"Log 2"
type: normal
http_host: example.org
status: 200
request_method: POST
scheme: https
header_length: 100
body_length: 800

"Log 3"
type: normal
http_host: example.net
status: 200
request_method: GET
scheme: http
header_length: 700
body_length: 800

"Log 4"
type: normal
http_host: aliyundoc.com
status: 404
request_method: GET
scheme: https
header_length: 100
body_length: 1300

  • Transformation requirement 4

    For all log events where the status field is 200, the request_method field is GET, and the sum of the header_length and body_length fields is less than or equal to 1000, add a new field named type with the value normal.

    • SLS DSL orchestration

      e_if(op_and(e_search('status: 200 and request_method: GET'), op_le(op_sum(v("header_length"), v("body_length")), 1000)), e_set("type", "normal"))
      Note

      For complex logical scenarios, you can combine e_search with other expression functions to build the SLS DSL orchestration.

    • Transformation result

      "Log 1"
type: normal
http_host: example.com
status: 200
request_method: GET
scheme: https
header_length: 700
body_length: 100

"Log 2"
http_host: example.org
status: 200
request_method: POST
scheme: https
header_length: 100
body_length: 800

"Log 3"
http_host: example.net
status: 200
request_method: GET
scheme: http
header_length: 700
body_length: 800

"Log 4"
http_host: aliyundoc.com
status: 404
request_method: GET
scheme: https
header_length: 100
body_length: 1300
Previous：Cleanse data by calling functionsNext：Handle datetimes