Data delay validation rules-阿里云帮助中心

Data delay validation checks time gaps between adjacent records within the same data scope. Specify the ordering field, time field, and maximum interval, then set a pass condition for the count of records that exceed the interval. The examples provide system template and assertion configurations.

Scope and metric

The SYSTEM:field:delay_count:fixed template orders records by orderBy, calculates the time difference between each record and its preceding record, and counts records whose gaps exceed delay.maxInterval. This metric describes gaps between adjacent records. It does not measure the difference between the current time and the timestamp of the latest record.

The dataset-level datasets.filter and rule-level filter determine which records participate in the check. Choose an ordering field that determines the business sequence. Duplicate ordering values can make the adjacent record sequence unstable.

The examples are rules fragments for a data quality Spec and use an existing dataset configuration. For a system template, specify templateId, fields, and the template parameters. For an assertion, specify the time field and ordering field in assertion. Choose one form for this check. For the complete Spec structure, see Data quality Spec.

Sample data

Assume that event_id defines the event sequence and event_time stores timestamps as strings. The following table lists records ordered by event_id to illustrate the relationship between adjacent gaps and the maximum interval.

`event_id`	`event_time`
`1`	`2026-01-01 10:00:00`
`2`	`2026-01-01 10:05:00`
`3`	`2026-01-01 10:20:00`

After the records are ordered by event_id, the time gap between record 2 and its preceding record is 5 minutes, and the gap between record 3 and its preceding record is 15 minutes. With a maximum allowed interval of 10 minutes, record 3 is counted as invalid. Record 1 has no preceding record and is not counted as invalid. The invalid record count is therefore 1, which does not satisfy the pass: when = 0 condition. The rule fails the check.

Adjacent record interval configuration

Both configurations order records by event_id, check adjacent gaps in event_time, and require an invalid record count of 0. Both use filter: event_time IS NOT NULL to exclude NULL timestamps, a maximum interval of 10 minutes, and the string time format yyyy-MM-dd HH:mm:ss.

System rule template

rules:
  - templateId: SYSTEM:field:delay_count:fixed
    name: Adjacent events within ten minutes
    fields:
      - event_time
    templateParameters:
      orderBy: event_id
    filter: event_time IS NOT NULL
    delay:
      maxInterval: 10 minutes
      fieldFormat: yyyy-MM-dd HH:mm:ss
    pass:
      - when = 0

Assertion expression

rules:
  - assertion: "delay_count(event_time, order_by=event_id)"
    name: Adjacent events within ten minutes
    filter: event_time IS NOT NULL
    delay:
      maxInterval: 10 minutes
      fieldFormat: yyyy-MM-dd HH:mm:ss
    pass:
      - when = 0

The first argument of delay_count is the time field. The order_by argument specifies the ordering field. The template parameter templateParameters.orderBy and assertion argument order_by specify the same field. Set the maximum interval and string time format in the delay property of the rule.

When comparing or switching configuration forms, keep the dataset scope, rule filter, time field, ordering field, maximum interval, time format, and pass threshold identical. Change delay.maxInterval to adjust the maximum interval. Change pass to adjust the allowed invalid record count.

Parameters

The following table describes the configuration items used in these examples. Dots in parameter names indicate nesting. Use the YAML examples for the corresponding structure.

Parameter	Description	Example value
`templateId`	The system rule template ID for counting gaps between adjacent records.	`SYSTEM:field:delay_count:fixed`
`assertion`	The assertion expression. The first argument specifies the time field, and `order_by` specifies the ordering field. Use this form as an alternative to a system template.	`delay_count(event_time, order_by=event_id)`
`name`	A rule name that describes the validation goal.	`Adjacent events within ten minutes`
`fields`	The list of time fields in the template form. In the assertion form, specify the time field as the first argument.	`event_time`
`templateParameters.orderBy`	The field that determines the adjacent record sequence in the template form. Choose a field that determines the business sequence. In the assertion form, specify this field in `order_by`.	`event_id`
`datasets.filter`	The dataset-level filter that defines the data scope for the rule. These examples use the existing dataset configuration.	Existing configuration
`filter`	The rule-level filter that limits participating records within the dataset scope. The examples exclude NULL timestamps. Records are filtered before adjacency between the remaining records is determined.	`event_time IS NOT NULL`
`delay.maxInterval`	The maximum allowed gap between adjacent timestamps. A gap that exceeds this value contributes to the invalid record count.	`10 minutes`
`delay.fieldFormat`	The parsing format for a string time field. Match the format to the field content. You can omit this parameter for a native time type.	`yyyy-MM-dd HH:mm:ss`
`pass`	The pass condition for the invalid record count. The list item `when = 0` passes the rule when the count is zero.	`when = 0`