Configure noise reduction for data quality rules

Limitations

• Only an Alibaba Cloud account can create and manage noise reduction rules.

• A noise reduction rule applies only to the specified data quality rules within the current workspace.

Create Denoise Rule

1. Go to the Noise Reduction Management page.

   1. Go to the Dashboard.

   2. In the left-side navigation pane, choose Quality O&M > Monitor. In the upper-right corner, click Denoise Management.

2. Create a noise reduction rule.

   1. In the upper-right corner, click Create Denoise Rule.

   2. In the Create Denoise Rule panel, click Add Denoise Rule and configure the following parameters.

      Parameter	Description
      Data Timestamp	Select the date of the data partition to which this rule applies. When enabled, the rule prevents the associated scheduling node from being blocked if data in this partition fails a data quality check.
      Rule Type	Select the data quality rule types that this noise reduction rule applies to. You can select multiple types.
      Expected Value	Specify how to handle the data that fails the check. Treat the data as a sample or a baseline to prevent the scheduling node from being blocked. Denoise Sample: Prevents a scheduling node from being blocked when its sample data fails a data quality check on a specific date. Denoise Baseline: Prevents subsequent scheduling nodes from being blocked when the baseline data fails a data quality check. Note Sample The value of the data collected on the current day. For example, for a rule that monitors the table row count of an SQL node, for a 1-day fluctuation check, the sample is the row count of the table partition for that day. Baseline A comparison value derived from historical samples: If the rule monitors the table row count of an SQL node, for a 1-day fluctuation check, the baseline is the row count of the previous day's table partition. If the rule monitors the table row count of an SQL node, for a 7-day average fluctuation check, the baseline is the average row count from the previous 7 days.
      Enabled	Enables or disables the noise reduction rule. When enabled, the rule prevents the associated scheduling node from being blocked by specified data quality check failures.

   3. Click Save to complete the configuration.

Manage noise reduction rules

On the Denoise Management page, you can view, Edit, and Delete noise reduction rules. The rules are listed in a table with columns for Data Timestamp, Expected Value, Creator, Creation Time, Last Modified Time, Enable, and Actions. The Expected Value column shows if Denoise Sample or Denoise Baseline is selected. In the Enable column, use the switch to activate or deactivate a rule. In the Actions column, you can Edit or Delete the rule.