ListDataQualityRuleTemplates

Updated at:

Queries a list of data quality rule templates.

Note

This API has been deprecated. We recommend migrating to dataworks-public(2024-05-18) - ListDataQualityTemplates for enhanced functionality and continued support. While the deprecated API remains temporarily accessible, no further updates or bug fixes will be provided.

Operation description

You must purchase DataWorks Basic Edition or a higher edition to use this feature.

Try it now

Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage.

Test

RAM authorization

No authorization for this operation. If you encounter issues with this operation, contact technical support.

Request parameters

Parameter

Type

Required

Description

Example

ProjectId

integer

Yes

The DataWorks workspace ID.

10000

CreationSource

string

Yes

The creation source of the rule template. This parameter is required. Valid values:

  • System: system template.

  • UserDefined: user-defined template.

System

DirectoryPath

string

No

The category directory where the custom template is stored. Levels are separated by forward slashes (/). Each level name can be up to 1024 characters in length and cannot contain whitespace characters or backslashes.

/ods/order_data

Name

string

No

The fuzzy match for the template rule name. If the template is a system template, the internationalized name of the system template is fuzzy matched based on the language.

Table rows

PageNumber

integer

Yes

The page size for the paging query. Default value: 10.

10

PageSize

integer

Yes

The page number for the paging query. Default value: 1.

1

Response elements

Element

Type

Description

Example

object

The response parameters.

RequestId

string

The request ID.

691CA452-D37A-****

PagingInfo

object

The paging result of the data quality rule template paged query.

TotalCount

integer

The total number of entries.

42

PageSize

integer

The page size.

10

PageNumber

integer

The page number.

1

DataQualityRuleTemplates

array<object>

The list of rule templates.

array<object>

The details of a rule template.

Code

string

The code of the rule template.

USER_DEFINED:123

Name

string

The name of the rule template. It can be a combination of digits, letters, Chinese characters, and half-width or full-width punctuation marks, and can be up to 512 characters in length.

Table row Count Verification

ProjectId

integer

The DataWorks workspace ID.

2043

SamplingConfig

object

The settings required for sample collection.

Metric

string

The name of the sampling metric.

  • Count: the number of table rows

  • Min: the minimum value of the field

  • Max: the maximum value of the field

  • Avg: the average value of the field

  • DistinctCount: the number of unique values of the field

  • DistinctPercent: the ratio of the number of unique values of the field to the number of data rows

  • DuplicatedCount: the number of duplicate values of the field

  • DuplicatedPercent: the ratio of the number of duplicate values of the field to the number of data rows

  • TableSize: the size of the table

  • NullValueCount: the number of rows in which the field is null

  • NullValuePercent: the ratio of rows in which the field is null

  • GroupCount: each value and the corresponding number of data rows after aggregation by field value

  • CountNotIn: the number of rows in which the enumeration value does not match

  • CountDistinctNotIn: the number of unique values in which the enumeration value does not match

  • UserDefinedSql: collect samples by using custom SQL

Max

MetricParameters

string

The parameters required for sample collection.

{"Sql": "select count(1) from table;"}

SettingConfig

string

The runtime parameter setting statements that are inserted and executed before the sampling statement is executed. This parameter can be up to 1,000 characters in length. Currently, only MaxCompute is supported.

SET odps.sql.udf.timeout=600s; SET odps.sql.python.version=cp27;

CheckingConfig

object

The sample verification settings.

Type

string

The threshold calculation method.

  • Fixed

  • Fluctation

  • FluctationDiscreate

  • Auto

  • Average

  • Variance

Fixed

ReferencedSamplesFilter

string

Some types of thresholds require reference samples to be queried, and then the values of the reference samples are aggregated to obtain the threshold for comparison. An expression is used here to indicate the query method of the reference samples.

{ "bizdate": [ "-1", "-7", "-1m" ] }

VisibleScope

string

The available scope of the template:

  • Tenant: available to all tenants

  • Project: available only in the current project

Project

DirectoryPath

string

The category directory in which the custom template is stored. Levels are separated by forward slashes (/). Each level name can be up to 1,024 characters in length and cannot contain whitespace characters or slashes.

/ods/order_data

Examples

Success response

JSON format

{
  "RequestId": "691CA452-D37A-****",
  "PagingInfo": {
    "TotalCount": 42,
    "PageSize": 10,
    "PageNumber": 1,
    "DataQualityRuleTemplates": [
      {
        "Code": "USER_DEFINED:123",
        "Name": "Table row Count Verification\n",
        "ProjectId": 2043,
        "SamplingConfig": {
          "Metric": "Max",
          "MetricParameters": "{\"Sql\": \"select count(1) from table;\"}\n",
          "SettingConfig": "SET odps.sql.udf.timeout=600s; \nSET odps.sql.python.version=cp27;\n"
        },
        "CheckingConfig": {
          "Type": "Fixed",
          "ReferencedSamplesFilter": "{ \"bizdate\": [ \"-1\", \"-7\", \"-1m\" ] }\n"
        },
        "VisibleScope": "Project",
        "DirectoryPath": "/ods/order_data"
      }
    ]
  }
}

Error codes

See Error Codes for a complete list.

Release notes

See Release Notes for a complete list.