ListDataQualityResults-DataWorks(DataWorks)-阿里云帮助中心

Queries a list of data quality results by using paging.

Note

This API has been deprecated. We recommend migrating to dataworks-public(2024-05-18) - ListDataQualityScanRuns for enhanced functionality and continued support. While the deprecated API remains temporarily accessible, no further updates or bug fixes will be provided.

Operation description

需要购买 DataWorks 基础版及以上版本才能使用

Try it now

Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage.

Test

RAM authorization

The table below describes the authorization required to call this API. You can define it in a Resource Access Management (RAM) policy. The table's columns are detailed below:

Action: The actions can be used in the Action element of RAM permission policy statements to grant permissions to perform the operation.
API: The API that you can call to perform the action.
Access level: The predefined level of access granted for each API. Valid values: create, list, get, update, and delete.
Resource type: The type of the resource that supports authorization to perform the action. It indicates if the action supports resource-level permission. The specified resource must be compatible with the action. Otherwise, the policy will be ineffective.
- For APIs with resource-level permissions, required resource types are marked with an asterisk (*). Specify the corresponding Alibaba Cloud Resource Name (ARN) in the Resource element of the policy.
- For APIs without resource-level permissions, it is shown as All Resources. Use an asterisk (*) in the Resource element of the policy.
Condition key: The condition keys defined by the service. The key allows for granular control, applying to either actions alone or actions associated with specific resources. In addition to service-specific condition keys, Alibaba Cloud provides a set of common condition keys applicable across all RAM-supported services.
Dependent action: The dependent actions required to run the action. To complete the action, the RAM user or the RAM role must have the permissions to perform all dependent actions.

Action

Access level

Resource type

Condition key

Dependent action

dataworks:*

list

*All Resource

*

None

Request parameters

Parameter	Type	Required	Description	Example
ProjectId	integer	Yes	The DataWorks workspace ID.	100
DataQualityRuleId	integer	No	The data quality rule ID.	100001
DataQualityEvaluationTaskId	integer	No	The data quality evaluation task ID.	200001
DataQualityEvaluationTaskInstanceId	integer	No	The data quality evaluation task instance ID.	10001
CreateTimeFrom	integer	No	The earliest time when the data quality evaluation result was generated.	1710239005403
CreateTimeTo	integer	No	The latest time when the data quality evaluation result was generated.	1710239005403
BizdateFrom	string	No	The start business date of the rule instance.	2024-05-01
BizdateTo	string	No	The end business date of the rule instance.	2024-05-04
PageSize	integer	Yes	The page size for the paging query. Default value: 10.	10
PageNumber	integer	Yes	The page number for the paging query. Default value: 1.	1

Response elements

Element	Type	Description	Example
	object	返回值。
RequestId	string	API 请求 ID。	691CA452-D37A-****
PagingInfo	object	数据质量校验结果分页查询结果。
PageNumber	integer	页码。	1
PageSize	integer	页大小。	10
TotalCount	integer	总条数。	219
DataQualityResults	array<object>	质量校验结果列表。
	array<object>	质量校验结果。
Id	integer	The ID of the check result.	16033
Status	string	The status of the check result. Valid values: Running Error Passed Warned Critical	Passed
CreateTime	integer	The time when the data quality check result was generated.	1708284916414
TaskInstanceId	integer	The ID of the instance generated by the check.	200001
Sample	string	The sample values used for the check.	[ { "gender": "male", "_count": 100 }, { "gender": "female", "_count": 100 } ]
Details	array<object>	The information about the data quality check.
	object	The information about the data quality check.
ReferencedValue	string	The value that is calculated based on sample data. The value serves as a baseline value during the calculation of the value of the CheckedValue parameter.	0.0
CheckedValue	string	The value that is used for comparison with the threshold.	100.0
Status	string	The comparison result between the value of CheckedValue and the threshold. Valid values: Error Passed Warned Critical	Passed
Rule	object	The snapshot of the rule configuration when the check starts.
Id	integer	The rule ID.	100001
Name	string	The name of the rule. The name can be up to 255 characters in length and can contain digits, letters, and punctuation marks.	The table cannot be empty.
ProjectId	integer	The DataWorks workspace ID.	100
Enabled	boolean	Indicates whether the rule is enabled.	true
Severity	string	The strength of the rule. Valid values: High Normal	Normal
Description	string	The description of the rule. The description can be up to 500 characters in length.	this is a odps _sql task
TemplateCode	string	The code of the template that is referenced when you create a rule.	SYSTEM:user_defined_sql
Target	object	The monitored object of the rule.
Type	string	The type of the monitored object. Valid values: Table	Table
DatabaseType	string	The type of the database to which the table belongs. Valid values: maxcompute emr cdh hologres analyticdb_for_postgresql analyticdb_for_mysql starrocks	maxcompute
TableGuid	string	The ID of the table in Data Map.	odps.unit_test.tb_unit_test
SamplingConfig	object	The sampling settings.
Metric	string	The metrics used for sampling. Valid values: Count: the number of rows in the table. Min: the minimum value of the field. Max: the maximum value of the field. Avg: the average value of the field. DistinctCount: the number of unique values of the field after deduplication. DistinctPercent: the proportion of the number of unique values of the field after deduplication to the number of rows in the table. DuplicatedCount: the number of duplicated values of the field. DuplicatedPercent: the proportion of the number of duplicated values of the field to the number of rows in the table. TableSize: the table size. NullValueCount: the number of rows in which the field value is null. NullValuePercent: the proportion of the number of rows in which the field value is null to the number of rows in the table. GroupCount: the field value and the number of rows for each field value. CountNotIn: the number of rows in which the field values are different from the referenced values that you specified in the rule. CountDistinctNotIn: the number of unique values that are different from the referenced values that you specified in the rule after deduplication. UserDefinedSql: indicates that data is sampled by executing custom SQL statements.	Count
MetricParameters	string	The parameters required for sampling.	{ "columns": [ "id", "name" ] }
SettingConfig	string	The statements that are used to configure the parameters required for sampling before you execute the sampling statements. The statements can be up to 1,000 characters in length. Only the MaxCompute database is supported.	SET odps.sql.udf.timeout=600s;
SamplingFilter	string	The statements that are used to filter unnecessary data during sampling. The statements can be up to 16,777,215 characters in length.	id IS NULL
CheckingConfig	object	The check settings for sample data.
Type	string	The threshold calculation method. Valid values: Fixed Fluctation FluctationDiscreate Auto Average Variance	Fixed
ReferencedSamplesFilter	string	The method that is used to query the referenced samples. To obtain some types of thresholds, you need to query reference samples and perform aggregate operations on the reference values. In this example, an expression is used to indicate the query method of referenced samples.	{ "bizdate": [ "-1", "-7", "-1m" ] }
Thresholds	object	The threshold settings.
Expected	object	The expected threshold setting.
Value	string	The threshold value.	100.0
Operator	string	The comparison operator. Valid values: > >= < <= != =	>
Expression	string	The threshold expression. If the template specified by the TemplateCode parameter is about fluctuation, you must use an expression to represent the threshold for fluctuation. Examples: $checkValue > 0.01 $checkValue < -0.01 abs($checkValue) > 0.01 If the template specified by the TemplateCode parameter is about fixed value, you can also use an expression to represent the threshold. If you configure the Expression, Operator, and Value parameters for the threshold at the same time, the Expression parameter takes precedence over the Operator and Value parameters.	$checkValue > 0.01
Warned	object	The threshold settings for normal alerts.
Value	string	The threshold value.	100.0
Operator	string	The comparison operator. Valid values: > >= < <= != =	>
Expression	string	The threshold expression. If the template specified by the TemplateCode parameter is about fluctuation, you must use an expression to represent the threshold for fluctuation. Examples: $checkValue > 0.01 $checkValue < -0.01 abs($checkValue) > 0.01 If the template specified by the TemplateCode parameter is about fixed value, you can also use an expression to represent the threshold. If you configure the Expression, Operator, and Value parameters for the threshold at the same time, the Expression parameter takes precedence over the Operator and Value parameters.	$checkValue > 0.01
Critical	object	The threshold settings for critical alerts.
Value	string	The threshold value.	100.0
Operator	string	The comparison operator. Valid values: > >= < <= != =	>
Expression	string	The threshold expression. If the template specified by the TemplateCode parameter is about fluctuation, you must use an expression to represent the threshold for fluctuation. Examples: $checkValue > 0.01 $checkValue < -0.01 abs($checkValue) > 0.01 If the template specified by the TemplateCode parameter is about fixed value, you can also use an expression to represent the threshold. If you configure the Expression, Operator, and Value parameters for the threshold at the same time, the Expression parameter takes precedence over the Operator and Value parameters.	$checkValue > 0.01
ErrorHandlers	array<object>	The operations that you can perform after the rule-based check fails.
	object	The operation that you can perform after the rule-based check fails.
Type	string	The type of the operation. Valid values: SaveErrorData	SaveErrorData
ErrorDataFilter	string	The SQL statement that is used to filter failed tasks. If the rule is defined by custom SQL statements, you must specify an SQL statement to filter failed tasks.	SELECT * FROM tb_api_log WHERE id IS NULL

Examples

Success response

JSON format

{
  "RequestId": "691CA452-D37A-****",
  "PagingInfo": {
    "PageNumber": 1,
    "PageSize": 10,
    "TotalCount": 219,
    "DataQualityResults": [
      {
        "Id": 16033,
        "Status": "Passed",
        "CreateTime": 1708284916414,
        "TaskInstanceId": 200001,
        "Sample": "[\n  {\n    \"gender\": \"male\",\n    \"_count\": 100\n  }, {\n    \"gender\": \"female\",\n    \"_count\": 100\n  }\n]",
        "Details": [
          {
            "ReferencedValue": "0.0",
            "CheckedValue": "100.0",
            "Status": "Passed"
          }
        ],
        "Rule": {
          "Id": 100001,
          "Name": "The table cannot be empty.",
          "ProjectId": 100,
          "Enabled": true,
          "Severity": "Normal",
          "Description": "this is a odps _sql task",
          "TemplateCode": "SYSTEM:user_defined_sql",
          "Target": {
            "Type": "Table",
            "DatabaseType": "maxcompute",
            "TableGuid": "odps.unit_test.tb_unit_test"
          },
          "SamplingConfig": {
            "Metric": "Count",
            "MetricParameters": "{ \"columns\": [ \"id\", \"name\" ] }",
            "SettingConfig": "SET odps.sql.udf.timeout=600s; ",
            "SamplingFilter": "id IS NULL"
          },
          "CheckingConfig": {
            "Type": "Fixed",
            "ReferencedSamplesFilter": "{ \"bizdate\": [ \"-1\", \"-7\", \"-1m\" ] }",
            "Thresholds": {
              "Expected": {
                "Value": "100.0",
                "Operator": ">",
                "Expression": "$checkValue > 0.01"
              },
              "Warned": {
                "Value": "100.0",
                "Operator": ">",
                "Expression": " $checkValue > 0.01"
              },
              "Critical": {
                "Value": "100.0",
                "Operator": ">",
                "Expression": " $checkValue > 0.01"
              }
            }
          },
          "ErrorHandlers": [
            {
              "Type": "SaveErrorData\n",
              "ErrorDataFilter": "SELECT * FROM tb_api_log WHERE id IS NULL"
            }
          ]
        }
      }
    ]
  }
}

Error codes

See Error Codes for a complete list.

Release notes

See Release Notes for a complete list.