ListCrawlers-DataWorks(DataWorks)-阿里云帮助中心

Queries metadata crawlers by paged query, and supports filtering by workspace, data source, type, environment, owner, and name.

Operation description

Scenarios

Performs a paged query of metadata crawlers that you have access to. Supports filtering by workspace, data source, crawler type, environment, owner, and name.

Recommended flow

Combine filter conditions as needed to perform a conditional query of the crawler list.
Use the returned crawler IDs to invoke the get details, update, run, stop, run records, or delete operations.

Version requirements

DataWorks Basic Edition or higher is required.

Precautions

When multiple filter conditions are provided at the same time, they take effect in combination. The name field supports fuzzy match.

Try it now

Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage.

Test

RAM authorization

The table below describes the authorization required to call this API. You can define it in a Resource Access Management (RAM) policy. The table's columns are detailed below:

Action: The actions can be used in the Action element of RAM permission policy statements to grant permissions to perform the operation.
API: The API that you can call to perform the action.
Access level: The predefined level of access granted for each API. Valid values: create, list, get, update, and delete.
Resource type: The type of the resource that supports authorization to perform the action. It indicates if the action supports resource-level permission. The specified resource must be compatible with the action. Otherwise, the policy will be ineffective.
- For APIs with resource-level permissions, required resource types are marked with an asterisk (*). Specify the corresponding Alibaba Cloud Resource Name (ARN) in the Resource element of the policy.
- For APIs without resource-level permissions, it is shown as All Resources. Use an asterisk (*) in the Resource element of the policy.
Condition key: The condition keys defined by the service. The key allows for granular control, applying to either actions alone or actions associated with specific resources. In addition to service-specific condition keys, Alibaba Cloud provides a set of common condition keys applicable across all RAM-supported services.
Dependent action: The dependent actions required to run the action. To complete the action, the RAM user or the RAM role must have the permissions to perform all dependent actions.

Action

Access level

Resource type

Condition key

Dependent action

dataworks:ListCrawlers

list

*All Resource

*

None

Request syntax

POST  HTTP/1.1

Request parameters

Parameter	Type	Required	Description	Example
ProjectId	integer	No	The DataWorks workspace ID.	100
DataSourceIds	array	No	The list of data source IDs. Up to 10 IDs are supported.
	integer	No	The ID of a single data source.	12345
Type	string	No	The crawler type. Call GetCrawlerTypeCapabilities to query the valid values supported in the current region.	starrocks
EnvType	string	No	The DataWorks environment type. Dev indicates the development environment. Prod indicates the production environment.	Prod
Owner	string	No	The DataWorks user ID of the crawler owner.	1000
Name	string	No	The metadata crawler name. Supports fuzzy match.	example_crawler
PageNumber	integer	No	The page number. Starts from 1. Default value: 1.	1
PageSize	integer	No	The number of entries per page. Default value: 20. Maximum value: 100.	20

When multiple filter conditions are provided at the same time, they take effect in combination. DataSourceIds supports up to 10 IDs.

Response elements

Element	Type	Description	Example
	object	The response result.
RequestId	string	The request ID. Used to locate logs and troubleshoot issues.	9252F32F-D855-549E-8898-61CF5A733050
Success	boolean	Indicates whether the request succeeded.
PagingInfo	object	The pagination information.
TotalCount	integer	The total number of records that match the query conditions.	1
PageNumber	integer	The current page number.	1
PageSize	integer	The number of entries per page.	20
Crawlers	array<object>	The list of metadata crawlers.
	array<object>	The summary information of a metadata crawler.
Id	integer	The crawler ID.	1234
Name	string	The crawler name.	example_crawler
DataSourceId	integer	The data source ID.	12345
Type	string	The crawler type.	starrocks
Status	string	The crawler status. Valid values: VALID: the crawler configuration is valid and the associated data source exists. INVALID: the crawler configuration is invalid or the associated data source does not exist.	VALID
MetaEntityId	string	The meta-entity ID associated with the crawler. You can use this ID to call metadata query API operations.	starrocks:example-instance
ProjectId	integer	The DataWorks workspace ID.	100
EnvType	string	The DataWorks environment type. Valid values: Dev, Prod.	Prod
ResourceGroupId	string	The ID of the Serverless 2.0 resource group used to run the collection task.	Serverless_res_group_1234567890123456_1234567890
ScheduleConfig	object	The scheduling configuration.
Type	string	The scheduling type. Valid values: MANUAL: manual scheduling. NORMAL: periodic scheduling.	NORMAL
CronExpress	string	The cron expression.	0 0 2 ? * *
LastRunStatus	string	The last run status. Valid values: WAITING, RUNNING, SUCCESS, ERROR, SHUTDOWN. This value may be empty if the crawler has not been run.	SUCCESS
Owner	string	The DataWorks user ID of the crawler owner.	1000
CreateTime	integer	The creation time, in millisecond-level UNIX timestamp.	1710239005403
ModifyTime	integer	The modification time, in millisecond-level UNIX timestamp.	1710239005403
TaskId	integer	The DataWorks scheduling task ID associated with the crawler. You can use this ID to call GetTask to query the task definition.	1234

Crawlers returns the summary information of metadata crawlers on the current page.

Examples

Success response

JSON format

{
  "RequestId": "9252F32F-D855-549E-8898-61CF5A733050",
  "Success": false,
  "PagingInfo": {
    "TotalCount": 1,
    "PageNumber": 1,
    "PageSize": 20,
    "Crawlers": [
      {
        "Id": 1234,
        "Name": "example_crawler",
        "DataSourceId": 12345,
        "Type": "starrocks",
        "Status": "VALID",
        "MetaEntityId": "starrocks:example-instance",
        "ProjectId": 100,
        "EnvType": "Prod",
        "ResourceGroupId": "Serverless_res_group_1234567890123456_1234567890",
        "ScheduleConfig": {
          "Type": "NORMAL",
          "CronExpress": "0 0 2 ? * *"
        },
        "LastRunStatus": "SUCCESS",
        "Owner": "1000",
        "CreateTime": 1710239005403,
        "ModifyTime": 1710239005403,
        "TaskId": 1234
      }
    ]
  }
}

Error codes

See Error Codes for a complete list.

Release notes

See Release Notes for a complete list.