DescribeDataFlowTasks-阿里云帮助中心

Queries the details of a data flow task.

Operation description

Querying data flow tasks is supported only on CPFS 2.2.0 or later and CPFS-AI 2.4.0 or later. You can check the version on the file system details page in the console.

Try it now

Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage.

Test

RAM authorization

The table below describes the authorization required to call this API. You can define it in a Resource Access Management (RAM) policy. The table's columns are detailed below:

Action: The actions can be used in the Action element of RAM permission policy statements to grant permissions to perform the operation.
API: The API that you can call to perform the action.
Access level: The predefined level of access granted for each API. Valid values: create, list, get, update, and delete.
Resource type: The type of the resource that supports authorization to perform the action. It indicates if the action supports resource-level permission. The specified resource must be compatible with the action. Otherwise, the policy will be ineffective.
- For APIs with resource-level permissions, required resource types are marked with an asterisk (*). Specify the corresponding Alibaba Cloud Resource Name (ARN) in the Resource element of the policy.
- For APIs without resource-level permissions, it is shown as All Resources. Use an asterisk (*) in the Resource element of the policy.
Condition key: The condition keys defined by the service. The key allows for granular control, applying to either actions alone or actions associated with specific resources. In addition to service-specific condition keys, Alibaba Cloud provides a set of common condition keys applicable across all RAM-supported services.
Dependent action: The dependent actions required to run the action. To complete the action, the RAM user or the RAM role must have the permissions to perform all dependent actions.

Action

Access level

Resource type

Condition key

Dependent action

nas:DescribeDataFlowTasks

get

*DataFlow

acs:nas:{#regionId}:{#accountId}:filesystem/{#filesystemId}

None

Request parameters

Parameter	Type	Required	Description	Example
FileSystemId	string	Yes	The ID of the file system. CPFS General-purpose Edition: The ID must start with `cpfs-`. Example: cpfs-099394bd928c**. CPFS AI Computing Edition: The ID must start with `bmcpfs-`. Example: bmcpfs-290w65p03ok64ya**.	bmcpfs-290w65p03ok64ya****
Filters	array<object>	No	An array of filters.
	object	No
Key	string	No	The name of the filter key. Valid values: DataFlowIds: Filters by data flow ID. TaskIds: Filters by data flow task ID. Originator: Filters by originator. TaskActions: Filters by task type. DataTypes: Filters by data type. Status: Filters by status. CreateTimeBegin: Filters for tasks created after the specified time. CreateTimeEnd: Filters for tasks created before the specified time. StartTimeBegin: Filters for tasks that started after the specified time. StartTimeEnd: Filters for tasks that started before the specified time. EndTimeBegin: Filters for tasks that ended after the specified time. EndTimeEnd: Filters for tasks that ended before the specified time.	DataFlowIds
Value	string	No	The value of the filter key. This parameter does not support wildcards. If Key is set to DataFlowIds, specify one or more data flow IDs. You can specify up to 10 IDs. Example: `df-194433a5be31**` or `df-194433a512a2,df-234533a5be31`. If Key is set to TaskIds, specify one or more data flow task IDs. You can specify up to 10 IDs. Example: `task-38aa8e890f45` or `task-38aa8e890f45,task-29ae8e890f45`. If Key is set to TaskActions, specify the task type. Valid values are Import, Export, Evict, Inventory, StreamImport, and StreamExport**. You can specify multiple values. Note that CPFS AI Computing Edition supports only Import, Export, StreamImport, and StreamExport. StreamImport and StreamExport are supported only on CPFS AI Computing Edition 2.6.0 and later. If Key is set to DataTypes, specify the data type. Valid values are MetaAndData, Metadata, and Data. You can specify multiple values. If Key is set to Originator, specify the originator of the task. Valid values are User and System. If Key is set to Status, specify the task status. Valid values are Pending, Executing, Failed, Completed, Canceling, and Canceled. You can specify multiple values. If Key is set to CreateTimeBegin, specify the earliest creation time for the query. The time must be in `yyyy-MM-ddThh:mmZ` format. If Key is set to CreateTimeEnd, specify the latest creation time for the query. The time must be in `yyyy-MM-ddThh:mmZ` format. If Key is set to StartTimeBegin, specify the earliest start time for the query. The time must be in `yyyy-MM-ddThh:mmZ` format. If Key is set to StartTimeEnd, specify the latest start time for the query. The time must be in `yyyy-MM-ddThh:mmZ` format. If Key is set to EndTimeBegin, specify the earliest end time for the query. The time must be in `yyyy-MM-ddThh:mmZ` format. If Key is set to EndTimeEnd, specify the latest end time for the query. The time must be in `yyyy-MM-ddThh:mmZ` format.	df-194433a5be31****
NextToken	string	No	A pagination token. If a request's response is truncated, include this token in a subsequent request to retrieve the next page of results.	TGlzdFJlc291cmNlU****mVzJjE1MTI2NjY4NzY5MTAzOTEmMiZORnI4NDhVeEtrUT0=
MaxResults	integer	No	The maximum number of results to return in a single request. Value range: 10 to 100. Default value: 20.	20
WithReports	boolean	No	Specifies whether to return report information. true (default): Returns report information. false: Does not return report information. Note Set this parameter to `False` to improve query performance. This parameter is supported only on CPFS AI Computing Edition.	True

Response elements

Element	Type	Description	Example
	object
RequestId	string	The request ID.	2D69A58F-345C-4FDE-88E4-BF518948****
NextToken	string	If the response is truncated, a pagination token is returned. You can use this token in the next request to retrieve the next page of results.	TGlzdFJlc291cmNlU****mVzJjE1MTI2NjY4NzY5MTAzOTEmMiZORnI4NDhVeEtrUT0=
TaskInfo	object
Task	array<object>	A collection of dataflow tasks.
	array<object>
FilesystemId	string	The file system ID.	bmcpfs-290w65p03ok64ya****
DataFlowId	string	The dataflow ID.	df-194433a5be31****
TaskId	string	The dataflow task ID.	task-38aa8e890f45****
SourceStorage	string	The access path of the source storage. Format: `<storage type>://[<account id>:]<path>`. Parameters: storage type: Only OSS is supported. account id: the UID of the account to which the source storage belongs. path: The name of the OSS bucket. The name must meet the following requirements: Contains only lowercase letters, digits, and hyphens (-), and must start and end with a lowercase letter or a digit. The length cannot exceed 128 characters. It must be encoded in UTF-8. Note The OSS bucket must exist in the current region. The account id parameter is supported only by CPFS AI-Computing Edition 2.6.0 and later.	oss://178321033379****:bucket-01
FileSystemPath	string	The path of the fileset in the CPFS file system. Limits: The path must be 2 to 1,024 characters in length and can contain English and Chinese characters. The path must be encoded in UTF-8. The path must start and end with a forward slash (/). The directory must be a fileset directory on the CPFS file system. Note This parameter is supported only for CPFS.	/a/b/c/
Originator	string	The initiator of the dataflow task. Valid values: User: The task was initiated by a user. System: The task is an auto-update task automatically initiated by CPFS at a predefined interval. Note This parameter is supported only for CPFS.	User
TaskAction	string	The type of the dataflow task. Valid values: Import: Imports specified data from the source storage to the CPFS file system. Export: Exports specified data from the CPFS file system to the source storage. StreamImport: Imports streaming data from the source storage to the CPFS file system. StreamExport: Exports streaming data from the CPFS file system to the source storage. Evict: Releases the data blocks of a file on the CPFS file system, retaining only its metadata. You can still query the file, but its data blocks are cleared and no longer consume storage capacity. When you access the file, its data is loaded from the source storage on demand. Inventory: Retrieves an inventory of files on the CPFS file system managed by a dataflow. This inventory shows the cache status of the files. Note StreamImport and StreamExport are supported only by CPFS AI-Computing Edition 2.6.0 and later.	Import
DataType	string	The type of data on which the dataflow task operates. Valid values: Metadata: The metadata of a file, such as its timestamp, ownership, and permissions. If you select this value, only the metadata is imported. The file is visible, but its data is loaded from the source storage on demand upon access. Data: The data blocks of the file. MetaAndData: The metadata and data blocks of the file. Note CPFS AI-Computing Edition supports only the MetaAndData type.	MetaAndData
Progress	integer	The progress of the dataflow task. The number of operations that have been performed for the dataflow task.	50
Status	string	The status of the dataflow task. Valid values: Pending: The dataflow task is created and is waiting to be executed. Executing: The dataflow task is in progress. Failed: The dataflow task failed. You can view the dataflow task report to identify the cause of the failure. Completed: The dataflow task is complete. You can view the dataflow task report to check whether all files are transferred as expected. Canceled: The dataflow task is canceled and not complete. Canceling: The dataflow task is being canceled.	Failed
ReportPath `deprecated`	string	The path in the CPFS file system where the dataflow task report is saved. For CPFS, the task report is generated in the `.dataflow_report` directory of the file system. For CPFS AI-Computing Edition, this parameter returns an OSS download link for the task report. You can use the link to download the task report.	/path_in_cpfs/reportfile.cvs
CreateTime	string	The time when the task was created.	2021-08-04 18:27:35
StartTime	string	The time when the task started.	2021-08-04 18:27:35
EndTime	string	The time when the task ended.	2021-08-04 18:27:35
FsPath	string	The path of the intelligent directory.	/aa/
ConflictPolicy	string	The conflict policy for files with the same name. SKIP_THE_FILE: skips the file with the same name. KEEP_LATEST: compares the update time and retains the latest version. OVERWRITE_EXISTING: forcibly overwrites the file with the same name.	KEEP_LATEST
Directory	string	The directory where the dataflow task is executed.	/path_in_cpfs/
DstDirectory	string	The destination directory to which the dataflow task is mapped.	/path_in_cpfs/
ErrorMsg	string	The cause of the task error. Note If this field is not returned or is empty, it indicates that no error occurred.	{"ErrorKey":"PATH_NOT_ACCESSIBLE","ErrorDetail":"lstat /cpfs/370lx1ev9ss27o****/test/abcdfnotfound: no such file or directory"}
ProgressStats	object	The progress of the dataflow task.
FilesTotal	integer	The number of files scanned in the source storage.	3
FilesDone	integer	The number of files whose data has been flowed (including skipped files).	3
ActualFiles	integer	The actual number of files whose data has been flowed.	3
BytesTotal	integer	The amount of data scanned in the source storage. Unit: bytes.	131092971520
BytesDone	integer	The amount of data that has been flowed (including skipped data). Unit: bytes.	131092971520
ActualBytes	integer	The actual amount of flowed data. Unit: bytes.	131092971520
RemainTime	integer	The estimated remaining time. Unit: seconds.	437
AverageSpeed	integer	The average flow speed. Unit: Byte/s.	342279299
Reports	object
Report	array<object>	The list of reports. Note Streaming tasks do not support reports. When WithReport is set to True, this parameter returns the report data of CPFS AI-Computing Edition. Only CPFS AI-Computing Edition supports the WithReport parameter.
	object
Name	string	The name of the report. CPFS: TotalFilesReport: the task report. CPFS AI-Computing Edition: FailedFilesReport: the report of failed files. SkippedFilesReport: the report of skipped files. SuccessFilesReport: the report of successful files.	TotalFilesReport
Path	string	The link to the report.	https://a-hbr-temp-cn-hangzhou-staging.oss-cn-hangzhou.aliyuncs.com/temp/report/162319438359**/job-000bb6fwqficjbxk/job-000bb6fwqficjbxk_failed.zip?Expires=1721201422&OSSAccessKeyId=LTA**************&Signature=Fp%2BvauORTIVxooXY2tec6z0T%2Bp4%3D
Includes	string	Filters directories under the directory and transfers the folders contained in the filter directory. Note This feature is supported only by CPFS AI-Computing Edition.	["/test/","/test1/"]
TransferFileListPath	string	Specifies an OSS directory and synchronizes data based on the content of the CSV file in the OSS directory. Note This feature is supported only by CPFS AI-Computing Edition.	/path_in_cpfs/

Examples

Success response

JSON format

{
  "RequestId": "2D69A58F-345C-4FDE-88E4-BF518948****",
  "NextToken": "TGlzdFJlc291cmNlU****mVzJjE1MTI2NjY4NzY5MTAzOTEmMiZORnI4NDhVeEtrUT0=",
  "TaskInfo": {
    "Task": [
      {
        "FilesystemId": "bmcpfs-290w65p03ok64ya****",
        "DataFlowId": "df-194433a5be31****",
        "TaskId": "task-38aa8e890f45****",
        "SourceStorage": "oss://178321033379****:bucket-01",
        "FileSystemPath": "/a/b/c/",
        "Originator": "User",
        "TaskAction": "Import",
        "DataType": "MetaAndData ",
        "Progress": 50,
        "Status": "Failed",
        "ReportPath": "/path_in_cpfs/reportfile.cvs",
        "CreateTime": "2021-08-04 18:27:35",
        "StartTime": "2021-08-04 18:27:35",
        "EndTime": "2021-08-04 18:27:35",
        "FsPath": "/aa/",
        "ConflictPolicy": "KEEP_LATEST",
        "Directory": "/path_in_cpfs/",
        "DstDirectory": "/path_in_cpfs/\n",
        "ErrorMsg": "{\"ErrorKey\":\"PATH_NOT_ACCESSIBLE\",\"ErrorDetail\":\"lstat /cpfs/370lx1ev9ss27o****/test/abcdfnotfound: no such file or directory\"}",
        "ProgressStats": {
          "FilesTotal": 3,
          "FilesDone": 3,
          "ActualFiles": 3,
          "BytesTotal": 131092971520,
          "BytesDone": 131092971520,
          "ActualBytes": 131092971520,
          "RemainTime": 437,
          "AverageSpeed": 342279299
        },
        "Reports": {
          "Report": [
            {
              "Name": "TotalFilesReport",
              "Path": "https://a-hbr-temp-cn-hangzhou-staging.oss-cn-hangzhou.aliyuncs.com/temp/report/162319438359****/job-000bb6fwqficjbxk****/job-000bb6fwqficjbxk****_failed.zip?Expires=1721201422&OSSAccessKeyId=LTA****************&Signature=Fp%2BvauORTIVxooXY2tec6z0T%2Bp4%3D"
            }
          ]
        },
        "Includes": "[\"/test/\",\"/test1/\"]\n",
        "TransferFileListPath": "/path_in_cpfs/\n"
      }
    ]
  }
}

Error codes

HTTP status code	Error code	Error message	Description
400	IllegalCharacters	The parameter contains illegal characters.	The parameter contains illegal characters.
400	MissingFileSystemId	FileSystemId is mandatory for this action.
400	InvalidFilesystemVersion.NotSupport	This Api does not support this fileSystem version.	This Api does not support this fileSystem version.
403	OperationDenied.DataFlowNotSupported	The operation is not supported.
404	InvalidParameter.InvalidNextToken	The specified NextToken is invaild.
404	InvalidFileSystem.NotFound	The specified file system does not exist.	The specified file system does not exist.
404	InvalidDataFlow.NotFound	The specified data flow does not exist.
404	InvalidParameter.InvalidMaxResults	The specified MaxResults is invalid.
404	InvalidFilterParam	The specified Filter.N.Key is invalid.

See Error Codes for a complete list.

Release notes

See Release Notes for a complete list.