Offline pipeline task running configuration
Configure task-level timeout periods and rerun policies for offline pipeline tasks to prevent resource waste from long-running tasks and improve execution reliability.
Procedure
-
In the top navigation bar of the Dataphin homepage, choose Develop > Data Integration.
-
In the top menu bar of the Integration page, select a project (In Dev-Prod mode, you also need to select an environment).
-
Click Batch Pipeline in the left-side navigation pane, and select the offline pipeline that you want to configure in the offline integration list on the right.
-
Click Property in the right sidebar of the current offline pipeline to open the Property panel.
-
In the Running Configuration section of the task Property panel, configure the following parameters.
Parameter
Description
Running Timeout
If a single execution exceeds the specified timeout threshold, the task is automatically terminated and marked as failed.
-
System Configuration: Selected by default. Uses the Recurring Node timeout set in the tenant-level running configuration in the O&M module. For more information, see Running Configuration.
-
Custom: You can enter a number between 0 and 168 with up to 2 decimal places.
Auto Rerun Upon Timeout
Specifies whether to automatically rerun recurring instances or data backfill instances that are terminated due to a timeout and marked as failed.
-
System Configuration: Selected by default. The behavior depends on the tenant-level configuration in the O&M module.
-
No Auto Rerun: Displayed when Auto Rerun upon Timeout is not enabled in O&M > Global Configuration > Rerun Configuration. The label shows System Configuration: No Auto Rerun.
-
Auto Rerun: Displayed when Auto Rerun upon Timeout is enabled in O&M > Global Configuration > Rerun Configuration. The label shows System Configuration: Auto Configuration.
-
-
Enable: Overrides the tenant-level rerun configuration. Timed-out instances that are terminated and marked as failed are automatically rerun.
-
Disable: Overrides the tenant-level rerun configuration. Timed-out instances that are terminated and marked as failed are not automatically rerun.
Auto Rerun Upon Failure
Specifies whether to automatically rerun task instances or data backfill instances that fail during execution.
-
System Configuration: Selected by default. Uses the Auto Rerun upon Failure setting from the tenant-level running configuration in the O&M module. For more information, see: Running Configuration.
-
Custom: For rerun times, you can enter an integer between 0 and 10. For rerun interval, you can enter an integer between 1 and 60.
-
-
Click OK.