Configure time properties
Time properties are the core scheduling conditions that define when and how DataWorks triggers a scheduled task. They include not only the scheduled time for setting execution timing, but also a set of advanced configurations such as instance generation mode, scheduling calendar, rerun settings, and timeout policies. Together, these parameters form the time behavior plan of a task, controlling its trigger rules, execution boundaries, and automatic recovery from errors. This guide provides an in-depth look at how to configure each time property and how it works, helping you build unattended, stable, and reliable automated data workflows.
Quick start: Configuration example
Scenario: An e-commerce company needs to automatically calculate the previous day's total sales at 02:00 every day. If the task fails due to factors such as network fluctuations, the system should automatically retry it 3 times.
Step 1: Set the scheduled time
Set the scheduling cycle: Double-click the task name to open the task configuration panel. Click Schedule Settings > Time Properties on the right side. Set Scheduling Cycle to Day.
Set the scheduled time: Set the scheduled time to 02:00.
Step 2: Set the scheduling policy
Set the rerun property: Set the rerun property to You can run again after successful or failed operation..
Auto rerun upon failure: Select Auto Rerun upon Failure, set the number of retries to
3, and set the interval to5 minutes.Keep the default values for the other properties.
Result:
After you deploy the task, the system automatically triggers it at 02:00 every day, starting from the next day (T+1). If the task fails, the system retries it every 5 minutes. The task runs a maximum of 4 times (1 normal run + 3 retries).
How it works
Time property configurations define the entire lifecycle of a scheduled task, from instance generation to execution. They cover the following core dimensions:
Scheduling time: Visually defines the frequency and specific time for task execution.
Instance lifecycle management: Determines when an instance is created and when it is valid. This includes:
Instance generation mode: Controls whether instances are generated on the deployment day or the next day.
Effective period: Sets the overall valid time range for the task.
Scheduling calendar: Used to exclude non-working days (such as holidays).
Execution policy: Defines the behavior of an instance after its scheduled time arrives through the scheduling type, determining whether the instance runs normally, performs a dry run (skip), or is suspended.
Exception handling and fault tolerance: Provides automatic handling mechanisms for exceptions such as task execution failures or timeouts through timeout definitions and rerun settings.
Scheduling time
The scheduling time controls when a task is planned to run (also known as the scheduled time), which determines how often a task node is automatically triggered in the production environment. Based on the cycle you configure, the system generates the corresponding number of periodic instances for the task and drives the automated execution of the entire workflow through the scheduled time and scheduling dependencies of these instances.
The scheduled time and the business date are the two most important baseline time concepts in DataWorks. For more information, see Scheduled time and business date.
Independence and dependencies of scheduling cycles
Frequency independence: The scheduling frequency of a task (such as "hourly" or "daily") is determined by its own configuration and is independent of the frequency of its upstream tasks. For example, a daily-scheduled report task can depend on an hourly-scheduled data preparation task. For more information about dependency configurations across different cycle frequencies, see Configure cross-cycle dependencies.
Instance dependency: Dependencies between tasks with different scheduling cycles (cross-cycle dependencies) are essentially instance dependencies. The system automatically resolves these dependencies and ensures that a downstream instance meets the dependency conditions only after all upstream instances corresponding to its business date have completed successfully.
Dry run mechanism: To prevent blocking, non-daily scheduled tasks (such as weekly, monthly, or yearly tasks) generate a dry run instance on their non-running days. This instance is immediately set to successful without executing code or consuming resources, ensuring that downstream daily-scheduled tasks can be triggered normally.
Scheduled time vs. actual run time
Scheduled time: The time you set in the scheduling configuration is the expected start time of the task, which is the earliest moment when the task can start running.
Actual run time: The actual start time of the task depends on both of the following conditions being met simultaneously:
All upstream instances have completed successfully.
Computing resources are currently available.
Scheduling time zone
The scheduling time of a task uses the time zone corresponding to the region of its workspace by default. To change the time zone, see Configure the scheduling time zone for scenarios such as daylight saving time transitions. Supported time zones
The following are configuration examples for each scenario:
Minute-level scheduling
Set the start time, end time, and run interval. The minimum interval is 1 minute. Within the specified time range, the system generates multiple instances at fixed intervals.
Configuration example
The target node is scheduled every 30 minutes within the 00:00 to 23:59 time range each day.
Instance details
The following figure shows the scheduled times and scheduling parameter substitution for instances of a node that is scheduled every 30 minutes.
Hour-level scheduling
Applicable to high-frequency synchronization or near-real-time computing scenarios.
Instance generation logic: The system calculates periodic instances based on a closed interval [start time, end time].
Example: If the time range is set to
[00:00,03:00]and the interval is1 hour, the system generates 4 instances with scheduled execution times at 00:00, 01:00, 02:00, and 03:00.
Configuration methods:
Interval-based trigger: Runs cyclically at a fixed interval (such as every hour) within a specified time range.
Point-in-time trigger: Runs at one or more specified discrete points in time.
Configuration example
The target task is automatically scheduled every 6 hours within the 00:00 to 23:59 time range each day.
Scheduling details
The scheduling system generates 4 instances per day and runs them at the scheduled times of 00:00, 06:00, 12:00, and 18:00.
Day-level scheduling
Day-level scheduling is the most common scheduling method. It allows you to run a task once a day at a specified time within a valid time range. By default, newly created tasks use day-level scheduling, and the scheduled time is randomly generated within the 00:00 to 00:30 time range. You can specify a custom run time as needed. For example, you can specify that the task runs once a day at 13:00.
Configuration example
Set Scheduling Cycle to Day and set Scheduled Time to 13:00. The system automatically generates the cron expression 00 00 13 * * ?.
Scheduling details
The scheduling system automatically generates and runs instances for the task. The following figure shows the data processing times for each business day.
Week-level scheduling
Applicable to weekly business aggregations or periodic data maintenance performed at a fixed frequency each week.
Instance generation logic: The system generates instances every day within the valid time range.
Normal execution: Code logic is triggered on schedule only on the selected run days (such as Monday and Friday).
Auto dry run: Instances generated on non-selected days (such as Tuesday, Wednesday, Thursday, Saturday, and Sunday) are automatically set to the "dry run succeeded" status. No code logic is actually executed, and no computing resources are consumed.
Configuration example
If the target task is configured to run on Monday and Friday each week, instances generated on Monday and Friday are scheduled and executed normally. Instances generated on the other five days (Tuesday, Wednesday, Thursday, Saturday, and Sunday) are dry-run instances. When the scheduled time arrives, these instances are directly set to a succeeded status without actually executing code logic.
Set Scheduled Time to 13:00. The system automatically generates the cron expression 00 00 13 ? * 1,5.
Scheduling details
The scheduling system automatically generates and runs instances for the task.
Month-level scheduling
Applicable to scenarios where data is processed on a monthly basis. This is a commonly used scheduling cycle for core tasks such as financial settlements, monthly performance reports, and monthly user behavior analyses.
Instance generation logic: The system generates instances based on the specific days of the month that you select. Each monthly instance represents a complete calendar month.
Normal execution: Code logic is triggered on schedule only on the selected run days (such as the 15th or the last day of each month).
Auto dry run: Instances generated on non-selected days (such as the 1st through the 14th of each month) are automatically set to the "dry run" status. No code logic is actually executed, and no computing resources are consumed.
Configuration example
If the target task is configured to run settlement on the last day of each month, the instance generated on the last day of each month is scheduled and executed normally. Instances generated on other dates are dry-run, meaning they are automatically set to a success state when the scheduled time arrives without executing any code logic.
In this example, Specified Time is set to the 15th of each month and the last day of each month, and the scheduled time is set to 02:00. The system automatically generates the cron expression 00 00 02 15,L * ?.
Scheduling details
The scheduling system automatically generates and runs instances for the task.
When you use the backfill data feature to backfill data for a monthly scheduled task, note that the selected date is the data timestamp, where data timestamp = scheduled date - 1.
Examples:
For a task scheduled on the first day of each month, select the data timestamp as the last day of the previous month when you backfill data.
For a task scheduled on the last day of each month, select the data timestamp as the day before the last day of the month when you backfill data.
If you select a different data timestamp for backfilling, the backfill instances are dry-run.
For more dependency scenarios, see Dependency scenarios.
Yearly schedule
Applicable to long-cycle data tasks such as quarterly summaries, annual audits, or specific holiday-related processing.
Instance generation logic: Instances are generated for all 365 days of the year, but actual computation is triggered only on the specified months and dates.
Flexible combinations: Cross-month multi-date selection is supported. For example, you can run a task only on the first or last day of each quarter.
Dry-run mechanism: On dates that do not meet the execution conditions, instances are dry-run in seconds without actually executing logic or consuming compute resources.
Configuration example
If the target task is configured to run on the 1st and last day of January, April, July, and October each year, instances generated on those specified dates are scheduled and executed normally. Instances generated on other dates are dry-run, meaning they are automatically set to a success state when the scheduled time arrives without actually executing any code logic.
In this example, Specified Time is set to the last day of each month, and the scheduled time is set to 00:00. The system automatically generates the cron expression 00 00 00 L 1,4,7,10 ?.
Scheduling details
The scheduling system automatically generates and runs instances for the task.
The cron expression is automatically generated based on the time you select and cannot be manually modified.
Instance lifecycle management
These configurations determine whether task instances are created. They do not concern the specific execution time of a task. Instead, they manage at a macro level whether instances should be created and whether the scheduling rules themselves are valid.
Instance generation mode: Determines whether your configuration changes take effect on the current day or the next day. The next-day mode (default) ensures that changes do not interfere with the current day's tasks, enabling a smooth transition. The immediate mode forcibly resets the current day's instances to apply changes. Use this mode only for urgent fixes after you fully understand the impact and have assessed the risks.
Effective period: Defines the valid time range for the entire set of scheduling rules.
Scheduling calendar: Allows you to bind the schedule to a specific calendar, such as running only on trading days. This provides more flexible control than a weekly cycle.
Instance generation mode
After a node is deployed to the production scheduling system, the platform generates Auto Triggered Instances based on the Instance Generation Mode configured for the node, which includes Next Day and Immediately After Deployment. Regardless of the instance generation mode, you can view the latest dependencies of the task on the Cycle Task page in Operation Center. However, when auto triggered instances take effect or when dependencies are updated depends on the instance generation mode you select, as described below.
To ensure that instances generated immediately after deployment are scheduled normally without expired dry-runs, make sure the scheduled time of the task is at least 10 minutes later than the deployment time. The task actually executes code logic or regenerates instances based on the latest scheduling configuration only at an effective time point that is at least 10 minutes after deployment.
Regardless of the instance generation mode you select, changes made during the
22:00~24:00time period take effect on the third day after the node is deployed to the production environment. Avoid making task changes during this time period.
Instance generation mode | Description |
T+1 next day (recommended) |
|
Instances for the current day are generated immediately after the node is deployed. An instance is triggered normally only when the scheduled time is at least 10 minutes later than the deployment time. If the scheduled time is earlier than this threshold (including past times and times within the buffer period), the instance automatically enters a dry-run state, which means it is directly set to success without actually executing any code.
|
Difference in how task code and scheduling configurations take effect: Regardless of the instance generation mode, an instance always reads the latest task code at runtime. However, scheduling configurations (such as scheduling parameters and rerun policies) are frozen into the instance at the time of instance generation. If you modify the scheduling configuration after an instance is generated, the existing instance does not automatically sync the changes and continues to run with the old configuration from the time of generation. To apply changes, you can use Refresh Instance in Operation Center to update the scheduling configuration of existing instances in place. If you only need a specific run to use the latest configuration, you can also run a backfill for the task to rerun it with the latest configuration. For more information, see Instance generation mode: Immediately after deployment.
Effective period
Defines the valid time range for automatic task scheduling. After the effective period expires, the task no longer generates instances and becomes an expired task, which you can monitor and manage in the O&M overview.
Scheduling calendar
Used to define the scheduling dates and scheduling methods for tasks. DataWorks supports the following two types of calendars:
Default calendar: Provided by the platform and applicable to general scenarios.
Custom scheduling calendar: User-defined and applicable to industries and scenarios that require flexible scheduling dates (for example, the finance industry). You can configure rules such as the workspaces to which the calendar applies, the validity period of the calendar, and the scheduling methods for tasks on specified dates. For more information, see Custom scheduling calendar.
You need to combine the selected scheduling calendar with scheduled time and instance generation mode and other scheduling configurations to schedule tasks at specified times.
Execution policy
This configuration defines how a task is executed after it is triggered.
Scheduling type
The scheduling types supported by DataWorks are described in the following table.
Scheduling type | Impact | Use case |
Normal | Runs code normally and triggers downstream tasks. | A periodic task that runs in normal state, and the generated periodic instances also run in normal state. |
Suspend scheduling | The instance does not run after the scheduled time arrives and is set to failed, which blocks downstream tasks. Suspending scheduling is equivalent to freezing in Operation Center. A node with suspended scheduling displays the freeze icon | Applicable to emergency truncation of workflows. When a workflow does not need to run for a period of time, you can select this scheduling type to freeze the root node of the workflow. When the workflow needs to run again, you can unfreeze the root node. For more information about unfreezing tasks, see Unfreeze a task. |
Dry Run | The instance is directly set to successful after the scheduled time arrives (with a run duration of | When a node does not need to run for a period of time and you do not want to block its downstream nodes, you can select this scheduling type. |
in Operation Center. A frozen periodic task generates periodic instances that are also in frozen state.Exception handling and fault tolerance
This is key to ensuring data pipeline stability by presetting response plans for various possible exceptions.
Rerun property: Defines whether and how the system should automatically retry when a task fails.
Timeout settings: Sets a reasonable maximum run duration for a task. If the task exceeds this duration, the system can automatically trigger an alert or terminate the task to prevent a single node from getting stuck and consuming excessive resources.
Timeout definition
Sets the maximum allowed run duration for a task. If the run duration exceeds this value, the task is automatically terminated and set to failed to prevent the task from getting stuck and affecting the entire workflow.
Scope: Applies to periodic instances, backfill instances, and test instances.
Default value: The default is 3 to 7 days, which the system dynamically adjusts based on actual load.
Limits: The maximum value that can be manually set is 168 hours (7 days), and the minimum value is 1 minute.
Rerun description
Rerun policies are used to implement automatic fault recovery for tasks.
When you use the rerun property, make sure that tasks are idempotent (except for special tasks) to avoid data quality issues after a task is rerun upon failure. For example, use the
insert overwritestatement instead of theinsert intostatement during ODPS SQL development.
Rerun property: Rerun policies are used to implement automatic fault recovery for tasks.
Type
Use case
Allow Rerun Regardless of Running Status
Applicable to idempotent tasks that can be repeatedly executed without affecting results.
Do not re-run after successful operation, and re-run after failure.
Prevents data contamination caused by accidentally rerunning a successful task.
Disallow Regardless of Running Status
Applicable to non-idempotent tasks (such as certain data synchronization tasks). After you select this option, the Auto Rerun upon Failure feature is unavailable.
Auto rerun upon failure: When a task fails, the system automatically triggers a rerun.
Parameter
Description
Rerun Times
The number of automatic retries after failure. Valid values: 1 to 10.
Rerun Interval
The interval between retries. Valid values: 1 to 30 minutes.
NoteFailures caused by timeout do not trigger automatic reruns.
FAQ and best practices
Q: Why does the actual run time of a task differ from the scheduled time?
A: The scheduled time is only the expected start time of a task. The actual run also requires two conditions to be met: ① All upstream dependency tasks have run successfully. ② Scheduling resources are available. If either condition is not met, the task remains in a waiting state.
In addition, if the scheduled time is set around
00:00, the instance may experience a delay of several minutes before being triggered, because this period is the peak scheduling time for DataWorks. This is normal. For details, see the next question.Q: The scheduled time of a task is
00:00. Why does the instance start running several minutes late?A: Around
00:00every day is the peak scheduling time for DataWorks. A large number of tasks are triggered at the same time, and there is typically a delay of several minutes between the scheduled time and the actual start of an instance. This delay occurs during the task triggering phase and is normal. You do not need to submit a ticket.To reduce the triggering delay, we recommend that you set the scheduled time to a time point after
00:00to stagger peak hours. If your business requires triggering at00:00, factor in this delay when evaluating your pipeline latency.Q: My upstream task runs on an hourly schedule and my downstream task runs on a daily schedule. Can they depend on each other?
A: DataWorks supports dependencies between tasks with different scheduling cycles. The system uses a complex dependency resolution algorithm to ensure that a downstream task correctly waits for all its upstream instances to complete before running. For details, see Cross-cycle dependency principles.
Q: I backfilled data for last Friday. Why doesn't the
bizdatevariable show Friday's date?A: This usually happens because you confuse "data timestamp" (bizdate) with "run date". In DataWorks, bizdate = scheduled time − 1 day. When you backfill a task whose scheduled time is early Saturday morning, the bizdate you need to select is Friday.
Q: My task contains data write logic. Will data be duplicated if I rerun the task?
A: It is possible. Therefore, we strongly recommend that you ensure task idempotency. For data write tasks, use
INSERT OVERWRITE(overwrite) instead ofINSERT INTO(append) to ensure consistent results across multiple reruns.