This topic describes the features of task scheduling.
Distributed task scheduling framework
The framework provides various job types for different scenarios to enable reliable and automated task scheduling.
Simple job
This is the most basic job type and is suitable for scenarios that involve simple business logic. A single simple job is processed by one handler and supports job sharding and steps:
Use job sharding for concurrent execution when resource utilization is not a concern.
Use the steps feature to divide a job into multiple steps that run sequentially.
Cluster job
This job type is suitable for complex scheduling scenarios. It lets you split a large job across multiple clients for concurrent execution based on custom splitting logic.
A cluster job has two phases:
Split phase: Data is partitioned. There is no limit on the number of splitting layers. The splitting results are reported to the server-side. The server-side then notifies clients to pull the data for processing based on the resulting Chunks. A Chunk is a collection of data shards to be processed.
Execution phase: After receiving the notification, the client pulls and processes a batch of data. The client continues to pull new batches until all data is processed.
Topology job
A topology job is a special job type that consists of a collection of event-triggered tasks. The topology job itself can be triggered by a CRON expression or an event. However, its sub-tasks must be event-triggered, and the communication method must be CALLBACK. The execution flow of a topology job proceeds from a start node to an end node, forming a directed acyclic graph.
Graphical task dependency orchestration
You can orchestrate multiple jobs into a topology graph based on their execution order. The graph runs automatically without manual intervention, which ensures that many jobs are scheduled quickly and efficiently.
Multiple dependency management gateways
You can manage relationships between jobs using direct dependencies, conditional gateways, and parallel gateways.
Manual intervention during execution
You can perform a graceful stop or resume a running job topology from a breakpoint.
Graphical orchestration
You can use the graphical interface to quickly orchestrate jobs and improve work efficiency.
Task configuration items
A wide range of configuration items is available, including multiple routing policies, failure retry patterns, and missed trigger handling policies. You can also configure input parameters for scheduled jobs online.
Multiple routing policies
You can trigger scheduled jobs on clients in a cluster randomly, by polling, or by targeting specific clients.
Multiple failure retry patterns
Supported retry patterns include manual retry, no retry, retry three times, and retry until the next trigger. You can also configure a timeout period.
Multiple missed trigger handling policies
Handling policies for missed triggers include do nothing, trigger immediately, and trigger immediately if time permits.
Online configuration of input parameters
You can configure custom parameters. These parameters are passed to the job when it is triggered.
Task management features
Various management features are available for jobs, including manual triggers, configuration import and export, and OpenAPI.
Automatic and manual triggers
You can trigger jobs on a schedule, periodically, by an event, or manually.
Configuration import/export and batch operations
You can import and export configurations in batches as files in the console. You can also enable or disable jobs in batches.
OpenAPI support
A wide range of management APIs is provided for developers to view, add, trigger, and delete jobs.
Graphical centralized management interface
A user-friendly management interface is provided that includes a task dependency view, task execution monitoring, client node information, and trigger record viewing.
Task dependency view
Obtain a panoramic view of jobs to clearly identify their upstream and downstream dependencies.
Task execution monitoring
You can view the real-time execution status, progress, and duration of jobs.
Trigger record viewing
This includes information such as the trigger time, the client IP address, the execution status, and the reason for any failures.
Unitized architecture
Task scheduling supports a unitized architecture. In this architecture, the scheduler can detect a job client's unit information and trigger the job in a specified unit. The architecture is implemented as follows:
The client connects to all servers and provides its unit information during connection registration.
When the server-side triggers a job, it uses the specified unit information to find the corresponding connection and trigger the job.