Perform throttling for cluster tasks

Working principle of throttling

As shown in the preceding figure, the execution of a chunk comprises three phases forming a cycle.

1. Read phase

   The reader reads a batch of data. The data read interface returns a data list and a new-data flag.

   The reader pushes the data to the processor based on the throttling rate of the limiter.

2. Process phase

   After the processor receives the data, the processor processes the data in multi-thread mode. The processor then pushes the data in a queue to the writer.

3. Write phase

   After the writer receives the data, the writer writes the data in multi-thread mode. After the writer writes the data, the writer checks for new data based on the new-data flag, and continues to read data if new data is received.

Throttling configuration

When data is placed in a queue in the Process phase, a limiter is added to control the data read and placement rate, so as to simplify thread processing in the Process and Write phases. Task Scheduler (TS) provides the dynamic throttling and static throttling modes.

Static throttling

In static throttling, the throttling rule is configured by using code, and the throttling rate can be dynamically adjusted in the TS console. The advantage of this mode is that rate throttling also takes effect based on the default value even if you do not configure the throttling rule in the console. In this case, you need to manually specify a limiter.

TS provides a default limiter, DefaultLimiter. The following sample code shows how to specify a limiter:

public class ClusterJobExecuteOneHandler implements IClusterJobExecuteHandler<Integer, Integer> {

    //Omitted.

    @Override
    public ILimiter getLimiter() {
        //Process 10 data entries per second. 
        return new DefaultLimiter(10);
    }
   
    ...//Omitted.
}

If the value is set to null and you do not configure the throttling rule in the TS console, throttling is not performed.

If the default limiter DefaultLimiter cannot meet your requirements, you can customize the throttling rule as an extension interface is provided by TS. For example, if you want to perform throttling based on the business logic, CPU, memory, or IO, you can customize the throttling rule. You can implement the ILimiter interface and use the getLimiter() method to obtain limiters. The following sample code shows how to use the ILimiter interface:

public interface ILimiter {

    /**
     * Set the throttling rate. If permitsPerSecond is set to null or its value is smaller than or equal to 0, throttling is not performed. 
     *
     * @param permitsPerSecond
     */
    void setRate(Integer permitsPerSecond);

    /**
     * Obtain a permit for data processing requests. Data is blocked until a permit is obtained. 
     *
     * @return
     */
    void acquire();
}

• setRate(): the method that is invoked by TS to notify users of configuration changes. If no configuration is specified, the value of permitsPerSecond is null. For more information, see the implementation logic of DefaultLimiter.

• acquire(): the method that is invoked to obtain a permit. If no permit is obtained, data access requests are blocked.

For information about how to quickly obtain a custom limiter, see the implementation of DefaultLimiter.

Dynamic throttling

You can dynamically adjust the throttling rate in the TS console. Dynamic throttling takes effect regardless of whether a limiter is provided. If no limiter is specified, the TS client uses the default limiter DefaultLimiter. You can configure dynamic throttling in one of the following modes:

• Global throttling

  The throttling rule that you set when you configure or edit task scheduling settings take effect globally and are triggered by each task execution.

  For more information about how to configure task scheduling settings, see Create a scheduled task. To set the throttling rule, turn on the Advanced Options switch and set the Maximum Processing Rate of a Single Machine parameter, as shown in the following figure.

• One-off throttling

  You can perform throttling on a task before or during the task execution.

  1. Log on to the Scalable Open Financial Architecture Stack (SOFAStack) console.

  2. In the left-side navigation pane, choose Middleware > Task Scheduling > Task Configuration.

  3. Click the name of the target task. On the page that appears, click the Scheduling Record tab.

  4. In the left-side pane, select the target task and click the icon on the right side of Processing Data Amount.

  5. Set the throttling rate as required and click the icon.

     The configured throttling rule takes effect immediately. The client executes tasks based on the specified rate.