Peak-load shifting queues requests when traffic to a consumer application surges. The system then processes these requests at a steady rate. This prevents sudden traffic spikes from causing a high system load.
Background information
In real-world applications, incoming traffic is often irregular. For example, an application can process 10 requests per second. In a single second, it suddenly receives 30 requests. Then, no requests arrive for the next two seconds. In this case, if the system immediately rejects the 20 excess requests, the application will be idle for the next two seconds. Therefore, the sudden burst of requests must be spread out over time. This keeps the system load within its processing capacity and processes as many requests as possible.
In the preceding figure, the yellow area represents requests that exceed the processing capacity. These requests are spread out and processed during subsequent idle periods. This method keeps the system load stable and processes as many messages as possible. A throttling rule ensures that messages are processed at a uniform rate.
How it works
The queueing feature for throttling in Application High Availability Service (AHAS) distributes a sudden surge of requests evenly. It allows requests to pass at fixed intervals. This creates a peak-load shifting effect and prevents traffic spikes from overloading the system. Excess requests are queued for processing. If the estimated queueing time for a request exceeds the maximum timeout period, AHAS rejects that request.
For example, if you set the queries per second (QPS) to 5 in uniform speed mode, one request is processed every 200 ms and excess requests are queued. If you also set the timeout period to 5 s, requests with an estimated queueing time longer than 5 s are immediately rejected. For more information, see Create a throttling rule.
The following is the diagram:
