Cluster throttling allows you to manage the real-time request volume for a service across an entire cluster. This approach solves issues caused by uneven traffic distribution, where individual instances might be throttled even when the overall cluster capacity has not been reached. Precisely controlling the total request volume for the entire cluster and using standalone throttling as a fallback provides more effective traffic protection. This topic explains how to configure cluster throttling rules.
Public preview notice
The cluster throttling feature is in public preview and free of charge. The trial tier is for testing only. The sum of QPS thresholds for a single application cannot exceed 2,000, and the total API traffic cannot exceed 3,000.
During the public preview, the real-time creation of the Token Server and the stability of throttling are not guaranteed. Exercise caution when using this feature in a production environment, even with the production tier.
Prerequisites
You must have AHAS Professional activated. If not, visit the Activation Page.
SDK version 1.6.0 or later and Agent version 1.7.5 or later.
Background
Cluster throttling is typically used in the following scenarios:
Uneven traffic distribution across instances: Traffic may be distributed unevenly across instances due to factors like load balancing. In this case, standalone throttling might be triggered on some instances even if the total cluster request volume is below the limit.
Low-volume cluster throttling: For some high-availability protection scenarios, you may need to limit the QPS for a service to a very low number. If the QPS per instance averages less than 1, standalone throttling cannot provide precise control. For example, you want to limit the total QPS to 50 across 100 nodes.
Business-driven traffic control: For example, you might want to limit a specific API to 10 calls per user per minute.
Communication between the Token Client and the Token Server adds network overhead, increasing response time by approximately 2 ms to 5 ms.
Step 1: Select a tier and create a cluster
Log on to the AHAS console and select a region in the upper-left corner.
In the left-side navigation pane, choose .
On the Application Protection page, click your target application on the Application Card tab.
In the left-side navigation pane of the application management page, choose .
In the Cluster Throttling Resource Configuration section, set Cluster Type to Production, use the slider to select the total configuration level for cluster throttling, click Create, and then click Confirm in the dialog box.
The total configuration level represents the maximum QPS for your throttled APIs and should match your maximum anticipated traffic.
NoteIf the actual traffic (whether throttled or not) exceeds the configured maximum QPS, the throttling policy degrades to standalone mode. To ensure effective throttling, the sum of all rule thresholds is capped at 95% of the configured maximum QPS. For example, if you set the maximum QPS to 100,000, the sum of all rule thresholds cannot exceed 95,000.
The calculation of the cluster's maximum QPS does not include standalone rules.
After you select a total configuration tier and create the cluster, the system automatically allocates a Token Server for the application.
Optional: In the Token Client Settings section, click Edit in the Actions column. Set the Token Request Timeout and click OK.
You may need to adjust the timeout to account for high network latency between the Token Client and the Token Server.
NoteToken Client configuration requires AHAS Sentinel Client 1.6.0 or later.
The Token Request Timeout is measured in milliseconds (ms), with a valid range from 1 to 10,000. We recommend a value of 20 ms or less. In public network environments with high latency, a timeout of around 50 ms is recommended, but it must not exceed 80 ms.
Step 2: Set cluster throttling rules
In the left-side navigation pane of the application management page, click Rule Management, select the target solution tab, click the Throttling Rule tab, and then click New Throttling Rule.
In the New Throttling Rule dialog box, enable Cluster Throttling and configure the following parameters:
Click Create to create the rule.
After the rule is created, you can view it on the Rule Settings page. The Threshold Mode for this rule is displayed as Cluster.
