Service throttling limits the rate or concurrency of service invocations. When resources are limited, distributed systems use this method to protect themselves and downstream systems from traffic bursts and ensure availability. SOFAMesh currently provides a single-machine throttling feature.
How throttling works
The service throttling process is as follows:
The microservice console (MS) sends the throttling configuration to the distributed configuration center (DRM).
DRM sends the dynamic configuration for throttling to MOSN.
The throttling rule takes effect on the client or the server-side. The following process is an example of client-side throttling.
When a client application sends a request, MOSN monitors the application traffic. If a throttling rule is triggered, the application traffic is limited.
Notes
You can add multiple throttling rules. If multiple rules have the same service name and match conditions but different rate limits, the rule with the lower rate limit takes effect.
Configure rate limit rules that are appropriate for your environment. Improperly configured rules may cause abnormal application access.
Add a throttling rule
Log on to the Service Mesh console.
In the navigation pane on the left, click Service Administration, and then click the Service Throttling tab.
Click Add Throttling Group Rule, and then configure the following parameters:
Parameter
Description
Rule Name
The name of the service throttling rule.
Application
The name of the application. An asterisk (*) indicates all applications.
Throttling Direction
The direction of traffic throttling. Valid values:
Server-side throttling: The rule takes effect on the server-side.
If you select this option, set Application to the server-side application, and set Service and Method to the server-side service and method.
Client-side throttling: The rule takes effect on the client.
If you select this option, set Application to the client application, set Service to the service that the client needs to invoke, and set Method to the method of the server-side service that the client needs to invoke.
Service
The service to be throttled, based on the throttling direction. Click Switch Input Mode to switch between manual entry and drop-down selection.
Method
The method to be throttled. An asterisk (*) indicates all methods.
Throttling Mode
Select the throttling mode. The available values are:
Single-machine throttling: Throttles traffic for a single machine.
Cluster throttling: Throttles traffic for a cluster.
Throttling Type
The type of throttling. Valid values:
QPS: The maximum number of requests allowed per unit of time.
Maximum concurrent threads: The maximum number of concurrent access threads allowed at the same time.
Counter Selection
The counter configured for cluster throttling.
Click Counter Configuration to go to the counter configuration page.
Run Mode
The run mode of the throttling rule. Valid values:
Block Mode: The throttling rule takes effect.
Observer Mode: The throttling rule does not take effect. It only prints throttling logs in MOSN.
Throttling Algorithm
Uses the Token Bucket algorithm for throttling. This parameter is set only when Throttling Type is QPS.
The system adds tokens to the bucket at a constant rate. To process a request, a token must be taken from the bucket. If no tokens are available, the service is denied. For more information, see Throttling algorithm selection.
Token Bucket Coefficient
The coefficient of the token bucket, which determines the maximum size of the bucket. The default value is 1.
Maximum number of tokens = (Throttling threshold ÷ Unit of time) × Token bucket coefficient
Unit of Time (ms)
The unit of time for the rate limit rule, in ms. This parameter is set only when Throttling Type is QPS.
Throttling Threshold
The throttling threshold, which varies based on the selected throttling type:
If Throttling Type is QPS: The maximum number of requests allowed per unit of time. Requests exceeding this value are throttled.
If Throttling Type is Maximum concurrent threads: The maximum number of concurrent access threads allowed at the same time. Threads exceeding this threshold are throttled.
Exact Traffic Match
The match conditions for traffic. Only traffic that meets these conditions is subject to the throttling rule. If left empty, all traffic is matched.
You can configure multiple match conditions. The conditions are combined with a logical AND and are matched in order. The parameters are as follows:
Field: You can select system fields and request headers.
Field Name: The value varies based on the field type.
System fields: Include traffic type, caller application name, caller IP address, and callee application name.
Request header: Refers to the request header of the protocol. For example, for the Dubbo protocol, it is the attachment. For the HTTP protocol, it is the Request Header. You can customize request header parameters and values in your application system.
Logic: Includes equals, not equals, belongs to, does not belong to, and regex.
Field Value: The value corresponding to the field name.
Click Submit, and then click OK.
In the list of throttling rules, change the status of the rule you just created to On.
Edit a throttling rule
You can edit existing throttling rules at any time. The changes take effect in real-time after you submit them.
On the Service Throttling tab, click Edit in the row of the target throttling rule.
Edit the throttling rule as needed, and then click Submit.
Export throttling rules
You can export configured throttling rules to create a backup.
On the Service Throttling tab, choose More > Export.
Select the target rules, and then click Export.
Import throttling rules
You can import backed-up throttling rules to quickly create new ones.
On the Service Throttling tab, choose More > Import.
Click Select File, select the object file, and then click OK.
Delete a throttling rule
You can delete existing throttling rules. The deletion takes effect in real-time. Proceed with caution.
On the Service Throttling tab, click Edit for the target throttling rule.
Click OK.