Manage consumer quotas
Use quota rules to control the total number of model calls that an AI Gateway consumer can make within a given cycle, so that you can manage costs at a fine granularity. Quota rules support two metering dimensions, Token and Credits, and reset automatically by natural day, natural week, or natural month.
Overview
The AI Gateway instance now includes FinOps capabilities, enabling instance-level quota rules for consumers. Quota rules limit the number of model invocations allowed for bound consumers by specifying: Quota type (Token-based or Credits-based), Quota limit, Reset cycle (daily, weekly, or monthly), and Time zone. When the quota limit is reached, consumers cannot invoke models until the next reset cycle begins.
The consumer quota feature supports:
Set Token quotas for consumers at the instance level.
Set the quota cycle to a natural day, natural week, or natural month. Quota rules for multiple cycles can coexist and are decremented at the same time.
Configure a time zone for a quota rule.
Reset a quota rule manually.
View quota utilization on a monitoring dashboard.
FinOps risk notifications
If you receive a FinOps risk notification for your AI Gateway instance, the notification indicates a potential risk in the way the instance uses FinOps features. Confirm what the risk means, and then adjust the configuration of the instance. Please upgrade to 2.1.21 or higher.
What the FinOps risk notification means
The notification indicates that, on the current version of the AI Gateway instance, using FinOps features carries a potential risk: metering accuracy can be affected, or consumers can be throttled unexpectedly. The risk is triggered when log fields are not properly configured, or when no consumer quota is configured for the instance.
How to resolve the risk
To clear the risk notification, complete the following two steps:
Log on to the AI Gateway console, go to the target instance, and choose Model API > Policies and Plugins > AI Request Logs. Enable the log policy and keep only the fields that are required for metering, such as
input_tokenandoutput_token. Disable all non-essential fields to reduce metering cost.Under FinOps > Consumer Quota, configure a Token Quota or Credits Quota periodic quota for the consumer. For the detailed procedure, see the "Create a quota rule" section in this topic.
After you complete both steps, the risk is cleared and the notification is no longer triggered.
Using a large quota to clear the notification without limiting traffic
If you do not want the quota to restrict production traffic, set Quota Limit to a value far above your actual usage. A quota is then configured, which clears the risk notification, and calls in your production environment are not affected.
To view usage statistics, log delivery must be enabled and the log index must not have been manually disabled.
Prerequisites
An AI Gateway instance is created and its engine version is 2.1.19 or later. To use Credits quotas, upgrade the engine version to 2.1.20 or later.
The gateway instance type is not Serverless (Legacy).
A consumer is created.
Simple Log Service is activated and log delivery is enabled. Log delivery is required to view usage statistics.
Consumer quotas and consumer throttling complement each other. A quota limits the total usage within a cycle. Throttling limits instantaneous usage by QPS or concurrency.
Create a quota rule
Use AI gateway console to manage quota of consumers.
Under FinOps, on the Consumer Quota page, click Create Quota Rule.
Configure the following settings.
Step 1: Basic configuration
Configuration item | Description |
Rule Name | Required. The name can contain letters, digits, underscores (_), and hyphens (-), and must be 1 to 64 characters in length. |
Rule Type | Required. Select Token Quota or Credits Quota. A Token quota is metered by the number of model tokens. A Credits quota is metered by a unified credits system. |
Quota | Required. The minimum value is 1 and the default value is 60000. The unit switches dynamically to Tokens or Credits based on the selected quota type. |
Reset Period | Required. Select Calendar Day, Calendar Week, or Calendar Month. You cannot modify this parameter after the rule is created. |
Time Zone | Required. Select the reference time zone for quota resets. The default value is UTC+8 (Beijing/Shanghai/Singapore/Hong Kong). Nine time zones are supported: UTC+9 (Tokyo), UTC+8 (Beijing/Shanghai/Singapore/Hong Kong), UTC+7 (Bangkok), UTC+5:30 (New Delhi), UTC+3 (Moscow), UTC+1 (Berlin/Paris), UTC+0 (London), UTC-5 (New York), and UTC-8 (Los Angeles). You cannot modify this parameter after the rule is created. |
Step 2: Select consumers
In the Select Consumer area, use the transfer box to select the consumers that this quota rule applies to.
The left list shows available consumers. The right list shows selected consumers.
Search by consumer name to filter the list.
You can select up to 1,000 consumers in a single create or edit operation.
A View link appears to the right of each selected consumer. Click the link to go to the consumer details page.
For details about how quotas work together with authentication, see Model API consumer authentication.
Step 3: Conflict preview
Before you submit the rule, the system runs conflict detection automatically. A consumer can have only one quota rule for a given reset period.
No conflict: the rule is created. The page returns to the quota rule list, a success message appears at the top of the page, and the new rule appears in the list.
Conflict detected: the Quota Rule Conflict dialog box appears and lists the conflicting consumers and the quota types that they already have. You can select one of the following options:
Overwrite Existing Rules: overwrite the existing quota rules with the current rule.
Back to Selection: return to the consumer selection step and adjust the consumer list.
Edit a quota rule
On the Consumer Quota list page, find the target quota rule and click Edit Quota Rule in the Actions column.
In edit mode, you cannot modify Reset Period or Time Zone.
Use the transfer box to add consumers to or remove consumers from the consumer list.
After you submit the edit, the system runs the same conflict detection as during rule creation.
The page returns to the quota rule list and a success message appears at the top of the page.
Enable or disable a quota rule
On the Consumer Quota list page, find the target quota rule and select Active or Disable in the Actions column.
After you enable the rule, the rule takes effect immediately and a success message appears. The status of the rule in the list changes to a green Enabled indicator.
After you disable the rule, the rule stops taking effect immediately and a success message appears. The status of the rule in the list changes to a gray Disabled indicator.
Reset a quota rule
On the Consumer Quota list page, find the target quota rule and click Quota Reset in the Actions column.
A reset clears the historical quota usage of the rule, but does not clear the log usage records. After the reset, the system runs conflict detection again. If a conflict is reported, follow the on-screen instructions to resolve it. The page then returns to the quota rule list, a success message appears at the top of the page, and the usage statistics of the rule are reset to zero.
Delete a quota rule
On the Consumer Quota list page, find the target quota rule and click Delete in the Actions column. Enter the rule name to confirm the deletion. The rule is then removed from the list and a success message appears at the top of the page.
View usage statistics
Click a rule name on the Consumer Quota page to go to the rule details page. The page displays Usage Statistics by default.
Left panel: consumer list
Lists the consumers that are bound to this quota rule. You can search the list by consumer name.
Each consumer shows its name and ID, the ID and name of the instance that it belongs to, and a usage progress bar (used amount / total limit).
A consumer is marked as Over Limit when its usage exceeds the limit, and as Within Limit when its usage is within the limit.
Right panel: usage details of the selected consumer
After you select a consumer, the right panel displays the usage details of that consumer. You can view both Token and Credits statistics.
The following usage data and records are available:
Token Usage/Credits Usage: used amount / total limit.
Input Token/Input Credits: inbound consumption in the current cycle.
Output Token/Output Credits: outbound consumption in the current cycle.
Cached Token/Cached Credits: cached consumption in the current cycle.
Consumption Records table: shows the consumption details of the current cycle, with columns such as Model Name, Input, Output, Usage, and Consumption Time.
Pagination: 5 or 10 records per page.
Rule details
This tab displays the configuration information for the quota rule:
Basic Information:
Quota: The configured limit value with its unit (Tokens or Credits).
Status: Whether the rule is enabled or disabled.
Rule Type: Token Quota or Credits Quota.
Effective Period:
Rule Type: Period Quota.
Reset Period: Daily, Weekly, or Monthly.
Time Zone: The configured timezone.
A list of consumers associated with this rule is also displayed on this tab, showing each consumer's current quota usage.
You can also perform Edit Quota Rule, Quota Reset, Enable / Disable, and Delete operations from buttons at the top of the details page.