AI token rate limiting

Top-level fields

Rule item fields

Each object in rule_items specifies a rate limiting key source and its token limits. Configure exactly one limit_by_* field per rule item.

Exact-match key sources apply a single shared quota to all requests that carry the specified key value:

Per-key sources maintain a separate quota for each distinct key value. Use limit_keys to define matching patterns, which support regular expressions and wildcards (*):

Each rule item also requires:

Limit key fields

Each object in limit_keys defines a key pattern and its token quota.

Configure exactly one token_per_* field per limit key.

Regular expression example: regexp:^d.* matches all key values that start with d.

Redis fields

Rate limit by URL parameter

Rate limit requests based on the apikey URL parameter. The first rule applies exact-match quotas to two specific API keys. The second rule uses per-key matching with regular expressions and a wildcard fallback.

rule_name: default_rule
rule_items:
  - limit_by_param: apikey
    limit_keys:
      - key: 9a342114-ba8a-11ec-b1bf-00163e1250b5
        token_per_minute: 10
      - key: a6a6d7f2-ba8a-11ec-bec2-00163e1250b5
        token_per_hour: 100
  - limit_by_per_param: apikey
    limit_keys:
      # Match keys starting with "a": 10 tokens per second each
      - key: "regexp:^a.*"
        token_per_second: 10
      # Match keys starting with "b": 100 tokens per minute each
      - key: "regexp:^b.*"
        token_per_minute: 100
      # Fallback for all other keys: 1,000 tokens per hour each
      - key: "*"
        token_per_hour: 1000
redis:
  service_name: redis.static

Rate limit by HTTP request header

Rate limit requests based on the x-ca-key header.

rule_name: default_rule
rule_items:
  - limit_by_header: x-ca-key
    limit_keys:
      - key: 102234
        token_per_minute: 10
      - key: 308239
        token_per_hour: 10
  - limit_by_per_header: x-ca-key
    limit_keys:
      # Match keys starting with "a": 10 tokens per second each
      - key: "regexp:^a.*"
        token_per_second: 10
      # Match keys starting with "b": 100 tokens per minute each
      - key: "regexp:^b.*"
        token_per_minute: 100
      # Fallback for all other keys: 1,000 tokens per hour each
      - key: "*"
        token_per_hour: 1000
redis:
  service_name: redis.static

Rate limit by client IP address

Extract client IP addresses from the x-forwarded-for header and apply per-IP token quotas. Supports exact IP addresses and CIDR blocks.

rule_name: default_rule
rule_items:
  - limit_by_per_ip: from-header-x-forwarded-for
    limit_keys:
      # Exact IP: 10 tokens per day
      - key: 1.1.1.1
        token_per_day: 10
      # CIDR block: 100 tokens per day per IP
      - key: 1.1.1.0/24
        token_per_day: 100
      # Default for all other IPs: 1,000 tokens per day per IP
      - key: 0.0.0.0/0
        token_per_day: 1000
redis:
  service_name: redis.static

Rate limit by consumer

Rate limit requests based on consumer identity.

rule_name: default_rule
rule_items:
  - limit_by_consumer: ''
    limit_keys:
      - key: consumer1
        token_per_second: 10
      - key: consumer2
        token_per_hour: 100
  - limit_by_per_consumer: ''
    limit_keys:
      # Match consumers starting with "a": 10 tokens per second each
      - key: "regexp:^a.*"
        token_per_second: 10
      # Match consumers starting with "b": 100 tokens per minute each
      - key: "regexp:^b.*"
        token_per_minute: 100
      # Fallback for all other consumers: 1,000 tokens per hour each
      - key: "*"
        token_per_hour: 1000
redis:
  service_name: redis.static

Rate limit by cookie value

Rate limit requests based on the value of cookie key key1. This example also customizes the throttle response to return HTTP 200 with a JSON error body.

rule_name: default_rule
rule_items:
  - limit_by_cookie: key1
    limit_keys:
      - key: value1
        token_per_minute: 10
      - key: value2
        token_per_hour: 100
  - limit_by_per_cookie: key1
    limit_keys:
      # Match values starting with "a": 10 tokens per second each
      - key: "regexp:^a.*"
        token_per_second: 10
      # Match values starting with "b": 100 tokens per minute each
      - key: "regexp:^b.*"
        token_per_minute: 100
      # Fallback for all other values: 1,000 tokens per hour each
      - key: "*"
        token_per_hour: 1000
rejected_code: 200
rejected_msg: '{"code":-1,"msg":"Too many requests"}'
redis:
  service_name: redis.static