ARMS convergence mechanisms

Spring annotation convergence

Using the request URL as a dimension value works well for traditional web APIs, but for RESTful APIs or URLs that contain variables, raw request URLs lead to high-cardinality dimensions. For Spring Web applications, ARMS extracts annotation information (such as @RequestMapping) as the interface dimension value.

How it works

ARMS reads the path information from the Spring URL routing annotation.

Result

The value configured in the routing annotation.

Supported data types

URL: Applies only to URLs for services you provide. This mechanism does not apply to external URLs that your application calls.

Location

Agent

Supported agent versions

2.9.1.2 and later

Example

The following APIController uses a path variable to get user information. ARMS extracts the annotation (/api/v1/user/{userId}/info) as the interface name.

@RestController
@RequestMapping("/api/v1")
public class APIController {
    @RequestMapping("/user/{userId}/info")
    public String getUserInfo(@PathVariable("userId") String userId) {
        return "hello " + userId;
    }
}

Convergence result:

The URL /api/v1/user/1234/info is converged to /api/v1/user/{userId}/info .

Memory-based statistical convergence

For applications that do not use the Spring Web framework, or when annotation extraction fails, ARMS uses a memory-based statistical convergence algorithm.

How it works

1. Splits each input string into a set of terms by using predefined delimiters, such as / or =.

2. Counts the number of unique terms (the cardinality) at each position. If the count exceeds a configured threshold, the term at that position is replaced with *.

Result

A string that contains *.

Supported data types

Any

Location

Agent

Supported agent versions

2.x and later versions that are earlier than 4.x.

Example

Application A provides a user information endpoint with the format /api/v1/user/${userId}/info, where ${userId} is the user ID. Because of the large number of unique user IDs, the memory statistics module detects high cardinality in the ${userId} segment and converges it

to /api/v1/user/*/info.

Custom convergence

This feature lets you define custom rules for specific convergence needs.

How it works

ARMS matches the input against your custom rules sequentially. If a rule matches, the process stops.

Result

The result is determined by your custom configuration.

Supported data types

URL

Location

Agent 4.x and later: agent

Agent versions earlier than 4.x: server

Supported agent versions

All

Example

You configure a rule to match /api/v1/user/[\d]+/info and converge it to /api/v1/getUserInfo.

All requests that match the regular expression /api/v1/user/[\d]+/info are converged to /api/v1/getUserInfo.

Static resource convergence

ARMS does not normally monitor static resource requests. However, some older agents collect these metrics. Because static resources provide limited monitoring value and their URLs change frequently, convergence is enabled by default.

How it works

ARMS checks if the request URL suffix matches a predefined list of static file extensions. If a match is found, the URL is converged.

Default static file extensions: .log .7z .tgz .jpg .jpeg .png .gif .css .js .ico .woff2 .xml .svg .pdf .txt .text .ppt .word .xlsx .tar.gz .tar.bz2 .sh .yml .yaml .zip .log .gz .ttf .woff .eot .rar .properties

Result

The default result is {ARMS_STATIC_REQ}. If advanced options are enabled (submit a ticket to enable), the result includes the resource file extension.

Supported data types

URL

Location

Agent 4.x and later: agent

Agent versions earlier than 4.x: server

Supported agent versions

All

Example

By default, /api/v1/hello.jpg is converged to {ARMS_STATIC_REQ}. If advanced options are enabled, it is converged to {ARMS_S_JPG}.

Attack request convergence

Services may receive malicious requests, such as attempts to read the /etc/passwd file. These attack requests change frequently, and recording them all would strain storage.

How it works

ARMS checks if the URL contains potential attack characters. If so, the URL is converged.

Default attack characters: ` $ \ ' !

Result

{ARMS_ATTACK_REQ}

Supported data types

URL

Location

Agent 4.x and later: agent

Agent versions earlier than 4.x: server

Supported agent versions

All

Example

The URL /app/v1/user/info?cmd=`more /etc/passwd` is converged to {ARMS_ATTACK_REQ}.

Request parameter convergence

By default, the agent does not collect query parameters from URLs. However, collected URLs may sometimes contain query parameters, which are often unpredictable and can easily cause high cardinality.

How it works

ARMS checks if the URL contains query parameters. If it does, the URL is converged.

Default query parameter delimiters: ; ? &

Result

The default result is {ARMS_PARAMED_REQ}. If advanced options are enabled (submit a ticket to enable), the result retains the URL path and replaces the parameters with {ARMS_REQ_PARAMS}.

Supported data types

URL

Location

Agent 4.x and later: agent

Agent versions earlier than 4.x: server

Supported agent versions

All

Example

By default, /api/v1/user/info?userId=12345 is converged to {ARMS_PARAMED_REQ}. If advanced options are enabled, it is converged to /api/v1/user/info?{ARMS_REQ_PARAMS}.

Long term convergence

URLs with unusually long words or numbers can cause high cardinality. To minimize this, ARMS replaces excessively long terms by default.

How it works

ARMS splits the URL by / to create an array of terms. It then iterates through the array and checks if the length of each term exceeds the following thresholds. If it does, the term is replaced.

• Maximum word length: 64

• Maximum number length: 10

• Maximum length of digits within a word: 10

Result

• An excessively long number is converged to: {ARMS_NUMBER}

• An excessively long word is converged to: {ARMS_WORD}

• A word containing an excessively long number is converged to: {ARMS_ANY}

Supported data types

URL

Location

Agent 4.x and later: agent

Agent versions earlier than 4.x: server

Supported agent versions

All

Example

The URL /api/2024040710/hello2024040710 is converged to /api/{ARMS_NUMBER}/{ARMS_ANY}.

Intelligent convergence

Even after the preceding mechanisms are applied, high-cardinality URLs may still remain. For these URLs, ARMS automatically computes convergence rules and replaces the original values.

How it works

The logic is complex, but here is a brief overview:

1. Groups sample URLs based on algorithms.

2. Converges URL patterns within each group and generates convergence rules.

3. Merges the convergence rules from all groups.

Result

• High-cardinality parts consisting of only numbers are replaced with {ARMS_NUMBER}.

• High-cardinality parts consisting of only letters are replaced with {ARMS_WORD}.

• High-cardinality parts consisting of both letters and numbers are replaced with {ARMS_ANY}.

Supported data types

URL

Location

Agent 4.x and later: agent

Agent versions earlier than 4.x: server

Supported agent versions

All

Example

/api/product/1/info
/api/product/2/info
....
/api/product/N/info

After analysis, the server generates a convergence rule using a regular expression: /api/product/[\d]+/info.

The convergence result is /api/product/{ARMS_NUMBER}/info.

Subsequently, any request matching this regular expression is converged to /api/product/{ARMS_NUMBER}/info.

SQL normalization

The agent may collect many unique SQL statements due to database sharding, inline comments, or literal values. ARMS normalizes SQL statements by default, replacing parts that are likely to cause high cardinality.

How it works

The logic is complex, but here is a brief overview:

1. Removes comments.

2. Replaces literal values.

3. Replaces sharded database and table names.

...

Result

The SQL statement after the high-cardinality parts have been replaced.

Supported data types

SQL

Location

Agent

Supported agent versions

4.x and later

Example

select * from cache_0 where ckey='23'

This is normalized to:

select * from cache_{NUM} where ckey=?

IP address convergence

If your application accesses many external services directly by IP address, the agent may collect a large number of unique IP addresses, causing high cardinality.

How it works

1. Groups IP addresses by port.

2. Counts the number of IP addresses for each port. If the number exceeds a threshold, convergence is applied. Default threshold: 50

Result

{ARMS_IP}:port

Supported data types

IP

Location

Agent 4.x and later: agent

Agent versions earlier than 4.x: server

Supported agent versions

All

Example

1.1.1.1:8080
...
1.1.1.255:8080

These are converged to {ARMS_IP}:8080.

Cardinality space convergence

The preceding mechanisms resolve most high-cardinality problems for URL dimensions. However, for types like SQL, a large number of unique dimension values may persist even after other convergence steps. ARMS addresses this by limiting the number of dimension values recorded within a time period.

How it works

The logic is complex, but here is a brief overview:

1. Periodically generates a cardinality space of a fixed size.

2. For each dimension value, ARMS checks if it is in the cardinality space. If the value exists, it is used. If not, ARMS tries to add it. If the space has capacity, the value is added and used; otherwise, it is converged to {ARMS_OTHERS}.

Result

Dimension values that exceed the space limit are converged to {ARMS_OTHERS}.

Default thresholds

Supported data types

Any

Location

Agent 4.x and later: agent

Agent versions earlier than 4.x: server

Supported agent versions

All

Example

Metrics can record external service addresses. Assume the cardinality space allows 100 records per hour and the addresses are as follows:

www.a1.com
www.a2.com
....
www.a1000.com

In this case, only the first 100 unique external service addresses are recorded each hour. Subsequent addresses are converged to {ARMS_OTHERS}.

Agent versions earlier than 4.x

Agent side

• URL

  Spring annotation convergence > Memory-based statistical convergence

• SQL

  Memory-based statistical convergence

• IP and other types

  Memory-based statistical convergence

Server side

• URL

  Custom convergence > Attack request convergence > Request parameter convergence > Static resource convergence > Long term convergence > Intelligent convergence > Cardinality space convergence

• SQL

  Custom convergence > Cardinality space convergence

• IP and other types

  Custom convergence > IP address convergence > Cardinality space convergence

Agent 4.x and later