Application Monitoring in Application Real-Time Monitoring Service (ARMS) provides the following alert metrics. All metric data is collected at one-minute intervals.
JVMs
The following JVM metrics are for reference only. For authoritative details, refer to the official JVM documentation.
Metrics
|
Metric name |
Unit |
Commonly used |
Description |
|
Number of JVM full GCs (instantaneous value) |
None |
Yes |
The number of full garbage collections (GCs) the JVM performed in the last N minutes. Frequent full GCs may indicate errors in your application. |
|
JVM full GC duration (instantaneous value) |
Milliseconds |
No |
The time spent on full GCs in the last N minutes. Shorter full GC durations indicate better JVM garbage collection performance. Prolonged full GCs can cause significant application stuttering and degrade user experience. |
|
Number of JVM young GCs (instantaneous value) |
None |
Yes |
The number of young GCs the JVM performed in the last N minutes. A high young GC count indicates rapid object creation and destruction in the young generation, and may suggest memory leaks or inefficient memory usage. |
|
JVM young GC duration (instantaneous value) |
Milliseconds |
No |
The time spent on young GCs in the last N minutes. Longer young GC durations indicate lower garbage collection efficiency, which can cause application stuttering. |
|
Total JVM heap memory |
MB |
No |
The total JVM heap memory, including young and old generations. Heap size should be configured based on application load and performance requirements. Too small a heap causes frequent garbage collections; too large a heap consumes excessive system resources and affects stability. |
|
Used JVM heap memory |
MB |
Yes |
The JVM heap memory used by Java programs. Monitor this metric closely to prevent performance degradation or memory overflow caused by leaks or excessive usage. |
|
Committed JVM non-heap memory |
MB |
No |
The non-heap memory used by Java programs. Monitor this metric to prevent excessive memory usage caused by class loading or large numbers of static variables and constants. |
|
Initial JVM non-heap memory |
MB |
No |
The initial JVM non-heap memory size, dynamically calculated based on JVM version, operating system, and JVM parameters. |
|
Maximum JVM non-heap memory |
MB |
No |
If you are using a Java version earlier than 8, this metric is controlled by the JVM parameter MaxPermSize. Otherwise, this metric is controlled by MaxMetaspaceSize. |
|
Used JVM non-heap memory |
MB |
Yes |
The size of used JVM non-heap memory, including Metaspace and PermGen. |
|
Used JVM metaspace |
MB |
No |
The space that stores class metadata, including the class structure, methods, and fields. Generally, this memory usage is stable. |
|
Number of JVM blocked threads |
None |
No |
The number of blocked threads waiting for monitor locks. Excessive blocked threads may cause system performance degradation. |
|
Total number of JVM threads |
None |
Yes |
The number of threads in all states. Too many threads can exhaust memory and CPU resources, affecting application performance and stability. |
|
Number of JVM deadlocked threads |
None |
No |
The number of deadlocked threads. A deadlock occurs when two or more threads wait for each other to release a resource. Increasing deadlock counts indicate a worsening situation that can cause the application to crash. |
|
Number of new JVM threads |
None |
No |
The number of threads created by the JVM. Excessive threads can waste system resources and increase thread scheduling pressure. |
|
Number of JVM runnable threads |
None |
No |
The maximum number of threads the JVM supports at runtime. Creating too many threads consumes excessive memory and can cause the system to slow down or crash. |
|
Number of JVM terminated threads |
None |
No |
The number of concurrently running threads in the JVM. Control this count based on actual needs to prevent thread resource waste or starvation. |
|
Number of JVM timed-out waiting threads |
None |
Yes |
The number of threads that timed out while waiting for a resource. A high count indicates potential system bottlenecks. Optimize resources to improve processing capacity and response speed. |
|
Number of JVM waiting threads |
None |
No |
The number of waiting threads in the JVM. In high-concurrency applications, an increase in waiting threads can degrade performance. |
|
Number of JVM GCs (cumulative value) |
None |
No |
The cumulative number of GCs performed in the JVM. |
|
JVM mark-and-sweep garbage collection cycles (cumulative value) |
None |
No |
The cumulative number of mark-and-sweep garbage collection cycles in the JVM. |
|
JVM heap memory usage (%) |
None |
No |
The ratio of allocated heap memory to total heap memory at JVM runtime. Keep this metric below 70% to avoid memory overflow. |
Dimension and filter condition
The preceding metrics are monitored by node IP address. You can use one of the following methods to filter IP addresses:
-
Traversal: traverses the IP address of each node and configures alerting for the metric data of each node.
-
Equals (=): specifies specific nodes for alerting. Example: =172.20.XX.XX.
-
No dimension: aggregates and configures alerting for the metric data of all nodes.
Scheduled tasks
ARMS application monitoring supports only XXL-JOB, SchedulerX, and JDK-Timer task types.
Metrics
|
Metric name |
Unit |
Commonly used |
Description |
|
Duration |
Milliseconds |
No |
The average duration of the scheduled task. |
|
Total number of executions |
None |
No |
The number of times that the scheduled task was executed. |
|
Number of execution errors |
None |
No |
The number of times that the scheduled task was not executed as expected within the specified time interval. |
|
Scheduling latency |
Milliseconds |
No |
The time spent on scheduling before the scheduled task was started. |
Dimension and filter condition
The preceding metrics are monitored by scheduled task. You can use one of the following methods to filter scheduled tasks:
-
Traversal: traverses the scheduled tasks and configures alerting for the metric data of each scheduled task.
-
Equals (=): specifies specific scheduled tasks for alerting. Example: =LoadGenerator.mockUserApiLoad.
-
No dimension: aggregates and configures alerting for the metric data of all scheduled tasks.
Exceptions
Metrics
|
Metric name |
Unit |
Commonly used |
Description |
|
Number of exceptions |
None |
Yes |
The number of runtime exceptions, such as null pointer exceptions, array out-of-bounds exceptions, and I/O exceptions. Use this metric to identify call stack errors and application call failures. |
|
Response time of abnormal interface calls |
Milliseconds |
Yes |
The response time of abnormal interface calls. Use this metric to assess how call stack errors affect interface response times. |
Dimensions and filter condition
The preceding metrics are monitored by interface name. You can use one of the following methods to filter interface names:
-
Traversal: traverses the accessed interfaces and configures alerting for the metric data of each interface.
-
Equals (=): specifies specific interfaces for alerting. Example: =/tb/api/users/{userId}.
-
Not Equals (!=): excludes specific interfaces from alerting, and separately configures alerting for other interfaces. Example:!=/tb/api/users/{userId}
-
Contains: configures alerting for interfaces that contain a specific keyword. Example: Contains api.
-
Does Not Contain: configures alerting for interfaces that do not contain a specific keyword. Example: Does Not Contain api.
-
Regular expression: configures alerting for interfaces that match the specified regular expression. Example: =/(api)/i.
-
No dimension: aggregates and configures alerting for the metric data of all interfaces.
The preceding metrics are monitored by exception. You can use one of the following methods to filter exceptions:
-
Traversal: traverses the exceptions and configures alerting for the metric data of each exception.
-
Equals (=): specifies specific exceptions for alerting. Example: =FeignException$InternalServerError.
-
Not Equals (!=): excludes specific exceptions from alerting, and separately configures alerting for other exceptions. Example:!=FeignException$InternalServerError.
-
Contains: configures alerting for exceptions that contain a specific keyword. Example: Contains data.
-
Does Not Contain: configures alerting for exceptions that do not contain a specific keyword. Example: Does Not Contain data.
-
Regular expression: configures alerting for exceptions that match the specified regular expression. Example: =/(data)/i.
-
No dimension: aggregates and configures alerting for the metric data of all exceptions.
Application dependency services
Metrics
|
Metric name |
Unit |
Commonly used |
Description |
|
Number of application dependency service calls |
None |
No |
The number of calls to downstream interfaces that the application depends on. Use this metric to monitor call volume changes. |
|
Application dependency service call error rate (%) |
None |
No |
Calculated as: number of abnormal downstream interface requests / total number of interface requests. Use this metric to detect increasing dependency service errors that may affect the application. |
|
Response time of application dependency service calls |
Milliseconds |
Yes |
The average response time of downstream interfaces the application depends on. Use this metric to detect increasing latency in dependency services. |
|
Number of slow calls of an application dependency service |
None |
No |
Calls to downstream interfaces that exceed the response time threshold are counted as slow calls. Use this metric to detect dependency services affecting application performance. |
Dimension and filter condition
The preceding metrics are monitored by interface call type. You can use one of the following methods to filter interface call types:
-
Traversal: traverses the interface call types and separately configures alerting for the metric data of each type, such as HTTP, MySQL, and Redis.
-
Equals (=): specifies specific interface call types for alerting. Example: =http.
-
No dimension: aggregates and configures alerting for the metric data of all interfaces.
ECS instances
Metrics
|
Metric name |
Unit |
Commonly used |
Description |
|
Node CPU utilization (%) |
None |
No |
The CPU utilization of the node (server). High CPU utilization can cause slow system response and service unavailability. |
|
Node CPU utilization in user mode (%) |
None |
No |
The ratio of CPU time used by user-mode processes to total CPU time. User-mode processes include user-space applications such as web services and databases. |
|
Idle node disk space |
MB |
Yes |
The unused disk space of the node. If disk space runs out, the system may crash or fail to function properly. |
|
Node disk utilization (%) |
None |
No |
The ratio of used disk space to total disk space. Higher utilization means less remaining storage capacity. |
|
Node system load |
None |
Yes |
Indicates the node workload. For a node with N CPU cores, the maximum load value is N. |
|
Idle node memory |
MB |
Yes |
The unused memory of the node. Insufficient memory can cause out-of-memory (OOM) errors. |
|
Node memory usage (%) |
None |
No |
The percentage of memory in use. If usage exceeds 80%, reduce memory pressure by adjusting node configurations or optimizing task memory usage. |
|
Number of error packets received on the node |
None |
No |
The number of error packets received by the node during network communication, possibly caused by network transmission or application issues. Error packets may disrupt network communication and affect system operations. |
|
Number of error packets sent from the node |
None |
No |
The number of error packets sent by the node during network communication, possibly caused by network transmission or application issues. Use this metric to monitor node network health. |
|
Number of JVM instances |
None |
Yes |
The number of JVM instances running in real time. Generally, this metric is used to configure service downtime alerting. |
|
Number of bytes sent from the node |
None |
No |
The amount of data sent by the node over a network, including data, system messages, and error messages sent by the application. |
|
Number of packets sent from the node |
None |
No |
The number of messages sent from the node over a network. |
|
Number of bytes received on the node |
None |
No |
The total amount of data received by the node over a network. |
|
Number of packets received on the node |
None |
No |
The number of packets received by the node over a network. |
Dimension and filter condition
The preceding metrics are monitored by node IP address. You can use one of the following methods to filter IP addresses:
-
Traversal: traverses the IP address of each node and configures alerting for the metric data of each node.
-
Equals (=): specifies specific nodes for alerting. Example: =172.20.XX.XX.
-
No dimension: aggregates and configures alerting for the metric data of all nodes.
Containers
The ARMS agent v4.1.0 and later collects monitoring data about CPU and memory for monitoring and alerting.
Metrics
|
Metric name |
Unit |
Commonly used |
Description |
|
CPU utilization in user mode |
None |
No |
The CPU time spent executing code in user mode, including application code and library functions that do not run in kernel mode. |
|
CPU utilization in kernel mode |
None |
No |
The CPU time spent executing in kernel mode (system mode), including system calls, interrupt handling, and kernel-provided features. Reflects the OS resources consumed in servicing the process. |
|
Total CPU utilization |
None |
Yes |
The total CPU utilization is CPU utilization in user mode plus CPU utilization in kernel mode. |
|
Memory usage |
Bytes |
Yes |
The memory actively used by the container at runtime, including non-swappable memory and cached data still considered active. |
|
Number of sent network packets |
None |
No |
The number of packets sent from the container over a network. |
|
Number of sent bytes |
Bytes |
Yes |
The number of bytes sent from the container over a network. |
|
Number of sent error packets |
None |
No |
The number of error packets sent by the container during network communication, possibly caused by network transmission or application issues. Use this metric to monitor container network health. |
|
Number of sent discarded packets |
None |
No |
The total number of outbound network packets that have been dropped by the system or network stack since the container network interface was brought up. |
|
Number of received packets |
None |
No |
The number of messages received by the container over a network. |
|
Number of received bytes |
Bytes |
Yes |
The total amount of data received by the container over a network. |
|
Number of received error packets |
None |
No |
The number of error packets received by the container during network communication, possibly caused by network transmission or application issues. Error packets may disrupt communication and affect system operations. |
|
Number of received discarded packets |
None |
No |
The total number of inbound network packets that have been dropped by the system or network stack since the container network interface was brought up. |
Dimension and filter condition
The preceding metrics are monitored by node IP address. You can use one of the following methods to filter IP addresses:
-
Traversal: traverses the containers and configures alerting for the metric data of each container.
-
Equals (=): specifies specific containers for alerting. Example: =172.20.XX.XX.
-
No dimension: aggregates and configures alerting for the metric data of all containers.
Application providing services
Metrics
|
Metric name |
Unit |
Commonly used |
Description |
|
Number of calls |
None |
Yes |
The number of application entry point calls, including HTTP and Dubbo calls. Use this metric to analyze call volume, estimate business traffic, and detect application exceptions. |
|
Number of slow calls |
None |
No |
Entry point calls (HTTP and Dubbo) with response times exceeding the threshold are counted as slow calls. Use this metric to detect application exceptions. |
|
Number of error calls |
None |
Yes |
The number of error entry point calls, including HTTP and Dubbo calls. A call is counted as an error if an HTTP 400 status code is returned or the call is intercepted at the top layer of Dubbo. |
|
Call error rate (%) |
None |
Yes |
Calculated as: number of error entry point calls / total number of entry point calls × 100%. |
|
Call response time |
Milliseconds |
Yes |
The response time of application entry point calls, such as HTTP and Dubbo calls. Use this metric to identify slow requests and exceptions. |
Dimensions and filter condition
The preceding metrics are monitored by interface name. You can use one of the following methods to filter interface names:
-
Traversal: traverses the accessed interfaces and configures alerting for the metric data of each interface.
-
Equals (=): specifies specific interfaces for alerting. Example: =/tb/api/users/{userId}.
-
Not Equals (!=): excludes specific interfaces from alerting, and separately configures alerting for other interfaces. Example:!=/tb/api/users/{userId}
-
Contains: configures alerting for interfaces that contain a specific keyword. Example: Contains api.
-
Does Not Contain: configures alerting for interfaces that do not contain a specific keyword. Example: Does Not Contain api.
-
Regular expression: configures alerting for interfaces that match the specified regular expression. Example: =/(api)/i.
-
No dimension: aggregates and configures alerting for the metric data of all interfaces.
The preceding metrics are monitored by interface call type. You can use one of the following methods to filter interface call types:
-
Traversal: traverses the interface call types and separately configures alerting for the metric data of each type, such as HTTP, MySQL, and Redis.
-
Equals (=): specifies specific interface call types for alerting. Example: =http.
-
No dimension: aggregates and configures alerting for the metric data of all interfaces.
Thread pools
Metrics
|
Metric name |
Commonly used |
Description |
|
Number of core threads |
Yes |
The number of always active threads in the thread pool. |
|
Maximum number of threads |
Yes |
The maximum number of threads that can exist simultaneously in the thread pool. |
|
Number of active threads |
Yes |
The number of threads executing tasks. Use this metric to monitor thread pool status and evaluate performance. |
|
Queue size |
Yes |
The thread queue size, determined by application requirements and system resource availability. A queue that is too small causes long task waits and reduced performance; a queue that is too large can consume excessive resources and cause instability. |
|
Current number of threads |
Yes |
The number of threads running or waiting to run. |
|
Number of executed tasks |
Yes |
The number of completed tasks in the task queue or thread pool. Use this metric to evaluate task processing performance. |
|
Thread pool usage (%) |
Yes |
The ratio between the number of threads in use in the thread pool and the total number of threads in the thread pool. |
Dimensions and filter condition
The preceding metrics are monitored by node IP address. You can use one of the following methods to filter IP addresses:
-
Traversal: traverses the IP address of each node and configures alerting for the metric data of each node.
-
Equals (=): specifies specific nodes for alerting. Example: =172.20.XX.XX.
-
No dimension: aggregates and configures alerting for the metric data of all nodes.
The preceding metrics are monitored by thread pool name. You can use one of the following methods to filter thread pool names:
-
Traversal: traverses the thread pools and configures alerting for the metric data of each thread pool.
-
Equals (=): specifies specific thread pools for alerting. Example: =pool-*-thread-*.
-
No dimension: aggregates and configures alerting for the metric data of all thread pools.
The preceding metrics are monitored by thread pool type. You can use one of the following methods to filter thread pool types:
-
Traversal: traverses the thread pool types and configures alerting for the metric data of each thread pool type.
-
Equals (=): specifies specific thread pool types for alerting. Example: =FixedThreadPool.
-
No dimension: aggregates and configures alerting for the metric data of all thread pool types.
HTTP status codes
Metrics
|
Metric name |
Commonly used |
Description |
|
Number of HTTP requests with 4xx status codes |
Yes |
The number of HTTP requests that returned 4xx status codes, which indicate client errors such as missing resources or parameters. Common codes include 400 and 404. |
|
Number of HTTP requests with 5xx status codes |
Yes |
The number of HTTP requests that returned 5xx status codes, which indicate server-side errors such as internal failures or system overload. Common codes include 500 and 503. |
Dimension and filter condition
The preceding metrics are monitored by interface name. You can use one of the following methods to filter interface names:
-
Traversal: traverses the accessed interfaces and configures alerting for the metric data of each interface.
-
Equals (=): specifies specific interfaces for alerting. Example: =/tb/api/users/{userId}.
-
Not Equals (!=): excludes specific interfaces from alerting, and separately configures alerting for other interfaces. Example:!=/tb/api/users/{userId}
-
Contains: configures alerting for interfaces that contain a specific keyword. Example: Contains api.
-
Does Not Contain: configures alerting for interfaces that do not contain a specific keyword. Example: Does Not Contain api.
-
Regular expression: configures alerting for interfaces that match the specified regular expression. Example: =/(api)/i.
-
No dimension: aggregates and configures alerting for the metric data of all interfaces.
Databases
Metrics
|
Metric name |
Unit |
Commonly used |
Description |
|
Number of database requests |
None |
Yes |
The number of read or write requests the application sent to a database at runtime. Database request volume affects application performance and response time. |
|
Number of database request errors |
None |
Yes |
The number of database request errors at runtime, such as connection failures, query errors, and insufficient permissions. A high error count indicates abnormal database interactions that can prevent the application from running properly. |
|
Database request response time |
Milliseconds |
Yes |
The interval between sending a database request and receiving a response. Excessively long response times can cause application stuttering or slowdowns. |
|
Number of slow database requests |
None |
No |
Database requests with response times exceeding the threshold are counted as slow calls. A high number of slow calls directly affects application performance and user experience. |
Dimension and filter condition
The preceding metrics are monitored by database name. You can use one of the following methods to filter database names:
-
Traversal: traverses the databases and configures alerting for the metric data of each database.
-
Equals (=): specifies specific database types for alerting. Example: =mysql-pod:3306(demo_db).
-
No dimension: aggregates and configures alerting for the metric data of all databases.