The ARMS agent for Java uses bytecode enhancement at runtime to provide application performance management. Like other bytecode-based solutions, the agent introduces some performance overhead. This report measures the CPU, memory, and response time overhead of the ARMS agent for Java V3.x under different traffic loads, helping you evaluate the impact before you connect your application to Application Monitoring.
Test scenario
Flowchart
Built on Spring Web MVC, the Java application accesses MySQL and Redis based on requests from Alibaba Cloud Performance Testing Service (PTS). The ${mall-gateway}/case/api/v1/mysql/execute request accesses MySQL, and the ${mall-gateway}/case/api/v1/redis/execute request accesses Redis. Each request type accounts for 50% of the total QPS.
Test environment
-
Stress testing is performed by PTS.
-
The Java application, MySQL, and Redis are deployed in the same Alibaba Cloud Container Service for Kubernetes (ACK) cluster. Nodes use the ecs.c6.2xlarge instance type and run CentOS Linux 7.9.2009 (Core).
-
Each pod of the application has 2 cores, 4 GB of memory, and two replicas.
-
The ARMS agent for Java version is 3.1.4.
Test procedure
-
Perform three stress tests at 500, 1,000, and 2,000 QPS. Each stress test lasts for 1 hour. Before each stress test, warm up the Java application at 100 QPS for 3 minutes. The test result will be used as baseline performance metrics.
-
Install the ARMS agent for Java V3.1.4. Configure a sampling rate of 10% in the sampling policy, and repeat the stress tests in Step 1 to compare the differences in the CPU overhead, memory overhead, and response time of the Java application.
-
Install the ARMS agent for Java V3.1.4. Configure a sampling rate of 100% in the sampling policy, and repeat the stress tests in Step 1 to compare the differences in the CPU overhead, memory overhead, and response time of the Java application.
-
For more information about how to configure the sampling policy, see Trace sampling modes for agents before 3.2.8.
-
Basic Application Monitoring features (metrics, traces, and quantiles) and all plug-ins are enabled.
Baseline performance metrics
|
Item |
CPU |
Memory |
Response time (Unit: milliseconds) |
|
500 QPS |
6.50% |
14.49% |
2 |
|
1,000 QPS |
17.01% |
14.86% |
2 |
|
2,000 QPS |
32.36% |
19.04% |
2 |
-
CPU: the percentage of pod CPU usage relative to the total CPU.
-
Memory: the percentage of pod memory usage relative to the total memory. Because pod memory usage grows naturally until it reaches the requests value, this report uses the actual memory usage at the end of each stress test.
-
Response time: the average response time of requests, in milliseconds.
Performance metrics of the ARMS agent for Java V3.x
|
Item |
10% sampling rate |
100% sampling rate |
||||
|
CPU |
Memory |
Response time (Unit: milliseconds) |
CPU |
Memory |
Response time (Unit: milliseconds) |
|
|
500 QPS |
7.72% |
18.37% |
2 |
7.90% |
18.83% |
2 |
|
1,000 QPS |
19.94% |
18.95% |
2 |
20.78% |
19.61% |
2 |
|
2,000 QPS |
40.49% |
25.69% |
3 |
42.25% |
25.80% |
3 |
Performance overhead of the ARMS agent for Java V3.x
|
Item |
10% sampling rate |
100% sampling rate |
||||
|
CPU |
Memory |
Response time (Unit: milliseconds) |
CPU |
Memory |
Response time (Unit: milliseconds) |
|
|
500 QPS |
+1.21% |
+3.88% |
- |
+1.39% |
+4.35% |
- |
|
1,000 QPS |
+2.92% |
+4.09% |
- |
+3.76% |
+4.75% |
- |
|
2,000 QPS |
+8.13% |
+6.66% |
+1 |
+9.89% |
+6.76% |
+1 |
Conclusion
-
The additional CPU and memory overhead caused by the ARMS agent for Java V3.x are within 10%.
-
The ARMS agent for Java V3.x has small impact on the response time, which increases only by 1 millisecond at 2,000 QPS.
-
The performance overhead at a 100% fixed sampling rate are slightly greater than those at a 10% fixed sampling rate.