The Application Real-Time Monitoring Service (ARMS) agent for Python uses monkey patching to add observability, which introduces some performance overhead. This report measures the CPU, memory, and response-time overhead of the ARMS agent for Python under different traffic loads. Review these results before you connect your application to ARMS Application Monitoring.
Test scenario
Flowchart
Performance Testing Service (PTS) directs traffic to a Python application built on the FastAPI framework. The Python instrumentor collects traces and generates metrics, then forwards the data to the ARMS server.
Test environment
-
The stress testing source is provided by PTS.
-
The Python application is deployed in an Alibaba Cloud Container Service for Kubernetes (ACK) cluster. The node instance type is ecs.c6.2xlarge and the operating system is Alibaba Cloud Linux 3.2104 LTS 64-bit.
-
Each application pod has 2 cores, 2 GB of memory, and two replicas.
-
ARMS agent for Python V1.0.0 is used.
Test procedure
-
Perform three stress tests at 500, 1,000, and 2,000 QPS. Each stress test lasts for 1 hour. Before each stress test, warm up the Python application at 100 QPS for 3 minutes. The test result will be used as baseline performance metrics.
-
Install the ARMS agent for Python. Configure a fixed sampling rate of 10% in the sampling policy, and repeat the stress tests in Step 1 to compare the differences in the CPU overhead, memory overhead, and response time of the Python application.
-
Disable data reporting, enable the plug-in, and repeat steps 1 to 2.
Baseline performance metrics
|
Item |
CPU |
Memory |
Response time (Unit: milliseconds) |
|
500 QPS |
8% |
2% |
17.28 |
|
1,000 QPS |
15% |
2% |
36.3 |
|
2,000 QPS |
27.12% |
2% |
44.9 |
Performance metrics of the ARMS agent for Python
|
Item |
CPU |
Memory |
Response time (Unit: milliseconds) |
|
500 QPS |
11.9% |
4.8% |
17.31 |
|
1,000 QPS |
22.5% |
4.8% |
37.1 |
|
2,000 QPS |
36.67% |
4.8% |
46.6 |
Performance overhead of the ARMS agent for Python
|
Item |
CPU |
Memory |
Response time (Unit: milliseconds) |
|
500 QPS |
+3.9% |
+2.8% |
+0.03 |
|
1,000 QPS |
+7.5% |
+2.8% |
+0.80 |
|
2,000 QPS |
+9.55% |
+2.8% |
+1.67 |
Conclusions
-
The ARMS agent for Python increases CPU and memory overhead by no more than 10%.
-
The ARMS agent for Python has minimal impact on response time.