Methods for stress testing NLB

Stress test topologies

This section describes the standard stress test topology and the single virtual IP address (VIP) stress test topology. The standard stress test process is used in most scenarios. You can use the single-VIP stress test process in specific scenarios. For more information, see Stress test methods and Possible causes of low performance on stress tests.

• Standard stress test topology

  

• Single-VIP stress test topology

  

Stress test methods

Stress test metrics

NLB has three key metrics: New Connections, Concurrent Connections, and data volume (requests and responses). Each metric requires a different stress test method.

• You can use short-lived connections to test the New Connections metric. This tests the new connection processing capacity of the Server Load Balancer service and backend servers.

  You can use a simple heartbeat service for the test to reduce the impact of high bandwidth consumption. When you test with short-lived connections, be aware of the potential for insufficient client ports.

• You can use persistent connections to test the Concurrent Connections metric. This tests the concurrent processing capacity of the Server Load Balancer service and backend servers.

  For each persistent connection, you can use a simple heartbeat service test to keep the session alive. When you use persistent connections to test high concurrency, be aware of the potential for insufficient client ports.

• You can use persistent connections to test the data volume. This is used to test the bandwidth limit or specific business services.

  Set a short timeout period in the stress test tool, such as 5 seconds. A long timeout period increases the average response time in the test results, which makes it difficult to determine if the stress level has been reached. A shorter timeout period is reflected in the success rate, making it easier to quickly determine the stress level.

Recommendations for listener configurations

When you perform a stress test, follow these recommendations for listener configurations.

Recommendations for server group configurations

When you perform a stress test, follow these recommendations for server group configurations.

• Do not use the consistent hashing algorithm that is based on source IP addresses. This prevents the load from concentrating on a few backend servers. If you require session persistence, select four-tuple hashing (source IP address, destination IP address, source port, and destination port) to increase hash value diversity. For more information about scheduling algorithms, see Introduction to Server Load Balancer scheduling algorithms.

• You can disable the health check feature to reduce access requests to backend servers.

• If you enable the Preserve Client IP feature, use the single-VIP stress test topology. If you use the standard stress test topology with a small number of clients or with the consistent hashing algorithm based on source IP addresses, the likelihood of session conflicts on backend servers increases.

• When the Preserve Client IP feature is disabled, the NLB instance uses its own IP address to connect to backend servers. The system provides a limited number of source addresses for each NLB instance. Ensure that there are enough backend servers for the stress test to prevent the number of concurrent connections from being limited.

Recommended stress test tools

• You can use Alibaba Cloud PTS. It lets you set a high level of concurrency. PTS allocates public IP addresses from various regions, which ensures that the load source is distributed. You can also integrate Cloud Monitor with PTS to view all end-to-end performance data in real time.

• Do not use Apache ab as a stress test tool. In high-concurrency scenarios, Apache ab experiences stepped pauses at 3, 6, and 9 seconds. Apache ab determines request success by checking the content length. If an NLB instance has multiple backend servers attached, the returned content lengths may be inconsistent, which causes inaccurate test results.

Example of a simple stress test using PTS

Create an NLB instance. Add a TCP listener on port 80 and a TCP/SSL listener on port 443 to the NLB instance. Add two ECS instances as backend servers to both the TCP listener and the TCP/SSL listener. Set the backend port to 80 for both.

Possible causes of low performance on stress tests

If you find that the performance on a stress test is low, it may be for one of the following reasons:

• Insufficient client ports

  During a stress test, insufficient client ports can cause connection establishment to fail. By default, Server Load Balancer removes the timestamp property of TCP connections. This prevents the `tw_reuse` feature of the Linux protocol stack (reusing connections in the TIME_WAIT state) from working. The resulting connection buildup in the TIME_WAIT state leads to insufficient client ports.

  Solution: On the client, use persistent connections instead of short-lived connections. You can use RST messages to disconnect by setting the SO_LINGER socket property.

• Full accept queue on backend servers

  If the accept queue on a backend server is full, the server does not reply with SYN-ACK messages, and the client times out.

  Solution: The default value of `net.core.somaxconn` is 128. You can run the sysctl -w net.core.somaxconn=1024 command to change the value, and then restart the application on the backend server.

• Small number of backend servers

  If you disable the preserve client IP feature for the server group, NLB uses its own address to connect to backend servers. By default, one backend server supports 120,000 concurrent connections. If the number of backend servers is small, the performance of the concurrent connection stress test is also limited.

  Solution: For a TCP or UDP listener, you can enable the preserve client IP feature for the server group. For a TCP/SSL listener, you can increase the number of NLB backend servers or use Auto Scaling.

• Application dependencies on backend servers become a bottleneck

  After requests pass through the load balancer to the backend servers, the load on the backend servers themselves may be normal. However, the applications on all backend servers depend on other applications, such as a database. If the database becomes a bottleneck, performance also decreases.

  Solution: As needed, you can clean up unnecessary applications on the backend servers.

• Abnormal health check status of backend servers

  During a stress test, it is easy to overlook the health check status of backend servers. If a backend server fails a health check, or if the health check status frequently changes from Normal to abnormal and back again, it can also lead to low performance on the stress test.

  Solution: You can disable the health check feature to reduce health check requests to the backend servers.

• Session conflicts on backend servers

  If you use the standard stress test topology with a small number of clients or with the consistent hashing algorithm based on source IP addresses, the possibility of session conflicts on backend servers increases. This results in backend servers frequently using RST messages to close connections, as shown in the following figure.

  

  Solution: You can use the single-VIP stress test topology. For a diagram of the single-VIP stress test topology, see Stress test topologies.