Windows network card driver performance tuning guide

Symptoms and causes

Typical symptoms: network monitoring occasionally shows packet loss, and the TCP retransmission rate rises briefly, but sustained stress tests may not reproduce the issue. Packet loss often coincides with traffic bursts or periods when the system is busy.

One common cause: the CPUs in the instance cannot process received data in time. When the instantaneous receive rate exceeds the processing speed of the driver, the receive buffers of the NIC gradually fill up. After the buffers are exhausted, newly arriving packets have nowhere to be stored and can only be dropped. After the burst passes, the CPUs catch up and the buffers are freed, and the packet loss disappears. This is why the issue is intermittent and hard to reproduce.

How to identify

Before tuning parameters, use the process of elimination to determine whether the packet loss may be caused by receive buffer exhaustion. This avoids blind parameter tuning when the cause of the packet loss is unclear:

• Rule out other stages: First confirm that the packet loss does not occur at the network link layer or on the peer side (for example, drops by security groups or bandwidth throttling). If other stages can be ruled out, receive buffer exhaustion in the instance is one of the most likely causes;

• Check time correlation: Compare the times of packet loss with the CPU utilization and network traffic curves of the instance to confirm whether the packet loss coincides with high CPU load or traffic bursts;

• Check CPU distribution: Observe the utilization of each CPU core and the distribution of receive processing to confirm whether receive processing is concentrated on a few CPU cores when packet loss occurs;

• Increase-and-verify (final confirmation): If the preceding clues point to buffer exhaustion, increase RxCapacity and observe whether the packet loss disappears. If it disappears, buffer exhaustion can be confirmed as the cause.

If the initial assessment points to delayed processing on the receive side, tune along the following two paths.

Tuning path 1: Increase RxCapacity (absorb bursts)

Principle: a larger receive buffer (for example, increased from the default 256 to 1024 or 2048) can cache more packets during transient bursts while waiting for CPU processing, which prevents drops caused by buffer exhaustion. In Device Manager, this parameter is displayed as Init.MaxRxBuffers.

Usage boundaries (important):

• Effective only for transient bursts: In the case of sustained overload (the CPU processing capacity is persistently insufficient), increasing the buffer only delays packet loss rather than eliminating it. You need to combine this with tuning path 2 (engage more CPUs in receive processing) or upgrade the instance;

• Confirm the effective upper limit first: Not all instances can use values greater than 256. This depends on the instance type and image version. For details, see Effective upper limit. Confirm that the instance meets the requirements before you increase this value;

• Estimate the memory overhead beforehand: The memory consumed by the receive buffers is proportional to the value and the number of NIC queues (the number of NIC queues is determined by the instance type and cannot be configured by the user). When the value is 4096, multi-queue instances can consume GB-level memory. For quantitative estimation, see Memory overhead. For instances with small memory, estimate the overhead before tuning.

Tuning path 2: Increase the number of RSS queues (engage more CPUs in receive processing)

Principle: RSS distributes received traffic across multiple receive queues by network connection, and different CPUs process the queues in parallel. On multi-core instances, increasing *NumRssQueues (for example, from the default 8 to a value close to the number of vCPUs) prevents the receive load from being concentrated on a few cores and improves overall receive processing capacity. In Device Manager, this parameter is displayed as Maximum Number of RSS Queues.

Prerequisites and boundaries (important):

• *RSS (displayed as Receive Side Scaling in Device Manager) must remain enabled (enabled by default);

• The configured value may differ from the effective value: The effective number of queues does not exceed the number of vCPUs of the instance. After configuration, check the RSS status information of the operating system to confirm the effective number of queues;

• Not effective for a single high-traffic connection: The packets of the same connection (single flow) are always processed by the same CPU. Adding queues cannot distribute the load of a single flow. This path applies to multi-flow concurrent scenarios. For single-flow packet loss, prefer tuning path 1 or split the connections at the application layer.

Decision recommendations

The preceding table also applies to preventive tuning for high-throughput/high-PPS scenarios: RxCapacity can be increased, and *NumRssQueues can be set to a value close to the number of vCPUs. Keep the other offload parameters (checksum/LSO/RSS/RSC) at their default values (all enabled by default).

Effective upper limit

On Alibaba Cloud ECS, whether the number of receive buffers can use values greater than 256 (up to 4096) depends on the instance type and image version. Support for 4096 is guaranteed when both of the following conditions are met:

For instances that do not meet the preceding conditions, the effective upper limit may be 256. Even if Device Manager allows you to select a larger value (such as 1024 or 4096), values beyond the effective upper limit do not take effect.

Memory overhead

Each receive buffer consumes about 72 KB of memory. This memory is fixed when the driver is loaded and does not vary with traffic. It is also unrelated to the MTU/*JumboPacket settings. Receive buffers are allocated per NIC queue (the number of NIC queues is determined by the instance type and cannot be configured by the user, and is unrelated to *NumRssQueues). Therefore:

Total memory of receive buffers ≈ 72 KB × effective number of buffers × number of NIC receive queues

Estimation examples (based on the effective number of buffers):

This parameter has a significant impact on memory consumption. The default value of 256 is a balance between memory usage and burst absorption capability. If you increase RxCapacity on instances with small memory, pay attention to the memory overhead (on instances with multiple NIC queues, memory consumption multiplies with the number of queues). Estimate against the preceding table based on the memory specification of the instance before tuning.

How to configure parameters

You can configure the parameters in Device Manager. Procedure:

1. Open Device Manager > Network adapters and find the virtio NIC;

2. Right-click the NIC and choose Properties > the Advanced tab;

3. In the Property list on the left, select a parameter (such as Init.MaxRxBuffers), change the value on the right, and click OK.

How changes take effect

The driver reads parameter settings only once during NIC initialization. Therefore, you must restart the NIC after modifying parameters. After you click OK in Device Manager to save the changes, the system automatically restarts the NIC, interrupting the network for a few seconds.

Parameter quick reference table

The NetKVM driver provides the following 24 configurable parameters:

Parameters by category