NIC multi-queue

Why use NIC multi-queue

A traditional single-queue network interface relies on one CPU core to process all packets. This can lead to CPU overload, increased latency, and packet loss. Modern servers have multi-core CPUs, and NIC multi-queue distributes network traffic across these cores to use resources more efficiently.

Under the same network packets per second (PPS) and bandwidth conditions, tests show that using two queues instead of one improves network performance by 50% to 100%. Using four queues offers even greater performance gains:

• Better utilization of multi-core CPU architecture: Distributes network traffic across multiple CPU cores for a more balanced load and higher CPU utilization.

• Higher throughput: Processing multiple packets simultaneously significantly increases network throughput, especially under high loads.

• Lower latency: Reduces congestion and latency by distributing packets across multiple queues.

• Reduced packet loss: Mitigates packet loss caused by an overloaded single queue during periods of high traffic.

Although NIC multi-queue offers these benefits, improper configuration can decrease performance or cause other issues. For example, incorrect settings for the queue count or CPU affinity can cause unnecessary context-switching overhead. Setting the queue count too low can underutilize your hardware resources.

When an elastic network interface (ENI) is attached to an instance, its queue count is automatically set to the default for that instance type. If you need to manually adjust the queue count for the ENI, first carefully consider your specific use case and hardware conditions to determine the optimal configuration.

How NIC multi-queue works

• Queue architecture

  An ENI supports multiple queues, specifically multiple combined queues. Each combined queue is processed by an independent CPU core. This enables parallel packet processing, reduces lock contention, and fully utilizes multi-core performance.

  Receive (RX) and Transmit (TX) queues are the two types of queues used for processing packets. Each combined queue consists of one RX queue and one TX queue:

  • RX queue (RX Queue): An RX queue is used to process incoming data packets from the network. When a packet arrives, the network interface distributes it to a specific RX queue based on Receive Side Scaling (RSS) rules, such as round-robin or flow-based hashing strategies.

    Some instance type families support custom RSS for ENIs. This feature allows you to use hash algorithms to intelligently distribute network traffic to different RX queues. For more information, see Custom RSS for ENIs.

  • TX queue (TX Queue): A TX queue is used to manage outgoing data packets. An application places generated packets into a TX queue, and the network interface then sends them based on factors like sequence or priority.

• IRQ affinity support

  Each queue is associated with an independent interrupt. IRQ affinity distributes interrupt handling across different CPU cores to prevent any single core from being overloaded.

  IRQ affinity is enabled by default on all public images except for Red Hat Enterprise Linux. For more information, see Configure IRQ affinity.

Limitations

• Only some instance types support NIC multi-queue. For more information, see Instance type families. If the value in the Multi-queue column is greater than 1, the instance type supports NIC multi-queue.

  • For instance types that support NIC multi-queue, the feature is enabled automatically after you attach an ENI to an instance.

  • The queue count listed for an instance type represents the maximum queues per ENI supported by that type.

  • You can call the DescribeInstanceTypes API operation to query queue-related metrics for an instance type family by specifying the InstanceTypeFamily parameter:

    • Default queue count

      The PrimaryEniQueueNumber response parameter indicates the default queue count for the primary ENI. The SecondaryEniQueueNumber parameter indicates the default queue count for a secondary ENI.

    • Maximum queues per ENI

      The MaximumQueueNumberPerEni response parameter indicates the maximum queues per ENI allowed for the instance type family.

    • Total queue quota

      The TotalEniQueueQuantity response parameter indicates the total queue quota allowed for the instance type family.

• Some older public images with kernel versions earlier than 2.6 may not support NIC multi-queue. Use the latest public images.

View the ENI queue count

Modify the ENI queue count

When you attach an ENI to an instance, its queue count is automatically set to the default for that instance type. If you need to manually adjust the queue count, you can do so in the console or by calling an API operation. Note the following:

• You can modify an ENI's queue count only when the ENI is in the Available state, or when it is Bound and the attached instance is Stopped.

• The queue count cannot exceed the maximum queues per ENI for the instance type.

• The total queue count of all ENIs on an instance cannot exceed the total queue quota for the instance type.

After you modify the queue count for an ENI, the new setting takes effect the next time the attached instance starts.

Modify the OS queue count

When you modify the queue count for an ENI, the new setting is automatically applied within the OS, and the number of active queues matches the new count.

You can also tune the number of queues that the ENI actively uses. This value must be less than or equal to the configured queue count for the ENI.

The following example shows how to adjust the number of queues used by an ENI on an Alibaba Cloud Linux 3 instance that supports NIC multi-queue.

1. Connect to a Linux instance.

   For more information, see Connect to a Linux instance by using Workbench.

2. Run the ip address show command to view network configuration information.

   

3. Run the following command to check if the primary ENI eth0 supports NIC multi-queue.

   This example uses the primary ENI. To check a secondary ENI, replace the network interface identifier with eth1, eth2, or another value.

   ethtool -l eth0

4. Check the command output to determine if NIC multi-queue is supported:

   • If the value of "Combined" under "Pre-set maximums" is greater than 1, the ENI supports NIC multi-queue. This value indicates the maximum queue count supported by the ENI.

   • The value of "Combined" under "Current hardware settings" indicates the current queue count in use.

   In this example, a maximum of three combined (RX+TX) queues are supported and three queues are currently in use.

   

5. Run the following command to set the number of active queues for the primary ENI eth0 to 2.

   This example uses the primary ENI. To adjust a secondary ENI, replace the network interface identifier with eth1, eth2, or another value.

   sudo ethtool -L eth0 combined N

   N is the number of queues that you want the ENI to use. N must be less than or equal to the "Combined" value under "Pre-set maximums".

   In this example, set the number of queues for the primary ENI to 2:

   sudo ethtool -L eth0 combined 2

Configure IRQ affinity

When you use NIC multi-queue, you typically need to configure IRQ affinity. This process assigns interrupts from different queues to specific CPU cores. This helps reduce CPU contention and improve network performance.

The following steps show how to use the ecs_mq script to automatically configure IRQ affinity for NIC multi-queue on a Red Hat Enterprise Linux 9.2 image. If your instance does not use a Red Hat Enterprise Linux image, IRQ affinity is enabled by default and requires no configuration.

1. Connect to a Linux instance.

   For more information, see Connect to a Linux instance by using Workbench.

2. (Optional) Stop the irqbalance service.

   The irqbalance service dynamically adjusts IRQ affinity, which can conflict with the ecs_mq script. To avoid this conflict, stop the irqbalance service.

   systemctl stop irqbalance.service

3. Run the following command to download the latest version of the ecs_mq automatic configuration script.

   wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/ecs_mq/ecs_mq_latest.tgz

   Fixes in the latest version of ecs_mq

   The following issues are fixed in the latest version of ecs_mq compared to the previous version of ecs_mq:

   • Prioritizes binding interrupts to CPUs on the same non-uniform memory access (NUMA) node as the ENI's PCIe device.

   • Optimizes tuning logic for multiple network devices.

   • Adjusts interrupt binding logic based on the ratio of NIC queues to CPU cores.

   • Optimizes interrupt binding based on the position of CPU siblings.

   • Fixes increased memory access latency caused by cross-NUMA bindings in the previous version.

   • The new mode of the script is enabled by default. You can use the following commands to switch between the new and old modes of the ecs_mq script:

     • Switch to the old version of ecs_mq: ecs_mq_rps_rfs old

     • Switch to the new version of ecs_mq: ecs_mq_rps_rfs new

   • Fixes an issue on some systems where the ethtool command could not be found during ENI hot-plugging, which caused the ecs_mq_rps_rfs script to fail.

   Note

   The new ecs_mq, compared to the old ecs_mq, delivers a 5% to 30% network performance improvement for most PPS and BPS in network performance tests.

4. Run the following command to decompress the ecs_mq script package.

   tar -xzf ecs_mq_latest.tgz

5. Run the following command to change the working directory.

   cd ecs_mq/

6. Run the ecs_mq installation script.

   bash install.sh redhat 9

   Note

   Replace redhat and 9 with the name and major version number of your operating system.

7. Run the following command to start the ecs_mq service.

   systemctl start ecs_mq

   After the service starts, it automatically enables IRQ affinity.

To find the optimal balance for your workload, test different combinations of queue counts and IRQ affinity settings while monitoring metrics like throughput and latency.