Terway network plugin

Before you begin

Understand Terway's working modes and network planning requirements before creating a cluster.

Read Networking overview and Terway vs. Flannel CNI plugins to choose a network plugin.

Plan CIDR blocks before creating a cluster. See ACK managed cluster network planning.

Billing

The Terway plugin is free, but its pods consume a small amount of resources on each node. For ACK cloud service billing, see Cloud resource costs.

Usage notes

The eni-config file contains critical system parameters. Modifying or deleting unpermitted fields can cause network disruptions or pod creation failures. For customizable parameters, see Customize Terway parameters.

Terway uses CRDs to track resource status. Do not modify these system resources—unauthorized changes can cause network disruptions or pod creation failures.

Maximum pods per node

The maximum pod count per node depends on the number of elastic network interfaces (ENIs) the ECS instance type supports. Each node must meet a minimum pod limit to join the cluster.

Install the Terway network plug-in

Select the Terway network plugin when creating a cluster. The network plugin cannot be changed afterward.

1. Log on to the ACK console. In the left navigation pane, click Clusters.

2. On the Clusters page, click Create Kubernetes Cluster.

3. Configure Terway network parameters. For other parameters, see Create an ACK managed cluster.

   Parameter	Description
   IPv6 Dual-stack	Select Enable to create a dual-stack cluster supporting IPv4 and IPv6. Supported only for Kubernetes 1.22 or later, only with Terway, and cannot be used together with eRDMA. The cluster supports both IPv4 and IPv6 protocols, but communication between worker nodes and the control plane still uses IPv4 addresses. Ensure the following: The cluster VPC supports IPv6 dual-stack. When using Terway in shared ENI mode, the instance type of the node must support IPv6 and have the same number of assignable IPv4 and IPv6 addresses.
   VPC	The VPC for the cluster.
   Network Plug-in	Select Terway.
   DataPath V2	Enable DataPath V2 acceleration, which uses a different traffic forwarding path than standard shared ENI mode for network acceleration. See Best practices for Terway with Datapath V2. Note For new clusters that run Kubernetes 1.34 or later and use DataPath V2, kube-proxy no longer runs on Terway nodes. This mode includes built-in portmap support. See Configure a custom CNI chain. DataPath V2 is supported only on the following operating system images and requires Linux kernel 5.10 or later: Alibaba Cloud Linux 4 Alibaba Cloud Linux 3 (all versions) ContainerOS Ubuntu When enabled, the policy container on each node consumes ~0.5 CPU cores and 512 MB memory, increasing with cluster size. Default CPU limit: 1 core; no memory limit. In DataPath V2 mode, conntrack information is stored in an eBPF map using LRU eviction. Configure map limits based on your workload to prevent exceeding the connection limit. See Optimize conntrack configurations in Terway mode.
   NetworkPolicy support	Select this option to enable native Kubernetes NetworkPolicy. Note From Terway v1.9.2, NetworkPolicy in new clusters uses eBPF with DataPath V2 for the data plane. The feature that allows you to manage NetworkPolicy resources in the console is in public preview. To use this feature, submit a request in Quota Center console.
   Support for ENI Trunking	Select to enable Trunk ENI mode, allowing a static IP, dedicated vSwitch, and separate security group per pod. Note You can enable Trunk ENI for ACK managed clusters without submitting a request. If you want to enable Trunk ENI for an ACK dedicated cluster, submit a request in Quota Center console first. For newly created ACK managed clusters that run Kubernetes 1.31 or later, the Trunk ENI feature is automatically enabled. You do not need to manually select it.
   vSwitch	The vSwitches for cluster nodes. Select vSwitches from at least three availability zones for high availability.
   Pod vSwitch	The CIDR block of the pod vSwitch can overlap with the CIDR block of the node vSwitch. Terway assigns IP addresses to pods from the pod vSwitch's CIDR block, which consumes IP resources from the VPC. If IP resources are depleted quickly, you can go to the VPC console - vSwitches page to check which cloud products, such as ECS, SLB, and NAT Gateway, are consuming the IP addresses.
   Service CIDR	The CIDR block for Kubernetes Services. Must not overlap with node or pod CIDR blocks.
   IPv6 Service CIDR Block	You can configure the IPv6 Service CIDR block after you enable the IPv6 dual-stack feature.

Terway working modes

Terway supports shared ENI and exclusive ENI modes, each with different performance and feature tradeoffs.

Shared ENI mode and exclusive ENI mode

Terway provides two modes for assigning IP addresses to pods: shared ENI mode and exclusive ENI mode.

Important

• From Terway v1.11.0, ENI mode is selected per node pool, no longer at cluster level.

• The node OS uses the primary ENI; Terway manages the rest for pod networking. Do not manually configure these ENIs. To manage specific ENIs, see Configure a filter for ENIs.

Item		Shared ENI mode	Exclusive ENI mode
Pod IP address management	ENI allocation	Multiple pods share an ENI.	Each pod is assigned a dedicated ENI on its node.
	Pod deployment density	High density, supporting hundreds of pods on a single node.	Low density. A typical node supports only a few pods.
	Network architecture
	Data path	Pod-to-pod and Service backend traffic passes through the node's network stack.	Service access uses the node's OS network stack, but pod-to-pod and Service backend traffic bypasses the stack via the dedicated ENI for better performance.
Scenarios		General-purpose Kubernetes workloads.	Delivers VM-comparable network performance, ideal for high-throughput or low-latency applications.
Network acceleration		Supports DataPath V2 network acceleration.	Not supported. Exclusive ENI already provides high network performance via dedicated pod ENIs.
NetworkPolicy support		Supports native Kubernetes NetworkPolicy for policy-based access control. See NetworkPolicy support.	Does not support NetworkPolicy.
Node-level network configuration		Supported. See Node-level network configuration.	Supported. See Node-level network configuration.
Access control		With Trunk ENI enabled, you can configure a static IP, separate vSwitch, and security group per pod.	Allows you to configure a static IP address, a separate vSwitch, and a separate security group for each pod.

Network acceleration

In shared ENI mode, you can enable DataPath V2 network acceleration, which uses a different traffic forwarding path to achieve higher performance.

Important

• DataPath V2 is an upgraded version of the previous IPvlan+eBPF acceleration mode. In Terway V1.8.0 and later, DataPath V2 is the only available acceleration mode when you create a cluster and install the Terway plugin.

• The DataPath V2 and IPvlan+eBPF acceleration modes apply only to node pools in shared ENI mode and do not affect node pools in exclusive ENI mode.

DataPath V2 features	Description
Applicable Terway versions	Clusters created with Terway v1.8.0 or later.
Network architecture
Accelerated data path	When a pod accesses a Service, eBPF resolves the Service IP address to the IP address of a backend pod. When a pod communicates with a pod on a different node, eBPF is used to bypass the network stacks on both nodes. When a pod communicates with a pod on the same node, traffic not only bypasses the node's network stack but is also forwarded directly within the node without leaving it.
Performance optimization	Simplifies the on-host forwarding path, achieving near-host performance and ~30% lower latency than standard mode. eBPF replaces kube-proxy for Service networking, bypassing iptables/IPVS to reduce latency and improve scalability in large clusters. eBPF replaces iptables for NetworkPolicy enforcement, avoiding excessive host iptables rules and minimizing performance impact.
Usage	When creating a cluster, set Network Plug-in to Terway and select DataPath V2.
Usage notes	Requires kernel 5.10+. Alibaba Cloud Linux OS images recommended. The Sandboxed-Container runtime is not supported. Limitations on NetworkPolicy: CIDR selectors cannot control traffic to pod IP ranges. Use pod selectors instead. The except keyword in CIDR selectors has limited support. Avoid using it. An egress NetworkPolicy may block access to pods with hostNetwork enabled or to node IP addresses within the cluster. In-cluster access to a public-facing SLB instance for a LoadBalancer Service may fail due to loopback. See Why am I unable to access a load balancer?. IPv6 hairpin access is not supported. Limitations on NodePort: ExternalTrafficPolicy=Local may block traffic. Change to ExternalTrafficPolicy=Cluster to resolve. With ExternalTrafficPolicy=Cluster, SNAT is applied to the source IP. SNAT port range: 32768 to 65535. The eBPF acceleration feature differs from the default Linux implementation. Adjust component configurations based on your traffic volume. See Best practices for Terway Datapath V2 to configure monitoring and eBPF map limits.

The IPvlan+eBPF acceleration mode may still be in use on older clusters.

IPvlan+eBPF acceleration mode

IPvlan+eBPF features	Description
Applicable Terway versions	Clusters created with Terway v1.7.0 or earlier.
Network architecture
Accelerated data path	When a pod accesses a Service, eBPF is used in the pod's network namespace to resolve the Service IP address to the IP address of a backend pod. When a pod communicates with another pod, IPvlan is used to bypass the network stacks of both nodes.
Usage	When you create a cluster, set Network Plug-in to Terway and select Pod IPvlan.
Usage notes	Requires kernel 4.19+. Alibaba Cloud Linux OS images recommended. The Sandboxed-Container runtime is not supported. Limitations on NetworkPolicy: CIDR selectors cannot control traffic to pod IP ranges. Use pod selectors instead. The except keyword in CIDR selectors has limited support. Avoid using it. An egress NetworkPolicy may block access to pods with hostNetwork enabled or to node IP addresses within the cluster. In-cluster access to a public-facing SLB instance for a LoadBalancer Service may fail due to loopback. See Why am I unable to access a load balancer?. IPv6 hairpin access is not supported. Limitations on NodePort: ExternalTrafficPolicy=Local may block traffic. Change to ExternalTrafficPolicy=Cluster to resolve. With ExternalTrafficPolicy=Cluster, SNAT is applied to the source IP. SNAT port range: 32768 to 65535. The eBPF acceleration feature differs from default Linux behavior. Adjust component configurations based on your traffic volume. See Best practices for Terway Datapath V2 to configure monitoring and eBPF map limits.

Access control

Terway provides fine-grained traffic management via NetworkPolicy and Trunk ENI in shared ENI mode, and per-pod ENI isolation in exclusive ENI mode.

NetworkPolicy

• Node pools in exclusive ENI mode do not support NetworkPolicy.

• Node pools in shared ENI mode support native Kubernetes NetworkPolicy, which allows you to control network traffic between pods by using user-defined rules.

  When creating a cluster, enable NetworkPolicy by setting Network Plug-in to Terway and selecting NetworkPolicy support. See Use network policies in ACK clusters.

  Note

  The feature that allows you to manage NetworkPolicy resources in the console is in public preview. To use this feature, submit a request in Quota Center console.

Static IP, vSwitch, and security group

• Node pools in exclusive ENI mode inherently support assigning a static IP address, a separate vSwitch, and a separate security group to each pod. This enables fine-grained traffic management, network isolation, policy configuration, and IP address management.

• For node pools in shared ENI mode, the optional Trunk ENI feature enables you to configure a static IP address, a separate vSwitch, and a separate security group for each pod.

  To enable this feature, set Network Plug-in to Terway and select Support for ENI Trunking when creating your cluster. See Configure a static IP address, separate vSwitch, and security group for a pod.

  Note

  • You can enable Trunk ENI for ACK managed clusters without submitting a request. If you want to enable Trunk ENI for an ACK dedicated cluster, submit a request in Quota Center console first.

  • For newly created ACK managed clusters that run Kubernetes 1.31 or later, the Trunk ENI feature is automatically enabled. You do not need to manually select it.

  • After you enable Trunk ENI mode, the terway-eniip and terway-controlplane components are installed.

Scaling limits

Terway uses cloud product APIs to manage node network interfaces and IP addresses. For API rate limits, see each cloud product's documentation.

• shared ENI mode: Up to 500 nodes can be allocated concurrently.

• exclusive ENI/TrunkENI mode: Up to 100 pods can be allocated concurrently.

These quotas are fixed.

Data plane configuration

The Terway data plane depends on the precise sequence and integrity of its kernel-level IP rule, IP route, and eBPF hook configurations. Uncoordinated changes by external components—priority adjustments, rule overrides, or program unloading—can cause pod network disruptions, ineffective network policies, and unintended traffic redirection. Validate all third-party components before integration to prevent conflicts.

TC filter rules

IP rules

FAQ

OS Type AliyunLinux4ContainerOptimized is not supported for addons: terway-eniip: required version >= v1.16.4, current version v1.14.3