Customize Terway configuration parameters-Container Service for Kubernetes(ACK)-阿里云帮助中心

The default Terway configuration meets the requirements of most scenarios. Modifying the Terway configuration is a high-risk operation that may cause errors, such as service interruptions. If you have specific requirements, such as increasing the reserved IP address resource pool to optimize pod IP address allocation, you can customize the Terway configuration by following the instructions in this topic.

Limitations

Terway provides a wide range of configuration parameters. This topic describes each parameter but does not guarantee that all parameter combinations work in all scenarios. We recommend that you thoroughly test your configuration before applying it.
Some parameters can be configured on the console. If you cannot find a parameter on the console, your current component version does not support it. In this case, you must upgrade the Terway component to configure the parameter.

Terway configuration example

The ConfigMap for the Terway component in a cluster uses the following format.

apiVersion: v1
data:
  10-terway.conf: |     # Terway Container Network Interface (CNI) configuration file parameters. The configuration is converted and written to /etc/cni/net.d/. Do not modify.
    {
      "cniVersion": "0.4.0",
      "name": "terway",
      "capabilities": {"bandwidth": true},
      "eniip_virtual_type": "datapathv2",
      "host_stack_cidrs": ["169.254.20.10/32"],
      "cilium_args": "",
      "type": "terway"
    }
  disable_network_policy: "false"    # Specifies whether to disable the NetworkPolicy feature.
  eni_conf: |                        # Main Terway configuration parameters, such as vSwitches and security groups.
    {
      "version": "1",
      "max_pool_size": 5,
      "min_pool_size": 0,
      "credential_path": "/var/addon/token-config",
      "enable_eni_trunking": true,
      "vswitches": {"cn-hangzhou-j":["vsw-foo"],"cn-hangzhou-k":["vsw-foo"]},
      "eni_tags": {"ack.aliyun.com":"c7c3cfoo"},
      "service_cidr": "192.168.0.0/16",
      "security_group": "sg-foo",
      "ip_stack": "ipv4",
      "vswitch_selection_policy": "ordered"
    }
  in_cluster_loadbalance: "true"   # In-cluster load balancing. This parameter takes effect only in DataPathV2 or IPvlan mode.
kind: ConfigMap
metadata:
  name: eni-config
  namespace: kube-system

Parameters

Top-level parameters

Parameter	Description	Modifiable	Configurable on console
`10-terway.conf`	Terway CNI configuration file parameters. The configuration is converted and written to `/etc/cni/net.d/`.	No	No
`10-terway.conflist`	Custom Terway CNI configuration parameters. For more information, see Configure a custom CNI chain.	Yes	No
`disable_network_policy`	Specifies whether to disable the NetworkPolicy feature. `false`: enables the NetworkPolicy feature. `true`: disables the NetworkPolicy feature.	Yes	Yes
`eni_conf`	Main Terway configuration parameters, such as vSwitches and security groups.	Yes	Partially
`in_cluster_loadbalance`	Specifies whether to enable in-cluster load balancing. This parameter takes effect only in DataPathV2 or IPvlan mode. For more information, see How do I enable in-cluster load balancing for a Terway IPvlan cluster?.	Yes	Yes

10-terway.conf parameters

Parameter	Type	Description	Modifiable	Configurable on console
`cniVersion`	string	The CNI configuration version.	No	No
`name`	string	The name of the CNI plug-in.	No	No
`type`	string	The type of the CNI plug-in.	No	No
`capabilities`	string	The CNI capabilities.	Yes	No
`eniip_virtual_type`	string	The ENI virtualization method.	No	No
`cilium_args`	string	The configuration parameters of Cilium. Example: `--bpf-map-dynamic-size-ratio=0.003`. Important Parameter support varies based on community versions, and backward compatibility is not guaranteed. If you want to use these parameters in a production environment, thoroughly test them. If the Terway pod fails to start after an upgrade, remove the configuration and try again. During a component upgrade, the system validates the parameter list. The following parameters are allowed. If you receive a `cilium_args validation` failure message when you upgrade Terway, remove the configuration and try again. `bpf-map-dynamic-size-ratio bpf-ct-timeout-regular-any bpf-ct-timeout-regular-tcp bpf-ct-timeout-regular-tcp-fin bpf-ct-timeout-regular-tcp-syn bpf-ct-timeout-service-any bpf-ct-timeout-service-tcp bpf-ct-timeout-service-tcp-grace bpf-lb-map-max bpf-node-map-max bpf-policy-map-max conntrack-gc-interval bpf-ct-global-any-max bpf-ct-global-tcp-max bpf-nat-global-max prometheus-serve-addr api-rate-limit k8s-client-qps k8s-client-burst`	Yes	Yes
`symmetric_routing`	bool	Requires terway-eniip 1.15.0 or later. Specifies whether to enable symmetric routing. If this feature is enabled, Terway configures policy-based routing rules to ensure that egress traffic uses the same network interface as ingress traffic. `true`: enabled. `false`: disabled.	Yes	No
`symmetric_routing_config`	map[string]any	Requires terway-eniip 1.15.0 or later. Symmetric routing configuration parameters. The following code shows the default values and a parameter example. In most cases, you do not need to modify these parameters. `{ "symmetric_routing_config": { "interface": "eth0", "mark": 16, "mask": 16, "table_id": 100, "rule_priority": 600, "comment": "terway-symmetric" } }` Important Incorrect configurations can affect existing network settings. Thoroughly test the configuration in a test environment before use.	Yes	No

eni_conf parameters

Parameter	Type	Description	Modifiable	Configurable on console
`credential_path`	string	The Security Token Service (STS) token for accessing OpenAPI. This parameter applies only to ACK managed clusters.	No	No
`service_cidr`	string	The Service CIDR block of the cluster.	No	No
`vswitches`	map[string][]string	The list of vSwitches used by elastic network interfaces (ENIs).	Yes	Yes
`eni_tags`	map[string]string	The tags to add for newly created ENIs. Modifications to this configuration do not affect existing ENIs.	No	No
`eni_tag_filter`	map[string]string	Supported only in shared ENI mode. Filters the ENIs managed by Terway. For more information, see Configure a whitelist for ENIs.	Yes	No
`max_pool_size`	int	Supported only in shared ENI mode. The maximum number of idle IP addresses maintained in the IP address resource pool. Default value: 5.	Yes	Yes
`min_pool_size`	int	Supported only in shared ENI mode. The minimum number of idle IP addresses maintained in the IP address resource pool. Default value: 0.	Yes	Yes
`security_group`	string	The security group to use for newly created ENIs. Modifications to this configuration do not affect existing ENIs.	Yes	No
`security_groups`	[]string	The security groups to use for newly created ENIs. You can specify up to 10 security groups, and all security groups must be of the same type. If you configure this parameter, Terway uses the union of this list and the `security_group` parameter. Modifications to this configuration do not affect existing ENIs.	Yes	Yes
`vswitch_selection_policy`	string	The policy for selecting a vSwitch when an ENI is created. `ordered`: Selects the vSwitch with the most available IP addresses. This is the default value. `random`: Randomly selects a vSwitch.	Yes	Yes
`enable_eip_migrate`	bool	Enables the elastic IP address (EIP) migration feature. For more information, see Migrate EIPs from Terway to ack-extend-network-controller.	Yes	No
`ip_stack`	string	The IP family of the cluster. This parameter cannot be modified.	No	No
`ipam_type`	string	The IPAM mode of the cluster. This parameter cannot be modified.	No	No
`enable_eni_trunking`	bool	Enables the `Trunk` feature. For more information, see Configure fixed IP addresses, and independent vSwitches and security groups for pods.	Yes	No
`kube_client_qps`	float32	The queries per second (QPS) for the Kubernetes client. You must configure this parameter together with `kube_client_burst`.	Yes	Yes
`kube_client_burst`	int	The burst value for the Kubernetes client. You must configure this parameter together with `kube_client_qps`, and its value must be greater than that of `kube_client_qps`.	Yes	Yes
`resource_group_id`	string	The resource group to use for newly created ENIs. Modifications to this configuration do not affect existing ENIs.	Yes	Yes
`ip_pool_sync_period`	string	Requires terway-eniip 1.15.0 or later. Supported only in shared ENI mode. The synchronization interval of the IP address resource pool. Default value: 120 seconds. Examples: `2m`, `120s`.	Yes	No
`idle_ip_reclaim_after`	string	Requires terway-eniip 1.16.0 or later. Supported only in shared ENI mode. The time an IP address must be idle before it can be reclaimed. Example: `60m`. Note If a pod is created or deleted before the reclamation, the timer is reset.	Yes	No
`idle_ip_reclaim_interval`	string	Requires terway-eniip 1.16.0 or later. Supported only in shared ENI mode. The interval between reclamation checks. Example: `30m`.	Yes	No
`idle_ip_reclaim_batch_size`	int	Requires terway-eniip 1.16.0 or later. Supported only in shared ENI mode. The maximum number of IP addresses that can be reclaimed per batch. The value must be in the range of [1, 10]. Example: `5`.	Yes	No
`idle_ip_reclaim_jitter_factor`	string	Requires terway-eniip 1.16.0 or later. Supported only in shared ENI mode. The jitter factor that is used to randomize the reclamation time. The value must be in the range of [0.0, 1.0]. Example: `0.1`.	Yes	No
`ip_warm_up_size`	int	Requires terway-eniip 1.16.4 or later and that the `ipam_type` of the cluster is set to `crd`. Supported only in shared ENI mode. The number of IP addresses to pre-warm at a time. This parameter takes effect only for newly added nodes. The pre-warming task stops after the number of allocated IP addresses on the node reaches the expected value. Example: `5`.	Yes	No
`enable_patch_pod_ips`	bool	Requires terway-eniip 1.13.7 or later. Supported only in shared ENI mode. Default value: `true`. When enabled, the `k8s.aliyun.com/pod-ips` annotation is added to pods.	Yes	No
`enable_ip_prefix`	bool	Requires terway-eniip 1.17.0 or later. Supported only in shared ENI mode. Default value: `false`. When enabled, an IP prefix is assigned to the ENI.	Yes	No
`ipv4_prefix_count`	int	Requires terway-eniip 1.17.0 or later. Supported only in shared ENI mode. This parameter is valid in IPv4 single-stack and dual-stack modes. In dual-stack mode, this parameter is ignored because IPv6 prefixes are automatically managed by the system, with one prefix per ENI. The number of IP prefixes configured for the node. This parameter is valid when the cluster network type is `ipv4` or `dual`. The value must be greater than or equal to 0. The upper limit is automatically restricted for burstable instance types.	Yes	No
`ipv6_prefix_count`	int	Requires terway-eniip 1.17.0 or later. Supported only in shared ENI mode. The total number of IPv6 prefixes on the node. This parameter is valid only in IPv6 single-stack mode. The value must be 0 or 1. In dual-stack mode, this parameter is ignored because IPv6 prefixes are automatically managed by the system, with one prefix per ENI. The number of IP prefixes configured for the node. This parameter is valid when the cluster network type is `ipv6`. The value must be in the range of [0, 1].	Yes	No

Configuration methods

Console

You can configure some common parameters on the console. The console validates your input and prevents invalid formats.

Log on to the ACK console. In the left navigation pane, click Clusters.
On the Clusters page, click the name of your cluster. In the left navigation pane, click Add-ons.
Click the Networking tab, and then click Configuration on the terway-eniip card.
In the panel that appears, configure the parameters and then click OK.

kubectl

Some parameters are not displayed on the console. You can use kubectl to configure these parameters.

Warning

Configuring Terway by using kubectl is a high-risk operation. kubectl does not validate input, and incorrect parameters can cause errors, such as preventing Terway from running. Ensure you fully understand the configuration before modifying it.

Connect to a cluster by using kubectl and run the following command to configure advanced parameters:
```
kubectl edit cm -n kube-system  eni-config 
```
After editing the configuration, save your changes and exit the editor. Then, run the following command to apply the changes:
```
 kubectl rollout restart -n kube-system daemonset.apps/terway-eniip
```