Configure a custom CNI chain

Updated at:

Container Network Interface (CNI) is the standard interface for Kubernetes network plugins. A CNI chain runs multiple plugins in sequence, each handling a specific networking task such as IP address allocation or routing, to extend cluster networking without replacing Terway.

Prerequisites

  • An ACK managed cluster that uses the Terway network plugin is created. For more information, see Create an ACK managed cluster.

  • The Terway version is v1.5.6 or later, and the cluster does not use exclusive elastic network interface (ENI) mode.

Warning

Container Service for Kubernetes (ACK) does not guarantee that multiple CNI plugins can work together. Configuring a custom CNI chain is a high-risk operation. Configure the chain only after you fully understand the capabilities of each CNI plugin, and proceed with caution to avoid business interruption caused by configuration errors.

Configure a custom CNI chain

To use a CNI chain, add the required CNI plugins to the Terway configuration file.

Use cni_chain

Modify the Terway configuration file eni-config.

kubectl edit cm -nkube-system eni-config

Parameter

Description

10-terway.conf

The Terway CNI configuration. Do not modify its contents.

Important

Do not modify the original configuration.

cni_chain

The additional CNI rules. These rules are appended after Terway's existing rules. This parameter supports per-node configuration.

Important
  • cni_chain and 10-terway.conflist are mutually exclusive.

  • cni_chain uses YAML format.

  • terway-eniip 1.18.0 or later is required.

Configuration file example

apiVersion: v1
kind: ConfigMap
metadata:
  name: eni-config
  namespace: kube-system
data:
  10-terway.conf: |
    # Keep unchanged if no configuration is required
  cni_chain: |
    - type: tuning
      sysctl:
        net.ipv6.conf.all.disable_ipv6: "1"

Run kubectl rollout restart -n kube-system daemonset.apps/terway-eniip to recreate the Terway pods.

If the configuration is correct, the custom plugin settings appear in the /etc/cni/net.d/10-terway.conflist configuration file on the node.

Use 10-terway.conflist (not recommended)

  1. Modify the Terway configuration file eni-config.

    kubectl edit cm -nkube-system eni-config

    Parameter

    Description

    10-terway.conf

    The Terway CNI configuration. Do not modify its contents.

    Important

    Do not modify the original configuration.

    10-terway.conflist

    The custom CNI chain configuration.

    plugins should list the configuration in 10-terway.conf as the first CNI configuration.

    Important

    The configurations in this topic are examples only. Do not copy them directly, because doing so may cause configuration errors.

    The configuration block should be in JSON format. Ensure that the format is correct.

    Configuration file example

    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: eni-config
      namespace: kube-system
    data:
      10-terway.conflist: |
          {
            "plugins": [
              {
                "cniVersion": "0.4.0",
                "name": "terway",
                "type": "terway",
                "capabilities": {"bandwidth": true}
              },
              {
                "type": "portmap",
                "capabilities": {"portMappings": true},
                "externalSetMarkChain":"KUBE-MARK-MASQ"
              }
            ]
          }
      10-terway.conf: |
        {
          "cniVersion": "0.4.0",
          "name": "terway",
          "type": "terway",
          "capabilities": {"bandwidth": true}
        }
  2. Run kubectl rollout restart -n kube-system daemonset.apps/terway-eniip to recreate the Terway pods.

    If the configuration is correct, the custom plugin settings appear in the /etc/cni/net.d/10-terway.conflist configuration file on the node.

Use cases

Configure portmap

The portmap plugin primarily maps a pod's internal ports to ports on the host, allowing external networks to access specific services in the pod.

Note

If you need public network access, ensure that the corresponding ports are open for inbound traffic in the security group.

The policy rules generated by portmap may be incompatible with Network Policies. Test them thoroughly before use.

Configuration example

  • Access with the private IP address and port of a node:

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: eni-config
      namespace: kube-system
    data:
      10-terway.conf: |
        {
          "cniVersion": "0.4.0",
          "name": "terway",
          "type": "terway",
          "capabilities": {"bandwidth": true}
        }
      cni_chain: |
        - type: portmap
          capabilities:
            portMappings: true
          externalSetMarkChain: KUBE-MARK-MASQ
  • Access with the public IP address and port:

    Public network access requires the following configuration:

    Parameter

    Non-Datapath V2

    Datapath V2

    masqAll

    Not required

    Required

    symmetric_routing

    Required

    Required

    • masqAll: a portmap plugin parameter. Requires portmap v1.7.1 or later.

    • symmetric_routing: a Terway plugin parameter. Requires Terway v1.15.0 or later.

    • Datapath V2 clusters with kube-proxy replacement support port mapping without the portmap plugin.

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: eni-config
      namespace: kube-system
    data:
      10-terway.conf: |
        {
          "cniVersion": "0.4.0",
          "name": "terway",
          "type": "terway",
          "symmetric_routing": true,
          "capabilities": {
            "bandwidth": true
          }
        }
      cni_chain: |
        - type: portmap
          capabilities:
            portMappings: true
          masqAll: false

Disable IPv6 link-local addresses for containers

Inside a container, even if your cluster does not have IPv6 dual-stack enabled, the network interface is still automatically assigned an IPv6 link-local address after it is created. The address belongs to the fe80::/64 range. This is the default behavior of the operating system kernel. It is usually harmless and does not interfere with normal operations.

However, if your application handles network addresses incorrectly, it may mistake this IPv6 link-local address for the pod IP address and try to use it for communication. But link-local addresses are intended for communication between devices on the same link and cannot be used for network communication, which may cause your application to behave abnormally. If your application mistakes this IPv6 link-local address for the pod IP address, the application code has a defect. Report the issue to the application vendor first and seek a solution.

If you still need to disable IPv6 link-local addresses in containers, configure the tuning plugin.

Configuration example

kind: ConfigMap
apiVersion: v1
metadata:
  name: eni-config
  namespace: kube-system
data:
  10-terway.conf: |
    {
      "cniVersion": "0.4.0",
      "name": "terway",
      "type": "terway",
      "capabilities": {"bandwidth": true}
    }
  cni_chain: |
    - type: tuning
      sysctl:
        net.ipv6.conf.all.disable_ipv6: "1"