How to configure a custom CNI chain-Container Service for Kubernetes(ACK)-阿里云帮助中心

CNI (Container Network Interface) is the Kubernetes network plug-in standard. CNI Chain links multiple CNI plug-ins in sequence: each plug-in handles specific tasks such as IP allocation or routing, passing output to the next until all tasks complete. This topic explains how to configure a custom CNI chain in an ACK Terway cluster.

Prerequisites

An ACK managed cluster with the Terway network plug-in is created.
Terway v1.5.6 or later. The cluster does not use exclusive ENI network mode.

Warning

ACK does not guarantee compatibility between CNI plug-ins. A custom CNI chain is a high-risk operation. Understand each plug-in before proceeding to avoid business interruption.

Configure a custom CNI chain

Add the CNI plugins to the Terway configuration file.

Edit the eni-config ConfigMap:

kubectl edit cm -nkube-system eni-config

Parameter

Description

10-terway.conf

The Terway CNI configuration. Do not modify.

Important

Do not modify this file.

10-terway.conflist

The custom CNI chain configuration.

The first entry in plugins must match 10-terway.conf.

Important

This is an example configuration. Do not use it directly.

The configuration block must be in JSON format.

Configuration file

kind: ConfigMap
apiVersion: v1
metadata:
  name: eni-config
  namespace: kube-system
data:
  10-terway.conflist: |
      {
        "plugins": [
          {
            "cniVersion": "0.4.0",
            "name": "terway",
            "type": "terway",
            "capabilities": {"bandwidth": true}
          },
          {
            "type": "portmap",
            "capabilities": {"portMappings": true},
            "externalSetMarkChain":"KUBE-MARK-MASQ"
          }
        ]
      }
  10-terway.conf: |
    {
      "cniVersion": "0.4.0",
      "name": "terway",
      "type": "terway",
      "capabilities": {"bandwidth": true}
    }

Run kubectl rollout restart -n kube-system daemonset.apps/terway-eniip to rebuild the Terway pods.

If configured correctly, verify the custom plugin configuration in etc/cni/net.d/10-terway.conflist on the node.

Use cases

Configure a portmap

The portmap plugin maps pod ports to host ports for external access.

Note

For public network access, configure security group inbound rules to allow traffic on the required ports.

These rules may conflict with Kubernetes network policies. Test thoroughly before deploying to production.

Configuration example

Access with the private IP address and port of a node:

kind: ConfigMap
apiVersion: v1
metadata:
  name: eni-config
  namespace: kube-system
data:
  10-terway.conflist: |
      {
        "plugins": [
          {
            "cniVersion": "0.4.0",
            "name": "terway",
            "type": "terway",
            "capabilities": {"bandwidth": true} 
          },
          {
            "type": "portmap",
            "capabilities": {"portMappings": true},
            "externalSetMarkChain":"KUBE-MARK-MASQ"
          }
        ]
      }

  10-terway.conf: |
    {
      "cniVersion": "0.4.0",
      "name": "terway",
      "type": "terway",
      "capabilities": {"bandwidth": true}
    }

Access with the public IP address and port:

Public network access requires the following configuration.

Parameter	Non-Datapath V2	Datapath V2
masqAll	Not required	Required
symmetric_routing	Required	Required

masqAll: Portmap plug-in parameter. Requires portmap v1.7.1 or later.
symmetric_routing: Terway plug-in parameter. Requires Terway v1.15.0 or later.
Datapath V2 clusters with kube-proxy replacement do not require the portmap plug-in.

kind: ConfigMap
apiVersion: v1
metadata:
  name: eni-config
  namespace: kube-system
data:
  10-terway.conflist: |
      {
        "name": "terway-chain",
        "plugins": [
          {
            "cniVersion": "0.4.0",
            "name": "terway",
            "type": "terway",
            "symmetric_routing": true,
            "capabilities": {"bandwidth": true} 
          },
          {
            "type": "portmap",
            "capabilities": {"portMappings": true},
            "masqAll": false
          }
        ]
      }

  10-terway.conf: |
    {
      "cniVersion": "0.4.0",
      "name": "terway",
      "type": "terway",
      "capabilities": {"bandwidth": true}
    }

Disable IPv6 link-local addresses for containers

Even without the IPv6 dual-stack feature, containers automatically receive an IPv6 link-local address in the fe80::/64 subnet. This default kernel behavior is typically harmless.

If an application incorrectly handles network addresses, it may use IPv6 link-local addresses for cross-network communication. Link-local addresses only work within the same link, causing failures. Report such issues to the application vendor first.

To disable IPv6 link-local addresses in containers, configure the tuning plugin.

Configuration example

kind: ConfigMap
apiVersion: v1
metadata:
  name: eni-config
  namespace: kube-system
data:
  10-terway.conflist: |
      {
        "plugins": [
          {
            "cniVersion": "0.4.0",
            "name": "terway",
            "type": "terway",
            "capabilities": {"bandwidth": true}
          },
          {
            "type": "tuning",
            "sysctl": {
              "net.ipv6.conf.all.disable_ipv6": "1",
              "net.ipv6.conf.default.disable_ipv6": "1",
              "net.ipv6.conf.lo.disable_ipv6": "1"
            }
          }
        ]
      }
  10-terway.conf: |
    {
      "cniVersion": "0.4.0",
      "name": "terway",
      "type": "terway",
      "capabilities": {"bandwidth": true}
    }