启用CNI插件提升安全性

阿里云服务网格ASM支持使用CNI插件,将Iptables规则配置移出Pod。CNI插件不要求Istio用户启用Priviledge权限,从而降低了对操作者的权限要求,提升了服务网格ASM的安全性。本文介绍如何启用CNI插件。

前提条件

背景信息

为了确保服务网格ASM正常工作,Istio需要在网格中的每个Pod中配置一个Envoy代理,通过Iptables规则控制Pod的流量,以便通过注入的Envoy代理将流量重定向到应用程序。因为每个Pod的Iptables规则都是网络命名,所以更改不会影响节点上的其他Pod。

默认情况下,Istio在Pod中注入istio-init容器,并在Pod中的其他容器启动之前设置必要的Iptables规则。这就要求操作者部署容器时具有足够的特权,包括部署具有NET_ADMIN功能的其他容器、重新配置网络等。

iptables规则

服务网格ASM支持使用CNI插件,将Iptables规则配置移出Pod。CNI插件不要求Istio用户启用Priviledge权限,在Pod的网络设置阶段执行流量重定向,从而消除了对NET_ADMIN能力的依赖要求。CNI插件启用后,其配置将被插入到容器CNI插件链中,从而在容器启动时被调用执行。

CNI插件根据以下条件进行查找需要重定向流量的Pod:

  • Pod的命名空间不在配置的excludeNamespaces列表中。

  • Pod中包含一个名为istio-proxy的容器。

  • Pod中有多个容器。

  • Pod中没有keysidecar.istio.io/inject的注解,或者注释sidecar.istio.io/inject的值不为true

启用CNI插件

您可以通过ASM控制台为ASM实例启用CNI插件,具体操作步骤如下:

  1. 登录ASM控制台

  2. 网格管理页面,单击目标实例名称,然后在左侧导航栏,选择数据面组件管理 > 网格CNI插件

  3. 网格CNI插件页面,打开是否启用网格CNI插件开关,选中需要排除的命名空间,然后单击更新设置

    被排除的命名空间的Pod启动时将通过istio-init进行网络配置,不通过CNI进行网络配置。当目标实例的状态更新中变为运行中,表示CNI插件启用成功。

验证Iptables规则

本文以在集群中部署bookinfo应用为例,验证拦截规则是否生效。

  1. 使用以下内容,创建bookinfo.yaml

    展开查看bookinfo.yaml

    ##################################################################################################
    # Details service
    ##################################################################################################
    apiVersion: v1
    kind: Service
    metadata:
      name: details
      labels:
        app: details
        service: details
    spec:
      ports:
      - port: 9080
        name: http
      selector:
        app: details
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: bookinfo-details
      labels:
        account: details
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: details-v1
      labels:
        app: details
        version: v1
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: details
          version: v1
      template:
        metadata:
          labels:
            app: details
            version: v1
        spec:
          serviceAccountName: bookinfo-details
          containers:
          - name: details
            image: docker.io/istio/examples-bookinfo-details-v1:1.16.2
            imagePullPolicy: IfNotPresent
            ports:
            - containerPort: 9080
            securityContext:
              runAsUser: 1000
    ---
    ##################################################################################################
    # Ratings service
    ##################################################################################################
    apiVersion: v1
    kind: Service
    metadata:
      name: ratings
      labels:
        app: ratings
        service: ratings
    spec:
      ports:
      - port: 9080
        name: http
      selector:
        app: ratings
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: bookinfo-ratings
      labels:
        account: ratings
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: ratings-v1
      labels:
        app: ratings
        version: v1
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: ratings
          version: v1
      template:
        metadata:
          labels:
            app: ratings
            version: v1
        spec:
          serviceAccountName: bookinfo-ratings
          containers:
          - name: ratings
            image: docker.io/istio/examples-bookinfo-ratings-v1:1.16.2
            imagePullPolicy: IfNotPresent
            ports:
            - containerPort: 9080
            securityContext:
              runAsUser: 1000
    ---
    ##################################################################################################
    # Reviews service
    ##################################################################################################
    apiVersion: v1
    kind: Service
    metadata:
      name: reviews
      labels:
        app: reviews
        service: reviews
    spec:
      ports:
      - port: 9080
        name: http
      selector:
        app: reviews
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: bookinfo-reviews
      labels:
        account: reviews
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: reviews-v1
      labels:
        app: reviews
        version: v1
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: reviews
          version: v1
      template:
        metadata:
          labels:
            app: reviews
            version: v1
        spec:
          serviceAccountName: bookinfo-reviews
          containers:
          - name: reviews
            image: docker.io/istio/examples-bookinfo-reviews-v1:1.16.2
            imagePullPolicy: IfNotPresent
            env:
            - name: LOG_DIR
              value: "/tmp/logs"
            ports:
            - containerPort: 9080
            volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: wlp-output
              mountPath: /opt/ibm/wlp/output
            securityContext:
              runAsUser: 1000
          volumes:
          - name: wlp-output
            emptyDir: {}
          - name: tmp
            emptyDir: {}
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: reviews-v2
      labels:
        app: reviews
        version: v2
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: reviews
          version: v2
      template:
        metadata:
          labels:
            app: reviews
            version: v2
        spec:
          serviceAccountName: bookinfo-reviews
          containers:
          - name: reviews
            image: docker.io/istio/examples-bookinfo-reviews-v2:1.16.2
            imagePullPolicy: IfNotPresent
            env:
            - name: LOG_DIR
              value: "/tmp/logs"
            ports:
            - containerPort: 9080
            volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: wlp-output
              mountPath: /opt/ibm/wlp/output
            securityContext:
              runAsUser: 1000
          volumes:
          - name: wlp-output
            emptyDir: {}
          - name: tmp
            emptyDir: {}
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: reviews-v3
      labels:
        app: reviews
        version: v3
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: reviews
          version: v3
      template:
        metadata:
          labels:
            app: reviews
            version: v3
        spec:
          serviceAccountName: bookinfo-reviews
          containers:
          - name: reviews
            image: docker.io/istio/examples-bookinfo-reviews-v3:1.16.2
            imagePullPolicy: IfNotPresent
            env:
            - name: LOG_DIR
              value: "/tmp/logs"
            ports:
            - containerPort: 9080
            volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: wlp-output
              mountPath: /opt/ibm/wlp/output
            securityContext:
              runAsUser: 1000
          volumes:
          - name: wlp-output
            emptyDir: {}
          - name: tmp
            emptyDir: {}
    ---
    ##################################################################################################
    # Productpage services
    ##################################################################################################
    apiVersion: v1
    kind: Service
    metadata:
      name: productpage
      labels:
        app: productpage
        service: productpage
    spec:
      ports:
      - port: 9080
        name: http
      selector:
        app: productpage
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: bookinfo-productpage
      labels:
        account: productpage
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: productpage-v1
      labels:
        app: productpage
        version: v1
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: productpage
          version: v1
      template:
        metadata:
          labels:
            app: productpage
            version: v1
        spec:
          serviceAccountName: bookinfo-productpage
          containers:
          - name: productpage
            image: docker.io/istio/examples-bookinfo-productpage-v1:1.16.2
            imagePullPolicy: IfNotPresent
            ports:
            - containerPort: 9080
            volumeMounts:
            - name: tmp
              mountPath: /tmp
            securityContext:
              runAsUser: 1000
          volumes:
          - name: tmp
            emptyDir: {}
    ---
                            
  2. 执行以下命令,部署bookinfo。

    kubectl apply -f bookinfo.yaml
  3. 执行以下命令,获取Productpage Pod的容器ID、节点名称。

    ns=default
    podname=kubectl get pod |grep productpage
    #容器运行时为Docker时,使用下面命令。
    container_id=$(kubectl get pod -n ${ns} ${podname} -o jsonpath="{.status.containerStatuses[?(@.name=='istio-proxy')].containerID}" | sed -n 's/docker:\/\/\(.*\)/\1/p')
    #容器运行时为Containerd时,使用下面命令。
    container_id=$(kubectl get pod -n ${ns} ${podname} -o jsonpath="{.status.containerStatuses[?(@.name=='istio-proxy')].containerID}" | sed -n 's/containerd:\/\/\(.*\)/\1/p')
    echo $container_id
    
    #获取节点名称。
    kubectl get pod ${podname} -o jsonpath="{.spec.nodeName}"
  4. 登录Productpage Pod的节点(例如使用SSH),执行以下命令,获取映射到Conatiner ID的对应进程。

    #容器运行时为Docker时,使用下面命令。
    docker inspect --format '{{ .State.Pid }}' $container_id
    #容器运行时为Containerd时,使用下面命令。
    crictl inspect $container_id|jq ".info.pid"
  5. 执行以下命令,进入Productpage容器的网络命名空间,获取当前配置。

    nsenter -t $ -n iptables -L -t nat -n -v --line-numbers -x

    展开查看预期输出

    Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
    num      pkts      bytes target     prot opt in     out     source               destination
    1       34938  2096280 ISTIO_INBOUND  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
    
    Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
    num      pkts      bytes target     prot opt in     out     source               destination
    
    Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
    num      pkts      bytes target     prot opt in     out     source               destination
    
    Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
    num      pkts      bytes target     prot opt in     out     source               destination
    1          17     1020 ISTIO_OUTPUT  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
    
    Chain ISTIO_INBOUND (1 references)
    num      pkts      bytes target     prot opt in     out     source               destination
    1           0        0 RETURN     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:15008
    2           0        0 RETURN     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22
    3           0        0 RETURN     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:15020
    4       34938  2096280 RETURN     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:15021
    5           0        0 RETURN     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:15090
    6           0        0 ISTIO_IN_REDIRECT  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
    
    Chain ISTIO_REDIRECT (1 references)
    num      pkts      bytes target     prot opt in     out     source               destination
    1           0        0 REDIRECT   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            redir ports 15001
    
    Chain ISTIO_IN_REDIRECT (3 references)
    num      pkts      bytes target     prot opt in     out     source               destination
    1           1       60 REDIRECT   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            redir ports 15006
    
    Chain ISTIO_OUTPUT (1 references)
    num      pkts      bytes target     prot opt in     out     source               destination
    1           2      120 RETURN     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:15020
    2           0        0 RETURN     all  --  *      lo      127.0.X.X            0.0.0.0/0
    3           1       60 ISTIO_IN_REDIRECT  all  --  *      lo      0.0.0.0/0           !127.0.X.X            owner UID match 1337
    4           0        0 RETURN     all  --  *      lo      0.0.0.0/0            0.0.0.0/0            ! owner UID match 1337
    5          14      840 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            owner UID match 1337
    6           0        0 ISTIO_IN_REDIRECT  all  --  *      lo      0.0.0.0/0           !127.0.X.X            owner GID match 1337
    7           0        0 RETURN     all  --  *      lo      0.0.0.0/0            0.0.0.0/0            ! owner GID match 1337
    8           0        0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            owner GID match 1337
    9           0        0 RETURN     all  --  *      *       0.0.0.0/0            127.0.X.X
    10          0        0 RETURN     all  --  *      *       0.0.0.0/0            192.168.0.1
    11          0        0 ISTIO_REDIRECT  all  --  *      *       0.0.0.0/0            0.0.0.0/0

    由预期输出得到,存在ISTIO_INBOUNDISTIO_REDIRECTISTIO_IN_REDIRECTISTIO_OUTPUT等规则,表明拦截规则已经生效。