【已弃用】使用Pod安全策略

重要

本文中含有需要您注意的重要提示信息,忽略该信息可能对您的业务造成影响,请务必仔细阅读。

KubernetesPod安全策略(Pod Security Policy)准入控制组件会基于您定义的规则验证在集群上创建和更新Pod的请求。如果创建或更新Pod的请求不符合定义的规则,系统将拒绝该请求并返回错误。本文将介绍如何在容器服务KubernetesACK(Container Service for Kubernetes)中使用Pod安全策略。

前提条件

您已完成以下操作:

说明

本文档仅适用于1.26以下版本的集群。

ACK默认的Pod安全策略

ACK中,Kubernetes 1.16.6版本的标准专有集群和标准托管集群将默认启用Pod安全策略准入控制组件,并配置一个名为ack.privilegedPod安全策略。这个安全策略将放行任意类型的Pod,效果等同于集群未开启Pod安全策略准入控制组件。

默认的Pod安全策略命令

$ kubectl get psp ack.privileged
NAME             PRIV   CAPS   SELINUX    RUNASUSER   FSGROUP    SUPGROUP   READONLYROOTFS   VOLUMES
ack.privileged   true   *      RunAsAny   RunAsAny    RunAsAny   RunAsAny   false            *

详细规则的Pod安全策略命令

$ kubectl describe psp ack.privileged
Name:  ack.privileged

Settings:
  Allow Privileged:                       true
  Allow Privilege Escalation:             true
  Default Add Capabilities:               <none>
  Required Drop Capabilities:             <none>
  Allowed Capabilities:                   *
  Allowed Volume Types:                   *
  Allow Host Network:                     true
  Allow Host Ports:                       0-65535
  Allow Host PID:                         true
  Allow Host IPC:                         true
  Read Only Root Filesystem:              false
  SELinux Context Strategy: RunAsAny
    User:                                 <none>
    Role:                                 <none>
    Type:                                 <none>
    Level:                                <none>
  Run As User Strategy: RunAsAny
    Ranges:                               <none>
  FSGroup Strategy: RunAsAny
    Ranges:                               <none>
  Supplemental Groups Strategy: RunAsAny
    Ranges:                               <none>

展开查看Pod安全策略、相应集群角色、集群角色绑定的完整YAML文件内容

---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: ack.privileged
  annotations:
    kubernetes.io/description: 'privileged allows full unrestricted access to
      pod features, as if the PodSecurityPolicy controller was not enabled.'
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*'
  labels:
    kubernetes.io/cluster-service: "true"
    ack.alicloud.com/component: pod-security-policy
spec:
  privileged: true
  allowPrivilegeEscalation: true
  allowedCapabilities:
  - '*'
  volumes:
  - '*'
  hostNetwork: true
  hostPorts:
  - min: 0
    max: 65535
  hostIPC: true
  hostPID: true
  runAsUser:
    rule: 'RunAsAny'
  seLinux:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'
  readOnlyRootFilesystem: false

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: ack:podsecuritypolicy:privileged
  labels:
    kubernetes.io/cluster-service: "true"
    ack.alicloud.com/component: pod-security-policy
rules:
- apiGroups:
  - policy
  resourceNames:
  - ack.privileged
  resources:
  - podsecuritypolicies
  verbs:
  - use

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ack:podsecuritypolicy:authenticated
  annotations:
    kubernetes.io/description: 'Allow all authenticated users to create privileged pods.'
  labels:
    kubernetes.io/cluster-service: "true"
    ack.alicloud.com/component: pod-security-policy
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ack:podsecuritypolicy:privileged
subjects:
  - kind: Group
    apiGroup: rbac.authorization.k8s.io
    name: system:authenticated

删除ACK默认Pod安全策略对应的集群角色绑定

警告

在删除ACK默认的Pod安全策略对应的集群角色绑定前必须先配置好自定义的Pod安全策略及其相应的RBAC绑定,否则所有用户、控制器、服务账号都将无法创建或更新Pod。

在配置好自定义的Pod安全策略及其相应的RBAC绑定后,您可以通过删除ACK默认Pod安全策略ack.privileged的集群角色绑定的方式来启用您自定义的Pod安全策略。

重要

请不要删除或修改名为ack.privilegedPod安全策略以及名为ack:podsecuritypolicy:privileged的集群角色,ACK集群的正常运行需要依赖这两个资源。

展开查看删除ACK默认Pod安全策略ack.privileged的集群角色绑定命令

$ cat <<EOF | kubectl delete -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ack:podsecuritypolicy:authenticated
  annotations:
    kubernetes.io/description: 'Allow all authenticated users to create privileged pods.'
  labels:
    kubernetes.io/cluster-service: "true"
    ack.alicloud.com/component: pod-security-policy
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ack:podsecuritypolicy:privileged
subjects:
  - kind: Group
    apiGroup: rbac.authorization.k8s.io
    name: system:authenticated
EOF

配置或恢复ACK默认的Pod安全策略

展开查看配置或恢复使用ACK默认的Pod安全策略及其RBAC绑定命令

cat <<EOF | kubectl apply -f -
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: ack.privileged
  annotations:
    kubernetes.io/description: 'privileged allows full unrestricted access to
      pod features, as if the PodSecurityPolicy controller was not enabled.'
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*'
  labels:
    kubernetes.io/cluster-service: "true"
    ack.alicloud.com/component: pod-security-policy
spec:
  privileged: true
  allowPrivilegeEscalation: true
  allowedCapabilities:
  - '*'
  volumes:
  - '*'
  hostNetwork: true
  hostPorts:
  - min: 0
    max: 65535
  hostIPC: true
  hostPID: true
  runAsUser:
    rule: 'RunAsAny'
  seLinux:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'
  readOnlyRootFilesystem: false

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: ack:podsecuritypolicy:privileged
  labels:
    kubernetes.io/cluster-service: "true"
    ack.alicloud.com/component: pod-security-policy
rules:
- apiGroups:
  - policy
  resourceNames:
  - ack.privileged
  resources:
  - podsecuritypolicies
  verbs:
  - use

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ack:podsecuritypolicy:authenticated
  annotations:
    kubernetes.io/description: 'Allow all authenticated users to create privileged pods.'
  labels:
    kubernetes.io/cluster-service: "true"
    ack.alicloud.com/component: pod-security-policy
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ack:podsecuritypolicy:privileged
subjects:
  - kind: Group
    apiGroup: rbac.authorization.k8s.io
    name: system:authenticated
EOF

常见问题

Pod创建失败,报错信息包含no providers available to validate pod request

问题现象

Pod创建失败,报错信息包含no providers available to validate pod request或者unable to validate against any pod security policy

解决方案

当前集群内预置的Pod安全策略被误删除,需手动恢复对应资源。详见配置或恢复ACK默认的Pod安全策略

Pod创建失败,报错信息包含PodSecurityPolicy: unable to admit pod: pod.spec.securityContext.sysctls[0]: Forbidden: unsafe sysctl

问题现象

Pod创建失败,报错信息包含PodSecurityPolicy: unable to admit pod: [pod.spec.securityContext.sysctls[0]: Forbidden: unsafe sysctl "***" is not allowed]

解决方案

出于安全考量,集群默认不允许创建使用“不安全”sysctl的 Pod。如需为特定应用开启此权限,可通过创建新的 Pod 安全策略来实现。

警告

请勿修改或删除以下集群预置的核心安全资源。ACK 集群的正常运行依赖这些核心资源。擅自修改可能导致集群功能异常,且相关更改可能会被系统自动重置。

  • 名为 ack.privileged的 Pod 安全策略。

  • 名称以 ack:podsecuritypolicy:开头的 Role、ClusterRole、RoleBinding 和 ClusterRoleBinding。

请通过新增 Pod 安全策略的方式,来配置所需的额外 sysctl策略。

  1. 使用以下内容,创建unsafe-sysctl-psp.yaml文件。

    可按需调整allowedUnsafeSysctls的参数取值。
    ---
    apiVersion: policy/v1beta1
    kind: PodSecurityPolicy
    metadata:
      name: psp.allow-unsafe-sysctls
    spec:
      allowedUnsafeSysctls:
      - '*'
      privileged: true
      allowPrivilegeEscalation: true
      allowedCapabilities:
      - '*'
      volumes:
      - '*'
      hostNetwork: true
      hostPorts:
      - min: 0
        max: 65535
      hostIPC: true
      hostPID: true
      runAsUser:
        rule: 'RunAsAny'
      seLinux:
        rule: 'RunAsAny'
      supplementalGroups:
        rule: 'RunAsAny'
      fsGroup:
        rule: 'RunAsAny'
      readOnlyRootFilesystem: false
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      name: podsecuritypolicy:allow-unsafe-sysctls
    rules:
    - apiGroups:
      - policy
      resourceNames:
      - psp.allow-unsafe-sysctls
      resources:
      - podsecuritypolicies
      verbs:
      - use
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      name: podsecuritypolicy:allow-unsafe-sysctls:authenticated
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: podsecuritypolicy:allow-unsafe-sysctls
    subjects:
      - kind: Group
        apiGroup: rbac.authorization.k8s.io
        name: system:authenticated
    
  2. 在集群内创建相应资源。

    kubectl create -f unsafe-sysctl-psp.yaml

    预期输出:

    podsecuritypolicy.policy/psp.allow-unsafe-sysctls created
    clusterrole.rbac.authorization.k8s.io/podsecuritypolicy:allow-unsafe-sysctls created
    clusterrolebinding.rbac.authorization.k8s.io/podsecuritypolicy:allow-unsafe-sysctls:authenticated created
  3. 自定义节点池的kubelet参数,允许使用不安全的sysctl。详见支持自定义的kubelet参数

  4. 部署一个使用不安全的sysctl的测试Pod。

    可按需调整sysctls参数内容。如果集群中仅有部分节点(例如特定节点池中的节点)的 kubelet配置了允许不安全的 sysctl,还需为 Pod 添加 nodeSelector,以确保Pod可被精确调度到目标节点上。
    cat <<EOF | kubectl apply -f -
    apiVersion: v1
    kind: Pod
    metadata:
      name: sysctl-example
    spec:
    #  nodeSelector:
    #    alibabacloud.com/nodepool-id: npd912756***  # 替换为目标节点池ID
      securityContext:
        sysctls:
        - name: net.ipv4.tcp_syncookies
          value: "1"
        - name: net.core.somaxconn
          value: "1024"
        - name: net.ipv4.tcp_max_syn_backlog
          value: "65536"
      containers:
      - name: test
        image: nginx
    EOF

    预期输出:

    如果Pod运行时提示SysctlForbidden事件,表明运行该Pod的节点上的kubelet未配置允许使用不安全的sysctl。请检查并调整 Pod 的 nodeSelector,确保被调度到已正确配置 kubelet 参数的节点。

    pod/sysctl-example created