文档

在注册集群中启用集群API Server审计功能

更新时间:

审计(Auditing)产生于API Server内部,用于记录对Kubernetes API的请求以及请求结果。ACK集群提供API Server的审计日志,帮助集群管理人员排查“什么人在什么时间对什么资源做了什么操作”,可用于追溯集群操作历史、排查集群故障等,降低集群安全运维压力。

前提条件

已创建注册集群,并将自建Kubernetes集群接入注册集群。具体操作,请参见通过控制台创建注册集群

步骤一:在Master节点上配置审计配置策略文件

依次登录所有Master节点,在审计配置策略文件的路径/etc/kubernetes/audit-policy.yaml,请根据以下内容修改审计配置策略:

apiVersion: audit.k8s.io/v1beta1 # This is required.
kind: Policy
# Don't generate audit events for all requests in RequestReceived stage.
omitStages:
  - "RequestReceived"
rules:
  # The following requests were manually identified as high-volume and low-risk,
  # so drop them.
  - level: None
    users: ["system:kube-proxy"]
    verbs: ["watch"]
    resources:
      - group: "" # core
        resources: ["endpoints", "services"]
  - level: None
    users: ["system:unsecured"]
    namespaces: ["kube-system"]
    verbs: ["get"]
    resources:
      - group: "" # core
        resources: ["configmaps"]
  - level: None
    users: ["kubelet"] # legacy kubelet identity
    verbs: ["get"]
    resources:
      - group: "" # core
        resources: ["nodes"]
  - level: None
    userGroups: ["system:nodes"]
    verbs: ["get"]
    resources:
      - group: "" # core
        resources: ["nodes"]
  - level: None
    users:
      - system:kube-controller-manager
      - system:kube-scheduler
      - system:serviceaccount:kube-system:endpoint-controller
    verbs: ["get", "update"]
    namespaces: ["kube-system"]
    resources:
      - group: "" # core
        resources: ["endpoints"]
  - level: None
    users: ["system:apiserver"]
    verbs: ["get"]
    resources:
      - group: "" # core
        resources: ["namespaces"]
  # Don't log these read-only URLs.
  - level: None
    nonResourceURLs:
      - /healthz*
      - /version
      - /swagger*
  # Don't log events requests.
  - level: None
    resources:
      - group: "" # core
        resources: ["events"]
  # Secrets, ConfigMaps, and TokenReviews can contain sensitive & binary data,
  # so only log at the Metadata level.
  - level: Metadata
    resources:
      - group: "" # core
        resources: ["secrets", "configmaps"]
      - group: authentication.k8s.io
        resources: ["tokenreviews"]
  # Get repsonses can be large; skip them.
  - level: Request
    verbs: ["get", "list", "watch"]
    resources:
      - group: "" # core
      - group: "admissionregistration.k8s.io"
      - group: "apps"
      - group: "authentication.k8s.io"
      - group: "authorization.k8s.io"
      - group: "autoscaling"
      - group: "batch"
      - group: "certificates.k8s.io"
      - group: "extensions"
      - group: "networking.k8s.io"
      - group: "policy"
      - group: "rbac.authorization.k8s.io"
      - group: "settings.k8s.io"
      - group: "storage.k8s.io"
  # Default level for known APIs
  - level: RequestResponse
    resources:
      - group: "" # core
      - group: "admissionregistration.k8s.io"
      - group: "apps"
      - group: "authentication.k8s.io"
      - group: "authorization.k8s.io"
      - group: "autoscaling"
      - group: "batch"
      - group: "certificates.k8s.io"
      - group: "extensions"
      - group: "networking.k8s.io"
      - group: "policy"
      - group: "rbac.authorization.k8s.io"
      - group: "settings.k8s.io"
      - group: "storage.k8s.io"
  # Default level for all other requests.
  - level: Metadata

步骤二:在Master节点上配置Kube API Server文件

依次登录所有Master节点机器,在Kube API Server文件的路径/etc/kubernetes/manifests/kube-apiserver.yaml,请完成以下相关配置:

  • 根据以下示例添加command参数--audit-log-*

    ...
    spec:
      containers:
      - command:
        - kube-apiserver
        - --audit-log-maxbackup=10
        - --audit-log-maxsize=100
        - --audit-log-path=/var/log/kubernetes/kubernetes.audit
        - --audit-log-maxage=30
        - --audit-policy-file=/etc/kubernetes/audit-policy.yaml
        ...
  • 根据以下示例添加env参数aliyun_logs_audit-*

    ...
    spec:
      containers:
      - command:
        - kube-apiserver
        - --audit-log-maxbackup=10
        - --audit-log-maxsize=100
        - --audit-log-path=/var/log/kubernetes/kubernetes.audit
        - --audit-log-maxage=30
        - --audit-policy-file=/etc/kubernetes/audit-policy.yaml
        ...
        ...
        env:
        - name: aliyun_logs_audit-${cluster_id}
          value: /var/log/kubernetes/kubernetes.audit
        - name: aliyun_logs_audit-${cluster_id}_tags
          value: audit=apiserver
        - name: aliyun_logs_audit-${cluster_id}_product
          value: k8s-audit
        - name: aliyun_logs_audit-${cluster_id}_jsonfile
          value: "true"
        image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/kube-apiserver:v1.20.4-aliyun.1
    重要

    需要将{cluster_id}替换为您集群的Cluster ID。关于集群ID的获取,请参见查看集群信息

  • 根据以下示例挂载/etc/kubernetes/audit-policy.yaml到API Server Pod。

    ...
    spec:
      containers:
      - command:
        - kube-apiserver
        - --audit-log-maxbackup=10
        - --audit-log-maxsize=100
        - --audit-log-path=/var/log/kubernetes/kubernetes.audit
        - --audit-log-maxage=30
        - --audit-policy-file=/etc/kubernetes/audit-policy.yaml
        ...
        ...
        env:
        - name: aliyun_logs_audit-${cluster_id}
          value: /var/log/kubernetes/kubernetes.audit
        - name: aliyun_logs_audit-${cluster_id}_tags
          value: audit=apiserver
        - name: aliyun_logs_audit-${cluster_id}_product
          value: k8s-audit
        - name: aliyun_logs_audit-${cluster_id}_jsonfile
          value: "true"
        image: registry-vpc.cn-shenzhen.aliyuncs.com/acs/kube-apiserver:v1.20.4-aliyun.1
        ...
        ...
        volumeMounts:
        - mountPath: /var/log/kubernetes
          name: k8s-audit
        - mountPath: /etc/kubernetes/audit-policy.yaml
          name: audit-policy
          readOnly: true
        ...
        ...
      volumes:
      - hostPath:
          path: /var/log/kubernetes
          type: DirectoryOrCreate
        name: k8s-audit
      - hostPath:
          path: /etc/kubernetes/audit-policy.yaml
          type: FileOrCreate
        name: audit-policy
      ...

步骤三:安装logtail-ds日志组件

关于安装logtail-ds日志组件的具体操作,请参见步骤二:安装logtail-ds组件

后续步骤

关于如何使用和查看集群API Server审计功能,请参见使用集群API Server审计功能

  • 本页导读 (1)