部署CSI-Provisioner组件

更新时间: 2023-05-22 16:48:38

对于自建的Kubernetes集群,您可以部署阿里云CSI-Provisioner组件,以使用阿里云的存储服务,包括云盘、NAS和OSS。本文介绍如何部署CSI-Provisioner组件。

前提条件

  • 自建Kubernetes集群中已部署VNode,且集群版本为1.16及以上版本。具体操作,请参见接入VNode

  • 如果您的Kubernetes集群部署在线下IDC,请确保已打通IDC与阿里云的网络。

背景信息

CSI(Container Storage Interface)是当前Kubernetes社区推荐的插件实现方案,可以实现外挂数据卷(Volume)到Pod内,为Pod内部的应用服务提供存储服务。阿里云CSI-Provisioner组件遵循标准CSI规范,支持云盘、NAS等类型的阿里云存储服务,实现数据卷的创建、挂载和卸载能力。

操作步骤

  1. 创建用于部署CSI-Provisioner的YAML文件。

    将以下内容保存为csi-provisioner.yaml,其中ACCESS_KEY_IDACCESS_KEY_SECRET的value请替换为自身阿里云账号的AccessKey ID和AccessKey Secret,获取方式请参见获取AccessKey

    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: csi-admin
      namespace: kube-system
    ---
    kind: ClusterRole
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: alicloud-csi-plugin
    rules:
      - apiGroups: [""]
        resources: ["secrets"]
        verbs: ["get", "list"]
      - apiGroups: [""]
        resources: ["persistentvolumes"]
        verbs: ["get", "list", "watch", "update", "create", "delete", "patch"]
      - apiGroups: [""]
        resources: ["persistentvolumeclaims"]
        verbs: ["get", "list", "watch", "update"]
      - apiGroups: [""]
        resources: ["persistentvolumeclaims/status"]
        verbs: ["get", "list", "watch", "update", "patch"]
      - apiGroups: ["storage.k8s.io"]
        resources: ["storageclasses"]
        verbs: ["get", "list", "watch"]
      - apiGroups: ["storage.k8s.io"]
        resources: ["csinodes"]
        verbs: ["get", "list", "watch"]
      - apiGroups: [""]
        resources: ["events"]
        verbs: ["get", "list", "watch", "create", "update", "patch"]
      - apiGroups: [""]
        resources: ["endpoints"]
        verbs: ["get", "watch", "list", "delete", "update", "create"]
      - apiGroups: [""]
        resources: ["configmaps"]
        verbs: ["get", "watch", "list", "delete", "update", "create"]
      - apiGroups: [""]
        resources: ["nodes"]
        verbs: ["get", "list", "watch", "update"]
      - apiGroups: ["csi.storage.k8s.io"]
        resources: ["csinodeinfos"]
        verbs: ["get", "list", "watch"]
      - apiGroups: ["storage.k8s.io"]
        resources: ["volumeattachments"]
        verbs: ["get", "list", "watch", "update", "patch"]
      - apiGroups: ["snapshot.storage.k8s.io"]
        resources: ["volumesnapshotclasses"]
        verbs: ["get", "list", "watch", "create"]
      - apiGroups: ["snapshot.storage.k8s.io"]
        resources: ["volumesnapshotcontents"]
        verbs: ["create", "get", "list", "watch", "update", "delete"]
      - apiGroups: ["snapshot.storage.k8s.io"]
        resources: ["volumesnapshots"]
        verbs: ["get", "list", "watch", "update"]
      - apiGroups: ["apiextensions.k8s.io"]
        resources: ["customresourcedefinitions"]
        verbs: ["create", "list", "watch", "delete", "get", "update", "patch"]
      - apiGroups: ["coordination.k8s.io"]
        resources: ["leases"]
        verbs: ["get", "create", "list", "watch", "delete", "update"]
      - apiGroups: ["snapshot.storage.k8s.io"]
        resources: ["volumesnapshotcontents/status"]
        verbs: ["update"]
      - apiGroups: ["storage.k8s.io"]
        resources: ["volumeattachments/status"]
        verbs: ["patch"]
      - apiGroups: ["snapshot.storage.k8s.io"]
        resources: ["volumesnapshots/status"]
        verbs: ["update"]
      - apiGroups: ["storage.k8s.io"]
        resources: ["storageclasses"]
        verbs: ["get", "list", "watch"]
      - apiGroups: [""]
        resources: ["namespaces"]
        verbs: ["get", "list"]
      - apiGroups: [""]
        resources: ["pods"]
        verbs: ["get", "list", "watch"]
    ---
    kind: ClusterRoleBinding
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: alicloud-csi-plugin
    subjects:
      - kind: ServiceAccount
        name: csi-admin
        namespace: kube-system
    roleRef:
      kind: ClusterRole
      name: alicloud-csi-plugin
      apiGroup: rbac.authorization.k8s.io
    ---
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
       name: alicloud-disk-topology
    provisioner: diskplugin.csi.alibabacloud.com
    parameters:
        type: available
    reclaimPolicy: Delete
    volumeBindingMode: WaitForFirstConsumer
    allowVolumeExpansion: true
    ---
    kind: Deployment
    apiVersion: apps/v1
    metadata:
      name: csi-provisioner
      namespace: kube-system
    spec:
      selector:
        matchLabels:
          app: csi-provisioner
      strategy:
        rollingUpdate:
          maxSurge: 0
          maxUnavailable: 1
        type: RollingUpdate
      replicas: 2
      template:
        metadata:
          labels:
            app: csi-provisioner
        spec:
          affinity:
            nodeAffinity:
              preferredDuringSchedulingIgnoredDuringExecution:
              - weight: 1
                preference:
                  matchExpressions:
                  - key: node-role.kubernetes.io/master
                    operator: Exists
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                - matchExpressions:
                  - key: type
                    operator: NotIn
                    values:
                    - virtual-kubelet
            podAntiAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
              - labelSelector:
                  matchExpressions:
                  - key: app
                    operator: In
                    values:
                    - csi-provisioner
                topologyKey: kubernetes.io/hostname
          tolerations:
          - effect: NoSchedule
            operator: Exists
            key: node-role.kubernetes.io/master
          - effect: NoSchedule
            operator: Exists
            key: node.cloudprovider.kubernetes.io/uninitialized
          serviceAccount: csi-admin
          priorityClassName: system-node-critical
          hostNetwork: true
          containers:
            - name: external-disk-provisioner
              image: registry.cn-hangzhou.aliyuncs.com/acs/csi-provisioner:v3.0.0-3f86569-aliyun
              args:
                - "--csi-address=$(ADDRESS)"
                - "--feature-gates=Topology=True"
                - "--volume-name-prefix=disk"
                - "--strict-topology=true"
                - "--timeout=150s"
                - "--leader-election=true"
                - "--extra-create-metadata=true"
                - "--retry-interval-start=500ms"
                - "--v=5"
              env:
                - name: ADDRESS
                  value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
              imagePullPolicy: "Always"
              volumeMounts:
                - name: disk-provisioner-dir
                  mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
            - name: external-disk-attacher
              image: registry.cn-hangzhou.aliyuncs.com/acs/csi-attacher:v2.1.0
              args:
                - "--v=5"
                - "--csi-address=$(ADDRESS)"
                - "--leader-election=true"
              env:
                - name: ADDRESS
                  value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
              imagePullPolicy: "Always"
              volumeMounts:
                - name: disk-provisioner-dir
                  mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
            - name: external-disk-resizer
              image: registry.cn-hangzhou.aliyuncs.com/acs/csi-resizer:v0.3.0
              args:
                - "--v=5"
                - "--csi-address=$(ADDRESS)"
                - "--leader-election"
              env:
                - name: ADDRESS
                  value: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com/csi.sock
              imagePullPolicy: "Always"
              volumeMounts:
                - name: disk-provisioner-dir
                  mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
            - name: external-nas-provisioner
              image: registry.cn-hangzhou.aliyuncs.com/acs/csi-provisioner:v3.0.0-3f86569-aliyun
              args:
                - "--csi-address=$(ADDRESS)"
                - "--volume-name-prefix=nas"
                - "--timeout=150s"
                - "--leader-election=true"
                - "--retry-interval-start=500ms"
                - "--v=5"
              env:
                - name: ADDRESS
                  value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
              imagePullPolicy: "Always"
              volumeMounts:
                - name: nas-provisioner-dir
                  mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
            - name: external-nas-resizer
              image: registry.cn-hangzhou.aliyuncs.com/acs/csi-resizer:v0.3.0
              args:
                - "--v=5"
                - "--csi-address=$(ADDRESS)"
                - "--leader-election"
              env:
                - name: ADDRESS
                  value: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com/csi.sock
              imagePullPolicy: "Always"
              volumeMounts:
                - name: nas-provisioner-dir
                  mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
            - name: external-csi-snapshotter
              image: registry.cn-hangzhou.aliyuncs.com/acs/csi-snapshotter:v4.0.0-4f8a2d347-aliyun
              args:
                - "--v=5"
                - "--csi-address=$(ADDRESS)"
                - "--leader-election=true"
                - "--extra-create-metadata=true"
              env:
                - name: ADDRESS
                  value: /csi/csi.sock
              imagePullPolicy: Always
              volumeMounts:
                - name: disk-provisioner-dir
                  mountPath: /csi
            - name: external-snapshot-controller
              image: registry.cn-hangzhou.aliyuncs.com/acs/snapshot-controller:v4.0.0-4f8a2d347-aliyun
              args:
                - "--v=5"
                - "--leader-election=true"
              imagePullPolicy: Always
            - name: csi-provisioner
              securityContext:
                privileged: true
                capabilities:
                  add: ["SYS_ADMIN"]
                allowPrivilegeEscalation: true
              image: registry.cn-hangzhou.aliyuncs.com/acs/csi-plugin:v1.22.8-cd0788a-aliyun
              imagePullPolicy: "Always"
              args:
                - "--endpoint=$(CSI_ENDPOINT)"
                - "--v=2"
                - "--driver=nas,disk"
              env:
                - name: ACCESS_KEY_ID
                  value: "xxxx"
                - name: ACCESS_KEY_SECRET
                  value: "nnqxxxxx"
                - name: CSI_ENDPOINT
                  value: unix://var/lib/kubelet/csi-provisioner/driverplugin.csi.alibabacloud.com-replace/csi.sock
                - name: MAX_VOLUMES_PERNODE
                  value: "15"
                - name: SERVICE_TYPE
                  value: "provisioner"
                - name: NODE_MULTI_ZONE_ENABLE
                  value: "true"
              resources:
                requests:
                  cpu: 100m
                  memory: 128Mi
                limits:
                  cpu: 500m
                  memory: 1024Mi
              livenessProbe:
                httpGet:
                  path: /healthz
                  port: healthz
                  scheme: HTTP
                initialDelaySeconds: 10
                periodSeconds: 30
                timeoutSeconds: 5
                failureThreshold: 5
              readinessProbe:
                httpGet:
                  path: /healthz
                  port: healthz
                initialDelaySeconds: 5
                periodSeconds: 20
              ports:
                - name: healthz
                  containerPort: 11270
              volumeMounts:
                - name: host-dev
                  mountPath: /dev
                  mountPropagation: "HostToContainer"
                - name: host-log
                  mountPath: /var/log/
                - name: etc
                  mountPath: /host/etc
                - name: disk-provisioner-dir
                  mountPath: /var/lib/kubelet/csi-provisioner/diskplugin.csi.alibabacloud.com
                - name: nas-provisioner-dir
                  mountPath: /var/lib/kubelet/csi-provisioner/nasplugin.csi.alibabacloud.com
                - mountPath: /var/addon
                  name: addon-token
                  readOnly: true
          volumes:
            - name: disk-provisioner-dir
              emptyDir: {}
            - name: nas-provisioner-dir
              emptyDir: {}
            - name: host-log
              hostPath:
                path: /var/log/
            - name: host-dev
              hostPath:
                path: /dev
            - name: etc
              hostPath:
                path: /etc
            - name: addon-token
              secret:
                defaultMode: 420
                optional: true
                items:
                - key: addon.token.config
                  path: token-config
                secretName: addon.csi.token
  2. 部署组件。

    kubectl create -f csi-provisioner.yaml
  3. 查看部署结果。

    kubectl -n kube-system get pods

    预期返回如下:

    NAME                                    READY   STATUS    RESTARTS   AGE
    
    csi-provisioner-6d9595dfb6-m4mgx        8/8     Running   0          114s
    csi-provisioner-6d9595dfb6-zp44s        8/8     Running   0          114s
    ......
阿里云首页 弹性容器实例 相关技术圈