LVM数据卷

您可以在本地盘上通过VolumeGroup进行磁盘虚拟化,并且通过LVM数据卷切分磁盘给应用使用。本文介绍如何使用LVM数据卷。

背景信息

使用HostPath和LocalVolume都可以实现Pod对主机存储空间的访问,但均具有其局限性:

  • Kubernetes没有提供上述本地存储卷的生命周期管理能力,需要管理员手动管理、运维存储卷。

  • 多个Pod共享一个本地存储时,需要共享存储目录或者分别使用其子目录,无法做到容量隔离。

  • 多个Pod共享一个本地存储时,IOPS、吞吐等指标共享了整个存储空间,无法进行限制。

  • 新建Pod使用本地存储时,不了解各节点存储空间余量,无法进行合理的存储卷调度。

为此ACK提供了LVM数据卷方案,以解决上述问题。

功能介绍

  • LVM数据卷生命周期管理:卷自动创建、删除、挂载、卸载。

  • LVM数据卷扩容功能。

  • LVM卷的监控能力。

  • LVM卷的IOPS限制。

  • 节点本地存储管理:自动运维VolumeGroup。

  • LVM卷的集群容量感知能力。

注意事项

  • LVM本地存储卷,不支持数据的跨节点迁移,不适合在高可用场景中使用。

  • VolumeGroup本地存储容量感知为可选项(暂缓提供),VolumeGroup的初始化与本地存储资源有关,您需要手动进行初始化或者使用自动初始化本地存储资源进行初始化,两者都需要您对本地存储资源有比较深入的了解,否则建议您直接使用云上的存储资源(云盘、CNFS) 等。

实现架构

实现架构

基础的LVM功能(卷的生命周期管理、扩容、挂载、格式化等)由CSI-Provisioner和CSI-Plugin实现。

单元

详情

存储管理器

节点本地存储(VolumeGroup)的运维管理以及容量统计。可以不使用该组件,由Worker管理运维VolumeGroup。

存储信息中心

保存节点本地存储信息,包括容量、VolumeGroup等信息。

本地存储调度器

实现新建PVC对集群存储容量感知功能。

步骤一:为Plugin和Provisioner组件添加Secrets的RBAC权限

1.20集群及以下

与csi-plugin组件共用同一个ServiceAccount,但需要先为clusterrole/alicloud-csi-plugin增加Secrets的RBAC权限。

  1. 执行以下命令,查看集群clusterrole/alicloud-csi-plugin是否有Secrets的create权限。

    echo `JSONPATH='{range .rules[*]}{@.resources}:{@.verbs} \r\n
    {end}' \
     && kubectl get clusterrole  alicloud-csi-plugin -o jsonpath="$JSONPATH";` | grep secrets

    预期输出:

    ["secrets"]:["get","list"]
  2. clusterrole/alicloud-csi-plugin没有Secrets的create权限,请执行以下命令增加此权限。

     kubectl patch clusterrole alicloud-csi-plugin --type='json' -p='[{"op": "add", "path": "/rules/0", "value":{ "apiGroups": [""], "resources": ["secrets"], "verbs": ["create"]}}]'

    预期输出:

    clusterrole.rbac.authorization.k8s.io/alicloud-csi-plugin patched
  3. 执行以下命令,查看集群中clusterrole/alicloud-csi-plugin Secrets的create权限是否添加成功。

    echo `JSONPATH='{range .rules[*]}{@.resources}:{@.verbs} \r\n
    {end}' \
     && kubectl get clusterrole  alicloud-csi-plugin -o jsonpath="$JSONPATH";` | grep secrets

    预期输出:

    ["secrets"]:["create"] 
    ["secrets"]:["get","list"]

    从预期输出可得,集群中clusterrole/alicloud-csi-plugin Secrets的create权限已添加成功。

1.22集群及以上

请通过下列YAML创建和授权ServiceAccount:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: alibaba-cloud-csi-local
  namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: alibaba-cloud-csi-local
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "update", "create", "delete", "patch"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims/status"]
    verbs: ["get", "list", "watch", "update", "patch"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["csinodes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "update", "patch", "list", "watch"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["volumeattachments"]
    verbs: ["get", "list", "watch", "update", "patch"]
  - apiGroups: ["coordination.k8s.io"]
    resources: ["leases"]
    verbs: ["get", "create", "list", "watch", "delete", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["volumeattachments/status"]
    verbs: ["patch"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: alibaba-cloud-csi-local
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: alibaba-cloud-csi-local
subjects:
- kind: ServiceAccount
  name: alibaba-cloud-csi-local
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: alibaba-cloud-csi-local
  namespace: kube-system
rules:
- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["csi-local-plugin-cert"]
  verbs: ["get"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["create"]
- apiGroups: [""]
  resources: ["configmaps"]
  resourceNames: ["csi-plugin", "ack-cluster-profile"]
  verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: alibaba-cloud-csi-local
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: alibaba-cloud-csi-local
subjects:
- kind: ServiceAccount
  name: alibaba-cloud-csi-local
  namespace: kube-system

步骤二:部署Plugin和Provisioner组件

LVM CSI插件分为2个组件:Plugin(负责挂载、卸载LVM卷)和Provisioner(负责创建LVM卷和PV对象)。

说明

请将以下YAML中的{{ regionId }}替换为您集群所在的地域ID。

  • 展开查看部署LVM CSI-Plugin(1.20集群及以下)

    apiVersion: storage.k8s.io/v1beta1
    kind: CSIDriver
    metadata:
      name: localplugin.csi.alibabacloud.com
    spec:
      attachRequired: false
      podInfoOnMount: true
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      labels:
        app: csi-local-plugin
      name: csi-local-plugin
      namespace: kube-system
    spec:
      revisionHistoryLimit: 10
      selector:
        matchLabels:
          app: csi-local-plugin
      template:
        metadata:
          labels:
            app: csi-local-plugin
        spec:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: type
                        operator: NotIn
                        values:
                          - virtual-kubelet
          containers:
            - args:
                - '--v=5'
                - '--csi-address=/csi/csi.sock'
                - >-
                  --kubelet-registration-path=/var/lib/kubelet/csi-plugins/localplugin.csi.alibabacloud.com/csi.sock
              env:
                - name: KUBE_NODE_NAME
                  valueFrom:
                    fieldRef:
                      apiVersion: v1
                      fieldPath: spec.nodeName
              image: >-
                registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-node-driver-registrar:v1.3.0-6e9fff3-aliyun
              imagePullPolicy: Always
              name: driver-registrar
              resources: {}
              terminationMessagePath: /dev/termination-log
              terminationMessagePolicy: File
              volumeMounts:
                - mountPath: /csi
                  name: plugin-dir
                - mountPath: /registration
                  name: registration-dir
            - args:
                - '--endpoint=$(CSI_ENDPOINT)'
                - '--v=5'
                - '--nodeid=$(KUBE_NODE_NAME)'
                - '--driver=localplugin.csi.alibabacloud.com'
              env:
                - name: KUBE_NODE_NAME
                  valueFrom:
                    fieldRef:
                      apiVersion: v1
                      fieldPath: spec.nodeName
                - name: SERVICE_PORT
                  value: '11293'
                - name: CSI_ENDPOINT
                  value: >-
                    unix://var/lib/kubelet/csi-plugins/localplugin.csi.alibabacloud.com/csi.sock
              image: >-
                registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-plugin:v1.20.7-aafce42-aliyun
              imagePullPolicy: Always
              name: csi-localplugin
              resources: {}
              securityContext:
                allowPrivilegeEscalation: true
                capabilities:
                  add:
                    - SYS_ADMIN
                privileged: true
              terminationMessagePath: /dev/termination-log
              terminationMessagePolicy: File
              volumeMounts:
                - mountPath: /var/lib/kubelet
                  mountPropagation: Bidirectional
                  name: pods-mount-dir
                - mountPath: /dev
                  mountPropagation: HostToContainer
                  name: host-dev
                - mountPath: /var/log/
                  name: host-log
                - mountPath: /mnt
                  mountPropagation: Bidirectional
                  name: quota-path-dir
                - mountPath: /tls/local/grpc
                  name: tls-token-dir
                  readOnly: true
          dnsPolicy: ClusterFirst
          hostNetwork: true
          hostPID: true
          priorityClassName: system-node-critical
          restartPolicy: Always
          schedulerName: default-scheduler
          securityContext: {}
          serviceAccount: csi-admin
          serviceAccountName: csi-admin
          terminationGracePeriodSeconds: 30
          tolerations:
            - operator: Exists
          volumes:
            - name: tls-token-dir
              secret:
                defaultMode: 420
                secretName: csi-local-plugin-cert
            - hostPath:
                path: /var/lib/kubelet/csi-plugins/localplugin.csi.alibabacloud.com
                type: DirectoryOrCreate
              name: plugin-dir
            - hostPath:
                path: /var/lib/kubelet/plugins_registry
                type: DirectoryOrCreate
              name: registration-dir
            - hostPath:
                path: /var/lib/kubelet
                type: Directory
              name: pods-mount-dir
            - hostPath:
                path: /dev
                type: ''
              name: host-dev
            - hostPath:
                path: /var/log/
                type: ''
              name: host-log
            - hostPath:
                path: /mnt
                type: Directory
              name: quota-path-dir
      updateStrategy:
        rollingUpdate:
          maxUnavailable: 10%
        type: RollingUpdate
  • 展开查看部署LVM CSI-Provisioner(1.20集群及以下)

    apiVersion: apps/v1
    kind: Deployment
    metadata:
     labels:
       app: csi-local-provisioner
     name: csi-local-provisioner
     namespace: kube-system
    spec:
     selector:
       matchLabels:
         app: csi-local-provisioner
     strategy:
       rollingUpdate:
         maxSurge: 25%
         maxUnavailable: 25%
       type: RollingUpdate
     template:
       metadata:
         labels:
           app: csi-local-provisioner
       spec:
         affinity:
           nodeAffinity:
             preferredDuringSchedulingIgnoredDuringExecution:
             - preference:
                 matchExpressions:
                 - key: node-role.kubernetes.io/master
                   operator: Exists
               weight: 1
         containers:
         - args:
           - --csi-address=$(ADDRESS)
           - --feature-gates=Topology=True
           - --volume-name-prefix=local
           - --strict-topology=true
           - --timeout=150s
           - --extra-create-metadata=true
           - --enable-leader-election=true
           - --leader-election-type=leases
           - --retry-interval-start=500ms
           - --v=5
           env:
           - name: ADDRESS
             value: /socketDir/csi.sock
           image: registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-provisioner:v1.6.0-71838bd-aliyun
           imagePullPolicy: Always
           name: external-local-provisioner
           volumeMounts:
           - mountPath: /socketDir
             name: socket-dir
         - name: csi-localprovisioner
           securityContext:
             privileged: true
           image: registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-plugin:v1.20.7-aafce42-aliyun
           imagePullPolicy: "Always"
           args:
             - "--endpoint=$(CSI_ENDPOINT)"
             - "--v=2"
             - "--driver=localplugin.csi.alibabacloud.com"
           env:
           - name: CSI_ENDPOINT
             value: unix://var/lib/kubelet/csi-provisioner/localplugin.csi.alibabacloud.com/csi.sock
           - name: SERVICE_TYPE
             value: "provisioner"
           - name: SERVICE_PORT
             value: "11290"
           volumeMounts:
           - name: socket-dir
             mountPath: /var/lib/kubelet/csi-provisioner/localplugin.csi.alibabacloud.com
           - mountPath: /var/log/
             name: host-log
           - mountPath: /tls/local/grpc/
             name: tls-token-dir
         - args:
           - --v=5
           - --csi-address=$(ADDRESS)
           - --leader-election
           env:
           - name: ADDRESS
             value: /socketDir/csi.sock
           image: registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-resizer:v1.1.0-7b30758-aliyun
           imagePullPolicy: Always
           name: external-local-resizer
           volumeMounts:
           - mountPath: /socketDir/
             name: socket-dir
         hostNetwork: true
         serviceAccount: csi-admin
         tolerations:
         - effect: NoSchedule
           operator: Exists
           key: node-role.kubernetes.io/master
         - effect: NoSchedule
           operator: Exists
           key: node.cloudprovider.kubernetes.io/uninitialized
         volumes:
         - name: socket-dir
           emptyDir: {}
         - name: tls-token-dir
           emptyDir: {}
         - hostPath:
             path: /dev
             type: ""
           name: host-dev
         - hostPath:
             path: /var/log/
             type: ""
           name: host-log
         - hostPath:
             path: /mnt
             type: Directory
           name: quota-path-dir
         - hostPath:
             path: /var/lib/kubelet
             type: Directory
           name: pods-mount-dir
  • 展开查看部署LVM CSI-Plugin(1.22集群及以上)

    apiVersion: storage.k8s.io/v1
    kind: CSIDriver
    metadata:
      name: localplugin.csi.alibabacloud.com
    spec:
      attachRequired: false
      podInfoOnMount: true
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      labels:
        app: csi-local-plugin
      name: csi-local-plugin
      namespace: kube-system
    spec:
      revisionHistoryLimit: 10
      selector:
        matchLabels:
          app: csi-local-plugin
      template:
        metadata:
          labels:
            app: csi-local-plugin
        spec:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: type
                        operator: NotIn
                        values:
                          - virtual-kubelet
          containers:
            - args:
                - '--v=5'
                - '--csi-address=/csi/csi.sock'
                - >-
                  --kubelet-registration-path=/var/lib/kubelet/csi-plugins/localplugin.csi.alibabacloud.com/csi.sock
              env:
                - name: KUBE_NODE_NAME
                  valueFrom:
                    fieldRef:
                      apiVersion: v1
                      fieldPath: spec.nodeName
              image: >-
                registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-node-driver-registrar:v2.3.1-038aeb6-aliyun
              imagePullPolicy: Always
              name: driver-registrar
              resources: {}
              terminationMessagePath: /dev/termination-log
              terminationMessagePolicy: File
              volumeMounts:
                - mountPath: /csi
                  name: plugin-dir
                - mountPath: /registration
                  name: registration-dir
            - args:
                - '--endpoint=$(CSI_ENDPOINT)'
                - '--v=5'
                - '--nodeid=$(KUBE_NODE_NAME)'
                - '--driver=localplugin.csi.alibabacloud.com'
              env:
                - name: KUBE_NODE_NAME
                  valueFrom:
                    fieldRef:
                      apiVersion: v1
                      fieldPath: spec.nodeName
                - name: SERVICE_PORT
                  value: '11293'
                - name: CSI_ENDPOINT
                  value: >-
                    unix://var/lib/kubelet/csi-plugins/localplugin.csi.alibabacloud.com/csi.sock
              image: >-
                 registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-plugin:v1.24.3-55228c1-aliyun
              imagePullPolicy: Always
              name: csi-localplugin
              resources: {}
              securityContext:
                allowPrivilegeEscalation: true
                capabilities:
                  add:
                    - SYS_ADMIN
                privileged: true
              terminationMessagePath: /dev/termination-log
              terminationMessagePolicy: File
              volumeMounts:
                - mountPath: /var/lib/kubelet
                  mountPropagation: Bidirectional
                  name: pods-mount-dir
                - mountPath: /dev
                  mountPropagation: HostToContainer
                  name: host-dev
                - mountPath: /var/log/
                  name: host-log
                - mountPath: /mnt
                  mountPropagation: Bidirectional
                  name: quota-path-dir
                - mountPath: /tls/local/grpc
                  name: tls-token-dir
                  readOnly: true
          dnsPolicy: ClusterFirst
          hostNetwork: true
          hostPID: true
          priorityClassName: system-node-critical
          restartPolicy: Always
          securityContext: {}
          serviceAccountName: alibaba-cloud-csi-local
          terminationGracePeriodSeconds: 30
          tolerations:
            - operator: Exists
          volumes:
            - name: tls-token-dir
              secret:
                defaultMode: 420
                secretName: csi-local-plugin-cert
            - hostPath:
                path: /var/lib/kubelet/csi-plugins/localplugin.csi.alibabacloud.com
                type: DirectoryOrCreate
              name: plugin-dir
            - hostPath:
                path: /var/lib/kubelet/plugins_registry
                type: DirectoryOrCreate
              name: registration-dir
            - hostPath:
                path: /var/lib/kubelet
                type: Directory
              name: pods-mount-dir
            - hostPath:
                path: /dev
                type: ''
              name: host-dev
            - hostPath:
                path: /var/log/
                type: ''
              name: host-log
            - hostPath:
                path: /mnt
                type: Directory
              name: quota-path-dir
      updateStrategy:
        rollingUpdate:
          maxUnavailable: 10%
        type: RollingUpdate
  • 展开查看部署LVM CSI-Provisioner(1.22集群及以上)

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      labels:
        app: csi-local-provisioner
      name: csi-local-provisioner
      namespace: kube-system
    spec:
      progressDeadlineSeconds: 600
      replicas: 1
      revisionHistoryLimit: 10
      selector:
        matchLabels:
          app: csi-local-provisioner
      strategy:
        rollingUpdate:
          maxSurge: 25%
          maxUnavailable: 25%
        type: RollingUpdate
      template:
        metadata:
          labels:
            app: csi-local-provisioner
        spec:
          affinity:
            nodeAffinity:
              preferredDuringSchedulingIgnoredDuringExecution:
              - preference:
                  matchExpressions:
                  - key: node-role.kubernetes.io/master
                    operator: Exists
                weight: 1
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                - matchExpressions:
                  - key: type
                    operator: NotIn
                    values:
                    - virtual-kubelet
          containers:
          - args:
            - --csi-address=$(ADDRESS)
            - --feature-gates=Topology=True
            - --volume-name-prefix=local
            - --strict-topology=true
            - --timeout=150s
            - --extra-create-metadata=true
            - --enable-leader-election=true
            - --leader-election-type=leases
            - --retry-interval-start=500ms
            - --default-fstype=ext4
            - --v=5
            env:
            - name: ADDRESS
              value: /socketDir/csi.sock
            image: registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-provisioner:v3.0.0-080f01e64-aliyun
            imagePullPolicy: Always
            name: external-local-provisioner
            resources: {}
            volumeMounts:
            - mountPath: /socketDir
              name: socket-dir
          - args:
            - --endpoint=$(CSI_ENDPOINT)
            - --v=2
            - --driver=localplugin.csi.alibabacloud.com
            env:
            - name: CSI_ENDPOINT
              value: unix://var/lib/kubelet/csi-provisioner/localplugin.csi.alibabacloud.com/csi.sock
            - name: SERVICE_TYPE
              value: provisioner
            - name: SERVICE_PORT
              value: "11290"
            image: registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-plugin:v1.24.3-55228c1-aliyun
            imagePullPolicy: Always
            name: csi-localprovisioner
            resources: {}
            securityContext:
              privileged: true
            volumeMounts:
            - mountPath: /var/lib/kubelet/csi-provisioner/localplugin.csi.alibabacloud.com
              name: socket-dir
            - mountPath: /var/log/
              name: host-log
            - mountPath: /tls/local/grpc/
              name: tls-token-dir
          - args:
            - --v=5
            - --csi-address=$(ADDRESS)
            - --leader-election
            env:
            - name: ADDRESS
              value: /socketDir/csi.sock
            image: registry-vpc.{{ regionId }}.aliyuncs.com/acs/csi-resizer:v1.3-ca84e84-aliyun
            imagePullPolicy: Always
            name: external-local-resizer
            resources: {}
            volumeMounts:
            - mountPath: /socketDir/
              name: socket-dir
          dnsPolicy: ClusterFirst
          hostNetwork: true
          restartPolicy: Always
          securityContext: {}
          serviceAccountName: alibaba-cloud-csi-local
          terminationGracePeriodSeconds: 30
          tolerations:
          - operator: Exists
          volumes:
          - emptyDir: {}
            name: socket-dir
          - emptyDir: {}
            name: tls-token-dir
          - hostPath:
              path: /dev
              type: ""
            name: host-dev
          - hostPath:
              path: /var/log/
              type: ""
            name: host-log
          - hostPath:
              path: /mnt
              type: Directory
            name: quota-path-dir
          - hostPath:
              path: /var/lib/kubelet
              type: Directory
            name: pods-mount-dir

步骤三:使用LVM存储卷

使用CSI-Provisioner自动创建PV,有以下特点:

  • StorageClass中需要指定VolumeGroup名字。

  • 如果期望创建的PV在某个节点,需要给PVC添加Annotations:volume.kubernetes.io/selected-node: nodeName

  1. 使用以下模板创建StorageClass。

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
        name: csi-local
    provisioner: localplugin.csi.alibabacloud.com
    parameters:
        volumeType: LVM
        vgName: volumegroup1
        fsType: ext4
        lvmType: "striping"
        writeIOPS: "10000"
        writeBPS: "1M"
    reclaimPolicy: Delete
    volumeBindingMode: WaitForFirstConsumer
    allowVolumeExpansion: true

    参数

    描述

    volumeType

    表示本地存储类型为LVM(后续将支持其他类型本地存储)。

    vgName

    必选,VolumeGroup的名字。

    fsType

    格式文件系统类型。

    lvmType

    生成的LVM类型,支持linear(线性)、striping(条带化)。

    writeIOPS

    使用StorageClass动态创建每一个LV时,限制每一个LV每秒写IO的次数。

    writeBPS

    使用StorageClass动态创建每一个LV时,限制每一个LV每秒写IO的大小,单位为Byte。

  2. 使用以下模板创建PVC。

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: lvm-pvc
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 2Gi
      storageClassName: csi-local
  3. 使用以下模板创建应用。

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: deployment-lvm
      labels:
        app: nginx
    spec:
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: nginx:1.7.9
            volumeMounts:
              - name: lvm-pvc
                mountPath: "/data"
          volumes:
            - name: lvm-pvc
              persistentVolumeClaim:
                claimName: lvm-pvc
  4. 查看应用状态。

    1. 执行以下命令查看Pod信息。

      kubectl get pod

      预期输出:

      NAME                             READY   STATUS    RESTARTS   AGE
      deployment-lvm-9f798687c-m****   1/1     Running   0          9s
    2. 执行以下命令查看PVC信息。

      kubectl get pvc

      预期输出:

      NAME      STATUS   VOLUME                                      CAPACITY   ACCESS MODES   STORAGECLASS   AGE
      lvm-pvc   Bound    disk-afacf7a9-3d1a-45da-b443-24f8fb35****   2Gi        RWO            csi-local      16s
    3. 执行以下命令查看PV信息。

      kubectl get pv

      预期输出:

      NAME                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM             STORAGECLASS   REASON   AGE
      disk-afacf7a9-3d1a-45da-b443-24f8fb35****   2Gi        RWO            Delete           Bound    default/lvm-pvc   csi-local               12s
    4. 执行以下命令查看Pod挂载详情。

      kubectl exec -ti deployment-lvm-9f798687c-m**** sh -- df /data

      预期输出:

      Filesystem                                                                1K-blocks  Used   Available Use% Mounted on
      /dev/mapper/volumegroup1-disk--afacf7a9--3d1a--45da--b443--24f8fb35****   1998672    6144   1976144   1%   /data
    5. 执行以下命令列出/data下的目录。

      ls /data

      预期输出:

      lost+found
    6. 执行以下命令在/data下新增test目录并查看。

      touch /data/test
      ls /data

      预期输出:

      lost+found  test
    7. 执行以下命令退出登录。

      exit
    8. 执行以下命令删除Pod重建。

      kubectl delete pod deployment-lvm-9f798687c-m****

      预期输出:

      pod "deployment-lvm-9f798687c-m****" deleted
    9. 执行以下命令查看Pod信息。

      kubectl get pod

      预期输出:

      NAME                             READY   STATUS    RESTARTS   AGE
      deployment-lvm-9f798687c-j****   1/1     Running   0          2m19s
    10. 执行以下命令查看Pod挂载详情。

      kubectl exec deployment-lvm-9f798687c-j**** -- ls /data

      预期输出:

      lost+found  
      test
  5. 扩容LVM卷。

    1. 执行以下命令查看PVC信息。

      kubectl get pvc

      预期输出:

      NAME      STATUS   VOLUME                                      CAPACITY   ACCESS MODES   STORAGECLASS   AGE
      lvm-pvc   Bound    disk-afacf7a9-3d1a-45da-b443-24f8fb35****   2Gi        RWO            csi-local      6m50s
    2. 执行以下命令将PVC扩容到4 GiB。

      kubectl patch pvc lvm-pvc -p '{"spec":{"resources":{"requests":{"storage":"4Gi"}}}}'

      预期输出:

      persistentvolumeclaim/lvm-pvc patched
    3. 执行以下命令查看PVC信息。

      kubectl get pvc

      预期输出:

      NAME      STATUS   VOLUME                                      CAPACITY   ACCESS MODES   STORAGECLASS   AGE
      lvm-pvc   Bound    disk-afacf7a9-3d1a-45da-b443-24f8fb35****   4Gi        RWO            csi-local      7m26s
    4. 执行以下命令查看LVM卷是否扩容到4 GiB。

      kubectl exec deployment-lvm-9f798687c-j**** -- df /data

      预期输出:

      Filesystem                                                                1K-blocks  Used   Available Use% Mounted on
      /dev/mapper/volumegroup1-disk--afacf7a9--3d1a--45da--b443--24f8fb35****   4062912    8184   4038344   1%   /data
  6. 执行以下命令监控LVM卷。

    curl -s localhost:10255/metrics | grep lvm-pvc

    预期输出:

    kubelet_volume_stats_available_bytes{namespace="default",persistentvolumeclaim="lvm-pvc"} 1.917165568e+09
    kubelet_volume_stats_capacity_bytes{namespace="default",persistentvolumeclaim="lvm-pvc"} 1.939816448e+09
    kubelet_volume_stats_inodes{namespace="default",persistentvolumeclaim="lvm-pvc"} 122400
    kubelet_volume_stats_inodes_free{namespace="default",persistentvolumeclaim="lvm-pvc"} 122389
    kubelet_volume_stats_inodes_used{namespace="default",persistentvolumeclaim="lvm-pvc"} 11
    kubelet_volume_stats_used_bytes{namespace="default",persistentvolumeclaim="lvm-pvc"} 5.873664e+06

    可以将上述监控数据接入Prometheus并在前端展示。具体操作,请参见开源Prometheus监控