在Windows容器中使用云盘及基于SMB的文件存储

为Windows容器配置阿里云云盘

阿里云云盘是为阿里云服务器（ECS）提供的一种块存储服务，它具有低时延，高性能，耐用性好及高可靠等特点。

您可以通过以下步骤为Windows容器配置阿里云云盘。

步骤一：创建存储类StorageClass

默认情况下，若存储插件选择为Flexvolume，则在ACK集群创建时会定义Linux文件系统适用的存储类（StorageClass），而Windows文件系统适用的存储类需要自定义。

1. 使用以下YAML示例自定义Windows文件系统适用的存储类（StorageClass），并创建storageclass.yaml文件。
   YAML示例

   kind: StorageClass
   apiVersion: storage.k8s.io/v1
   metadata:
     name: alicloud-disk-common-windows
   provisioner: alicloud/disk
   parameters:
     type: cloud
     fstype: ntfs
   ---
   kind: StorageClass
   apiVersion: storage.k8s.io/v1
   metadata:
     name: alicloud-disk-efficiency-windows
   provisioner: alicloud/disk
   parameters:
     type: cloud_efficiency
     fstype: ntfs
   ---
   kind: StorageClass
   apiVersion: storage.k8s.io/v1
   metadata:
     name: alicloud-disk-ssd-windows
   provisioner: alicloud/disk
   parameters:
     type: cloud_ssd
     fstype: ntfs
   ---
   kind: StorageClass
   apiVersion: storage.k8s.io/v1
   metadata:
     name: alicloud-disk-essd-windows
   provisioner: alicloud/disk
   parameters:
     type: cloud_essd
     fstype: ntfs
   ---
   kind: StorageClass
   apiVersion: storage.k8s.io/v1
   metadata:
     name: alicloud-disk-available-windows
   provisioner: alicloud/disk
   parameters:
     type: available
     fstype: ntfs
   	  

   以下为一些StorageClass的配置示例说明：

   YAML示例	说明
   kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: alicloud-disk-ssd-windows provisioner: alicloud/disk parameters: type: cloud_ssd fstype: ntfs + allowVolumeExpansion: true # useless	目前Windows不支持盘区扩展。例如，配置的allowVolumeExpansion: true并不能起到预期扩展云盘的作用。
   kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: alicloud-disk-ssd-windows provisioner: alicloud/disk parameters: type: cloud_ssd fstype: ntfs + reclaimPolicy: Retain # only unmount	根据需要使用保留回收策略reclaimPolicy: Retain可以减少由于错误操作导致数据删除的问题。
   kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: alicloud-disk-ssd-windows provisioner: alicloud/disk parameters: type: cloud_ssd fstype: ntfs + regionId: cn-hangzhou + zoneId: cn-hangzhou-b	若缺少地域及可用区的定义，云盘可能会挂载失败。使用参数regionId及zoneId指定地域及可用区可以避免挂载失败。

2. 执行以下命令创建Windows环境下的StorageClass。

   kubectl create -f storageclass.yaml

步骤二：为Windows节点安装Flexvolume插件

1. 使用以下YAML示例创建flexvolume-windows.yaml文件。
   YAML示例

   apiVersion: apps/v1
   kind: DaemonSet
   metadata:
     labels:
       app: flexvolume-windows
     name: flexvolume-windows
     namespace: kube-system
   spec:
     selector:
       matchLabels:
         k8s-volume: flexvolume
     updateStrategy:
       type: RollingUpdate
     template:
       metadata:
         metadata:
         labels:
           k8s-volume: flexvolume
         annotations:
           scheduler.alpha.kubernetes.io/critical-pod: ""
       spec:
         tolerations:
         - operator: Exists
         /*since 1.18, we can specify "hostNetwork: true" for Windows workloads, so we can deploy an application without NetworkReady.*/
         hostNetwork: true
         priorityClassName: system-node-critical
         affinity:
           nodeAffinity:
             requiredDuringSchedulingIgnoredDuringExecution:
               nodeSelectorTerms:
               - matchExpressions:
                 - key: type
                   operator: NotIn
                   values:
                   - virtual-kubelet
                 - key: beta.kubernetes.io/os
                   operator: In
                   values:
                   - windows
               - matchExpressions:
                 - key: type
                   operator: NotIn
                   values:
                   - virtual-kubelet
                 - key: kubernetes.io/os
                   operator: In
                   values:
                   - windows
         containers:
         - name: acs-flexvolume
           command:
           - pwsh.exe
           - -NoLogo
           - -NonInteractive
           - -File
           - entrypoint.ps1
           /*根据不同集群的地域，您需修改以下镜像地址中的地域cn-hangzhou信息。*/
           image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/flexvolume-windows:v1.0.0
           imagePullPolicy: Always
           volumeMounts:
           - name: host-flexvolume-binary
             mountPath: c:/host/usr/libexec/kubernetes/kubelet-plugins/volume/exec
           - name: host-flexvolume-socket
             mountPath: c:/host/etc/kubernetes/flexvolume/socket
           - name: host-log
             mountPath: c:/var/log/alicloud
           /*use managed addon token from Alibaba Cloud*/
           - name: managed-addon-token
             mountPath: c:/var/addon/
             readOnly: true
        /* or specify the customized access key of Alibaba Cloud*/
        /* - name: customized-access-key*/
        /*   mountPath: c:/host/etc/.volumeak*/
        /* env:*/
        /* - name: ACCESS_KEY_ID*/
        /*   value: ""*/
        /* - name: ACCESS_KEY_SECRET*/
        /*   value: ""*/
         volumes:
         - name: host-flexvolume-binary
           hostPath:
             path: c:/usr/libexec/kubernetes/kubelet-plugins/volume/exec
             type: DirectoryOrCreate
         - name: host-flexvolume-socket
           hostPath:
             path: c:/etc/kubernetes/flexvolume/socket
             type: DirectoryOrCreate
         - name: host-log
           hostPath:
             path: c:/var/log/alicloud
             type: DirectoryOrCreate
         - name: managed-addon-token
           secret:
             defaultMode: 420
             optional: true
             items:
             - key: addon.token.config
               path: token-config
             secretName: addon.csi.token
        /* - name: customized-access-key*/
        /*   hostPath:*/
        /*     path: c:/etc/.volumeak*/
        /*     type: DirectoryOrCreate*/
      

2. 执行以下命令为所有的Windows工作节点安装Flexvolume插件。

   kubectl create -f flexvolume-windows.yaml

3. DaemonSet部署成功后，执行以下命令可以在每个WIndows节点上查看FlexVolume插件alicloud-disk-controller。

   ls C:\usr\libexec\kubernetes\kubelet-plugins\volume\exec\alicloud~disk.exe\

   预期输出：

   Directory: C:\usr\libexec\kubernetes\kubelet-plugins\volume\exec\alicloud~disk.exe
   
   Mode                LastWriteTime         Length Name
   ----                -------------         ------ ----
   -a----        8/12/2021   2:45 PM        5636096 disk.exe

步骤三：创建云盘并挂载到Windows容器中

以有状态工作负载（StatefulSet）为例，您可以在volumeClaimTemplates中定义需要的存储类（StroageClass），Windows工作节点上的FlexVolume插件会自动完成所有操作：验证您的账户信息，创建阿里云云盘，并挂载到阿里云服务器上，最后挂载到Windows容器中。

apiVersion: v1
kind: Service
metadata:
  labels:
    app: web-windows-disk
  name: web-windows-disk
spec:
  type: LoadBalancer
  ports:
    - port: 80
      protocol: TCP
      targetPort: 80
  selector:
    app: web-windows-disk
---
apiVersion: v1
kind: Service
metadata:
  name: web-windows-disk-headless
  labels:
    app: web-windows-disk
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  clusterIP: None
  selector:
    app: web-windows-disk
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web-windows-disk
spec:
  serviceName: "web-windows-disk-headless"
  selector:
    matchLabels:
      app: web-windows-disk
  replicas: 1
  template:
    metadata:
      labels:
        app: web-windows-disk
    spec:
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      tolerations:
      - key: os
        value: windows
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: beta.kubernetes.io/os
                operator: In
                values:
                - windows
            - matchExpressions:
              - key: kubernetes.io/os
                operator: In
                values:
                - windows
      containers:
      - name: windows
        /*根据不同集群的地域，您需要修改以下镜像地址中的地域cn-hangzhou信息。*/
        image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/sample-web-windows:v1.0.1
        volumeMounts:
          - name: ssd
            mountPath: c:/openresty/logs
  volumeClaimTemplates:
  - metadata:
      name: ssd
    spec:
      accessModes: 
      - ReadWriteOnce
      /*请选择您集群所在的地域、可用区所支持的云盘类型。*/
      storageClassName: alicloud-disk-ssd-windows
      resources:
        requests:
          storage: 50Gi 
   

kubectl get pv

NAME                     STATUS   VOLUME                   CAPACITY   ACCESS MODES   STORAGECLASS                AGE
ssd-web-windows-disk-0   Bound    d-wz9dihqdkusysc56****   50Gi       RWO            alicloud-disk-ssd-windows   5s

步骤四：验证云盘的持久化存储

为Windows容器配置基于SMB的文件存储

阿里云为ECS提供阿里云文件存储NAS的分布式文件系统。阿里云文件存储NAS基于POSIX文件接口，天然适配原生操作系统，提供共享访问，同时保证数据一致性和锁互斥。

SMB（Server Message Block）是阿里云文件存储NAS实现的一种协议，在Windows平台上工作良好。和阿里云云盘FlexVolume插件不同的是，使用SMB Flexvolume插件并不依赖Linux工作节点或Kubernetes控制器。

没有Kubernetes控制器意味着SMB FlexVolume插件仅能处理静态的持久化存储卷（PersistentVolume）资源对象，而无法利用存储类（StorageClass）动态分配卷。

步骤一：为Windows节点安装Flexvolume插件

为所有的Windows工作节点安装Flexvolume插件。具体操作，请参见为Windows容器配置阿里云云盘的步骤二：为Windows节点安装Flexvolume插件。

步骤二：使用SMB创建持久卷并挂载到Windows容器中

1. 创建持久化存储卷。
   使用以下YAML示例创建PV及PVC，向阿里云文件系统NAS申请10 GiB的文件存储。

   YAML示例

   apiVersion: v1
   kind: PersistentVolume
   metadata:
     labels:
       alicloud-pvname: pv-smb-10g
     name: pv-smb-10g
   spec:
     persistentVolumeReclaimPolicy: Retain
     accessModes:
     - ReadWriteMany
     capacity:
       storage: 10Gi
     flexVolume:
       driver: alicloud/smb.exe
       options:
         server: 25f3f4819c-e****.cn-hangzhou.nas.aliyuncs.com
         path: /myshare/web-windows-logs
         user: workgroup/Administrator
         password: ******
   ---
   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: web-windows-smb
   spec:
     selector:
       matchLabels:
         alicloud-pvname: pv-smb-10g
     accessModes:
     - ReadWriteMany
     resources:
       requests:
         storage: 10Gi
      

   配置项的部分参数说明如下所示：

   参数	说明
   driver	挂载存储卷的驱动类型，必须为alicloud/smb.exe。
   server	NAS文件系统中挂载点的挂载地址，必须与Windows节点的VPC在同一地域。
   path	NAS在容器中挂载的位置，必须以/myshare/为父目录并创建子目录。例如，/myshare/web-windows-logs。
   user	登录ECS的用户名，建议使用workgroup/administrator。
   password	登录ECS的用户密码。

2. 使用以下YAML示例创建Service及Deployment，并挂载已创建的持久化存储卷。
   YAML示例

   apiVersion: v1
   kind: Service
   metadata:
     labels:
       app: web-windows-smb
     name: web-windows-smb
   spec:
     type: LoadBalancer
     ports:
       - port: 80
         protocol: TCP
         targetPort: 80
     selector:
       app: web-windows-smb
   ---
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: web-windows-smb
   spec:
     selector:
       matchLabels:
         app: web-windows-smb
     replicas: 1
     template:
       metadata:
         labels:
           app: web-windows-smb
       spec:
         restartPolicy: Always
         terminationGracePeriodSeconds: 30
         tolerations:
         - key: os
           value: windows
         affinity:
           nodeAffinity:
             requiredDuringSchedulingIgnoredDuringExecution:
               nodeSelectorTerms:
               - matchExpressions:
                 - key: beta.kubernetes.io/os
                   operator: In
                   values:
                   - windows
               - matchExpressions:
                 - key: kubernetes.io/os
                   operator: In
                   values:
                   - windows
         containers:
         - name: windows
           /*根据不同集群的地域，您需要修改以下镜像地址中的地域cn-hangzhou信息。*/
           image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/sample-web-windows:v1.0.1
           volumeMounts:
             - name: disk-smb
               mountPath: c:/openresty/logs
         volumes:
         - name: disk-smb
           persistentVolumeClaim:
             claimName: web-windows-smb
      

3. Deployment部署完成后，执行以下命令查看应用及持久化存储卷（PersistentVolume）资源对象。
   • 执行以下命令查看部署的工作负载Deployment。

     kubectl get pod

     预期输出：

     NAME                               READY   STATUS              RESTARTS   AGE
     web-windows-smb-584676df65-t4mmh   1/1     Running             0          26d

   • 执行以下命令查看持久化存储卷（PersistentVolume）资源对象。

     kubectl get pv

     预期输出：

     NAME         CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                       STORAGECLASS      REASON   AGE
     pv-smb-10g   10Gi       ReadWriteMany  Retain           Bound       default/web-windows-smb                                10d

步骤四：验证NAS的持久化存储