Spread workloads across zones-Container Compute Service-阿里云帮助中心

High availability is a critical requirement for distributed applications. In an Alibaba Cloud Container Compute Service (ACS) cluster, you can use Kubernetes-native scheduling semantics to spread workloads across zones to achieve high availability. You can enable this cross-zone distribution by specifying a topology label in thetopologyKey field of a topology spread constraint. This topic describes the conditions and usage of topology spread constraints in ACS.

Prerequisites

Before you begin, make sure you have:

kube-scheduler installed at a version that meets the following requirements:
ACS cluster version Minimum scheduler version
1.31 v1.31.0-aliyun-1.2.0
1.30 v1.30.3-aliyun-1.1.1
1.28 v1.28.9-aliyun-1.1.0
acs-virtual-node installed at v2.12.0-acs.4 or later

ACS cluster version	Minimum scheduler version
1.31	v1.31.0-aliyun-1.2.0
1.30	v1.30.3-aliyun-1.1.1
1.28	v1.28.9-aliyun-1.1.0

Usage notes

Note

The Enable custom labels and scheduler for GPU-HPN nodes setting in the ACS scheduler component configuration is enabled by default in recent versions. For more information, see kube-scheduler.

When you use topology spread constraints in an ACS cluster, some fields have limitations for Pods that meet all of the following conditions:

The Pod's compute type is GPU-HPN.
The Pod's schedulerName is set to default-scheduler.
The Enable Custom Tags And Scheduler For GPU-HPN Nodes option is not selected in the scheduler component configuration.

Parameter	Description	Constraint
labelSelector	Finds matching Pods, which are counted to determine the number of Pods in the corresponding topology domain.	Pods of other compute types, such as general-purpose, performance-enhanced, and gpu-accelerated, are not included in the count.
matchLabelKeys	A list of Pod label keys that, together with `labelSelector`, defines the group of Pods for calculating distribution.
nodeAffinityPolicy	Defines how a Pod's nodeAffinity/nodeSelector is treated when calculating topology spread skew.	Not supported.
nodeTaintsPolicy	Defines how node taints are treated when calculating topology spread skew.	Not supported.

For general-purpose, performance-enhanced, and gpu-accelerated instances, the preceding constraints on topology spread do not apply. For more information about each field, see Pod Topology Spread Constraints.

Procedure

Run the following command to view the nodes in the cluster:

kubectl get node

Expected output:

NAME                            STATUS   ROLES   AGE     VERSION
virtual-kubelet-cn-hangzhou-i   Ready    agent   5h42m   v1.28.3-xx
virtual-kubelet-cn-hangzhou-j   Ready    agent   5h42m   v1.28.3-xx

Create a file named dep-spread-demo.yaml and add the following content to the file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: dep-spread-demo
  labels:
    app: spread-demo
spec:
  replicas: 4
  selector:
    matchLabels:
      app: spread-demo
  template:
    metadata:
      labels:
        app: spread-demo
    spec:
      containers:
      - name: spread-demo
        image: registry-cn-beijing.ack.aliyuncs.com/acs/stress:v1.0.4
        command:
        - "sleep"
        - "infinity"
      # Specifies the zone spread constraint. maxSkew=1 means the number of Pods can differ by at most one across zones.
      topologySpreadConstraints:
      - maxSkew: 1
        topologyKey: topology.kubernetes.io/zone
        whenUnsatisfiable: DoNotSchedule
        labelSelector:
          matchLabels:
            app: spread-demo

Run the following command to deploy dep-spread-demo to the cluster:
```
kubectl apply -f dep-spread-demo.yaml
```

Run the following command to view the distribution results of pods:

kubectl get pod -o wide

Expected output:

NAME                               READY   STATUS    RESTARTS   AGE     IP               NODE                            NOMINATED NODE   READINESS GATES
dep-spread-demo-7c656dbf5f-6twkc   1/1     Running   0          2m29s   192.168.xx.xxx   virtual-kubelet-cn-hangzhou-i   <none>           <none>
dep-spread-demo-7c656dbf5f-cgxr8   1/1     Running   0          2m29s   192.168.xx.xxx   virtual-kubelet-cn-hangzhou-j   <none>           <none>
dep-spread-demo-7c656dbf5f-f4fz9   1/1     Running   0          2m29s   192.168.xx.xxx   virtual-kubelet-cn-hangzhou-j   <none>           <none>
dep-spread-demo-7c656dbf5f-kc6xf   1/1     Running   0          2m29s   192.168.xx.xxx   virtual-kubelet-cn-hangzhou-i   <none>           <none>

The output indicates that four pods are distributed in two zones.