Implement topology-aware scheduling

Introduction

Machine learning and big data jobs demand high inter-pod bandwidth. The default Kubernetes scheduler spreads pods across the cluster, increasing network hops and latency. Native affinity scheduling (NodeAffinity/PodAffinity) can colocate pods in the same zone or rack, but has the following limitations.

• Pods cannot retry across topology domains. Native affinity scheduling pins the entire job to whichever domain the first pod lands in—if that domain lacks capacity, remaining pods stay Pending even when another domain could fit them all.

• Node labels exist only at the availability zone level, so pod affinity cannot target finer-grained topology domains.

Retry scheduling across multiple topology domains

Add a gang scheduling identifier to a job to enable simultaneous resource allocation and scheduling retries across multiple topology domains.

1. Add a gang scheduling identifier to the pod labels (Work with gang scheduling).

   ...
   labels:
     pod-group.scheduling.sigs.k8s.io/name: tf-smoke-gpu # tf-smoke-gpu is the name of the PodGroup. Specify a custom value.
     pod-group.scheduling.sigs.k8s.io/min-available: "3" # You can set this value to the number of pods in the job.
   ...

2. Add the topology-aware scheduling constraint to the pod's annotation.

   annotations:
     alibabacloud.com/topology-aware-constraint: {\"name\":\"test\",\"required\":{\"topologies\":[{\"key\":\"kubernetes.io/hostname\"}],\"nodeSelectors\":[{\"matchLabels\":{\"test\":\"abc\"}}]}}

   The alibabacloud.com/topology-aware-constraint value must be a valid JSON string:

   {
     "name": xxx, # Any name.
     "required": {
       "topologies": [
         {
           "key": xxx # The key that specifies the topology domain for affinity.
         }
       ],
       "nodeSelectors": [
         {
           # This structure follows the labelSelector format in the native Kubernetes nodeAffinity.
           "matchLabels": {},
           "matchExpressions": {} 
         }
       ]
     }
   }

   With this configuration, the scheduler places all pods labeled pod-group.scheduling.sigs.k8s.io/name: tf-smoke-gpu onto nodes matching the test=abc label. Example output:

   kubectl get pod -ojson | jq '.items[] | {"name":.metadata.name,"ann":.metadata.annotations["alibabacloud.com/topology-aware-constraint"], "node": spec.nodeName}'
   {
       "name": "nginx-deployment-basic-69f47fc6db-6****",
       "ann": "{\"name\": \"test\", \"required\": {\"topologies\":[{\"key\": \"kubernetes.io/hostname\"}], \"nodeSelectors\": [{\"matchLabels\": {\"test\": \"a\"}}]}} ",
       "node": "cn-shenzhen.10.0.2.4"
   }
   {
       "name":"nginx-deployment-basic-69f47fc6db-h****",
       "ann": "{\"name\": \"test\", \"required\": {\"topologies\":[{\"key\": \"kubernetes.io/hostname\"}], \"nodeSelectors\": [{\"matchLabels\": {\"test\": \"a\"}}]}} ",
       "node": "cn-shenzhen.10.0.2.4"
   }

Schedule to a low-latency deployment set

Some jobs require affinity within fine-grained topology domains for optimal performance. ECS low-latency deployment sets constrain node placement—see Best practices for associating deployment sets with node pools.

When creating a node pool with a low-latency deployment set, add a custom node label to distinguish it from other pools.

After completing the preceding setup, use the following annotation and labels to schedule a job within a low-latency deployment set.

1. Add a gang scheduling identifier to the pod labels (Work with gang scheduling).

   labels:
     pod-group.scheduling.sigs.k8s.io/name: xxx # xxx is the name of the PodGroup. Specify a custom value.
     pod-group.scheduling.sigs.k8s.io/min-available: "x" # You can set this value to the number of pods in the job.

2. Add the topology-aware scheduling constraint to the pod's annotation.

   Important

   Replace matchLabels with the custom node label for your low-latency deployment set and modify name as needed.

   annotations:
     alibabacloud.com/topology-aware-constraint: {\"name\":\"test\",\"required\":{\"topologies\":[{\"key\":\"alibabacloud.com/nodepool-id\"}],\"nodeSelectors\":[{\"matchLabels\":{\"np-type\":\"low-latency\"}}]}}