Topology-aware Scheduling

Introduction

In machine learning and big data analytics jobs, pods often have high network communication demands. By default, the native Kubernetes scheduler spreads pods evenly across the cluster. This increases the communication distance and leads to longer job completion times. To optimize job execution, you can deploy pods in the same availability zone or on the same rack to reduce network hops and latency. Kubernetes natively supports affinity scheduling through NodeAffinity and PodAffinity. However, this native affinity scheduling has the following limitations.

• Pods cannot retry scheduling across different topology domains. In native Kubernetes affinity scheduling, the scheduling of an entire job is determined by the placement of the first pod. If the location of the first pod cannot accommodate all the pods in the job, some pods will remain in a Pending state. The scheduler will not automatically switch to another availability zone, even if a different topology domain could accommodate the entire job.

• Currently, nodes only have labels at the availability zone level. As a result, pod affinity is limited to a single availability zone, and pods cannot be pinned to more fine-grained topology domains.

Retry scheduling across multiple topology domains

You can add a gang scheduling identifier to a job to enable simultaneous resource allocation for all pods and allow scheduling retries across multiple topology domains.

1. Add a gang scheduling identifier to the pod labels. To learn more about gang scheduling, see Work with gang scheduling.

   ...
   labels:
     pod-group.scheduling.sigs.k8s.io/name: tf-smoke-gpu # tf-smoke-gpu is the name of the PodGroup. Specify a custom value.
     pod-group.scheduling.sigs.k8s.io/min-available: "3" # You can set this value to the number of pods in the job.
   ...

2. Add the topology-aware scheduling constraint to the pod's annotation.

   annotations:
     alibabacloud.com/topology-aware-constraint: {\"name\":\"test\",\"required\":{\"topologies\":[{\"key\":\"kubernetes.io/hostname\"}],\"nodeSelectors\":[{\"matchLabels\":{\"test\":\"abc\"}}]}}

   The value of alibabacloud.com/topology-aware-constraint must be a valid JSON string with the following structure.

   {
     "name": xxx, # Any name.
     "required": {
       "topologies": [
         {
           "key": xxx # The key that specifies the topology domain for affinity.
         }
       ],
       "nodeSelectors": [
         {
           # This structure follows the labelSelector format in the native Kubernetes nodeAffinity.
           "matchLabels": {},
           "matchExpressions": {} 
         }
       ]
     }
   }

   After you apply this configuration, the scheduler places all pods with the label pod-group.scheduling.sigs.k8s.io/name: tf-smoke-gpu onto nodes that match the test=abc label. The following output is an example:

   kubectl get pod -ojson | jq '.items[] | {"name":.metadata.name,"ann":.metadata.annotations["alibabacloud.com/topology-aware-constraint"], "node": spec.nodeName}'
   {
       "name": "nginx-deployment-basic-69f47fc6db-6****",
       "ann": "{\"name\": \"test\", \"required\": {\"topologies\":[{\"key\": \"kubernetes.io/hostname\"}], \"nodeSelectors\": [{\"matchLabels\": {\"test\": \"a\"}}]}} ",
       "node": "cn-shenzhen.10.0.2.4"
   }
   {
       "name":"nginx-deployment-basic-69f47fc6db-h****",
       "ann": "{\"name\": \"test\", \"required\": {\"topologies\":[{\"key\": \"kubernetes.io/hostname\"}], \"nodeSelectors\": [{\"matchLabels\": {\"test\": \"a\"}}]}} ",
       "node": "cn-shenzhen.10.0.2.4"
   }

Schedule to a low-latency deployment set

In some scenarios, jobs require affinity within more fine-grained topology domains to achieve optimal performance. ECS provides low-latency deployment sets that constrain the placement of ECS nodes. For more information, see Best practices for associating deployment sets with node pools.

When you create a node pool that uses a low-latency deployment set, you must also add a custom node label to distinguish it from other node pools.

After you complete the preceding steps, use the following annotation and labels to schedule a job within a low-latency deployment set.

1. Add a gang scheduling identifier to the pod labels. To learn more about gang scheduling, see Work with gang scheduling.

   labels:
     pod-group.scheduling.sigs.k8s.io/name: xxx # xxx is the name of the PodGroup. Specify a custom value.
     pod-group.scheduling.sigs.k8s.io/min-available: "x" # You can set this value to the number of pods in the job.

2. Add the topology-aware scheduling constraint to the pod's annotation.

   Important

   Replace matchLabels with the custom node label for your low-latency deployment set and modify name as needed.

   annotations:
     alibabacloud.com/topology-aware-constraint: {\"name\":\"test\",\"required\":{\"topologies\":[{\"key\":\"alibabacloud.com/nodepool-id\"}],\"nodeSelectors\":[{\"matchLabels\":{\"np-type\":\"low-latency\"}}]}}