DocsICP FilingConsole
官方文档
首页

Schedule Arm architecture or multi-architecture workloads to Arm virtual nodes

更新时间：
复制 MD 格式
产品详情
我的收藏

By default, and ACK Serverless clusters schedule all workloads to x86 virtual nodes. If your cluster has both Arm and non-Arm virtual nodes, such as x86 virtual nodes, you can use native Kubernetes scheduling features. This allows you to schedule Arm-only workloads to Arm virtual nodes, or preferentially schedule multi-architecture images to Arm virtual nodes.

Prerequisites

Usage notes

If your cluster runs a Kubernetes version earlier than 1.24, you must add a toleration for the kubernetes.io/arch=arm64:NoSchedule taint when using nodeSelector or nodeAffinity to schedule applications to Arm nodes. For clusters that run Kubernetes 1.24 or later, the scheduler automatically recognizes the kubernetes.io/arch=arm64:NoSchedule taint on Arm nodes, and you do not need to declare the toleration.

Billing

For ECS instance types that use the ARM architecture and their pricing, see:

Step 1: Add Arm virtual nodes

Before you deploy a workload on the Arm architecture in your cluster, you must first create an Arm virtual node. You can create a virtual node that supports the Arm architecture by configuring an ECI Profile. You can edit the eci-profile ConfigMap by using one of the following methods. For more information about ECI Profile, see Configure an eci-profile.

Console

  1. Log on to the ACK console. In the left navigation pane, click Clusters.

  2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Configurations > ConfigMaps.

  3. From the Namespace drop-down list, select kube-system. Find eci-profile and click Edit in the Actions column. Set the value of the enableLinuxArm64Node key to true. Then, click OK.

    Note

    If all vSwitches in your cluster are in zones that do not support Arm-based instances, you must first create a vSwitch in a supported zone. After the vSwitch is created, add its ID to the vSwitchIds field. For more information about how to create a vSwitch in a specific zone, see Create and manage vSwitches.

Kubectl

Prerequisites

Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.

Procedure

Run the following command to edit the ConfigMap:

kubectl edit configmap eci-profile -n kube-system

  1. Add or update the enableLinuxArm64Node parameter and set its value to true.

  2. Set the vSwitchIds parameter. Make sure that at least one vSwitch specified in the vSwitchIds list resides in a zone that supports Arm-based instances.

    Note

    If all vSwitches in your cluster are in zones that do not support Arm-based instances, you must first create a vSwitch in a supported zone. After the vSwitch is created, add its ID to the vSwitchIds field. For more information about how to create a vSwitch in a specific zone, see Create and manage vSwitches.

Step 2: Schedule to an Arm virtual node

Schedule Arm-only workloads

If your cluster has both Arm and non-Arm nodes and your application is Arm-only, you must schedule it to an Arm node. This prevents the pod from failing to start on a non-Arm node. By default, all Arm nodes have the label kubernetes.io/arch=arm64. You can use a nodeSelector or nodeAffinity to deploy the application to Arm nodes.

nodeSelector

You can add the following nodeSelector constraint to your pod specification to schedule it to an Arm virtual node. This constraint ensures that the workload is scheduled only to nodes that have the arm64 label. All Arm virtual nodes in an or ACK Serverless cluster have this label.

nodeSelector:
  kubernetes.io/arch: arm64 # Specify an Arm node.

You can use the following sample YAML to deploy a stateless application to an Arm virtual node.

Expand to view the YAML file

Note

The following YAML adds a toleration for the kubernetes.io/arch=arm64:NoSchedule taint. If your cluster is an ACK Pro cluster that runs Kubernetes 1.24 or later, the ACK scheduler automatically recognizes this taint and you do not need to add the toleration.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: only-arm
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      nodeSelector:
        kubernetes.io/arch: arm64 # Specify an Arm node.
      tolerations:
      # Tolerate the taint of the virtual node.
        - key: virtual-kubelet.io/provider
          operator: Exists
          effect: NoSchedule
      # Tolerate the taint on the Arm virtual node.
        - key: kubernetes.io/arch
          operator: Equal
          value: arm64
          effect: NoSchedule
      containers:
      - name: nginx
        image: alibaba-cloud-linux-3-registry.cn-hangzhou.cr.aliyuncs.com/alinux3/nginx_optimized:20240221-1.20.1-2.3.0

nodeAffinity

Prerequisites

The virtual node scheduling feature is enabled for the cluster, and the cluster and component versions meet the requirements.

Example

You can add the following nodeAffinity constraint to your pod specification to deploy the application to an Arm node. This constraint ensures that the pod is scheduled only to nodes that have the label kubernetes.io/arch=arm64.

When the pod specification contains this constraint, the scheduler automatically tolerates the kubernetes.io/arch=arm64:NoSchedule taint on the node.

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/arch
          operator: In
          values:
          - arm64

You can use the following sample YAML to deploy a stateless application to an Arm virtual node.

Expand to view the YAML file

Note

The following YAML adds a toleration for the kubernetes.io/arch=arm64:NoSchedule taint. If your cluster is an ACK Pro cluster that runs Kubernetes 1.24 or later, the ACK scheduler automatically recognizes this taint and you do not need to add the toleration.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: only-arm
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - arm64
      tolerations:
       # Tolerate the taint of the virtual node.
        - key: virtual-kubelet.io/provider
          operator: Exists
          effect: NoSchedule
       # Tolerate the taint on the Arm virtual node.
        - key: kubernetes.io/arch
          operator: Equal
          value: arm64
          effect: NoSchedule         
      containers:
      - name: nginx
        image: nginx

Schedule multi-architecture images

Prerequisites

The virtual node scheduling feature is enabled for the cluster, and the cluster and component versions meet the requirements.

Example

By default, and ACK Serverless clusters schedule all workloads to x86 virtual nodes. If x86 node resources are insufficient, the pods remain pending, waiting for available x86 resources. If your application uses a multi-architecture image that supports both x86 and Arm architectures, you must configure scheduling across both node architectures.

For example, you can configure node affinity to preferentially schedule workloads to either Arm or x86 virtual nodes. If resources of the preferred type are insufficient, the scheduler attempts to schedule the workload to a virtual node of the other architecture.

      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            preference:
              matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - arm64

Prefer Arm architecture

The following is a sample YAML for a workload that is preferentially scheduled to an Arm virtual node.

Expand to view the YAML file

Note

The following YAML adds a toleration for the kubernetes.io/arch=arm64:NoSchedule taint. If your cluster is an ACK Pro cluster that runs Kubernetes 1.24 or later, the ACK scheduler automatically recognizes this taint and you do not need to add the toleration.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: arm-prefer
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      tolerations:
      # Tolerate the taint of the virtual node.
      - key: virtual-kubelet.io/provider
        operator: Exists
        effect: NoSchedule
      # Tolerate the taint on the Arm virtual node.
      - key: kubernetes.io/arch
        operator: Equal
        value: arm64
        effect: NoSchedule
      # Preferentially schedule the workload to a node that uses the Arm architecture.
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            preference:
              matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - arm64
      containers:
      - name: my-container
        image: nginx

Prefer x86 architecture

Expand to view the YAML file

Note

The following YAML adds a toleration for the kubernetes.io/arch=arm64:NoSchedule taint. If your cluster is an ACK Pro cluster that runs Kubernetes 1.24 or later, the ACK scheduler automatically recognizes this taint and you do not need to add the toleration.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: amd-prefer
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      tolerations:
      # Tolerate the taint of the virtual node.
      - key: virtual-kubelet.io/provider
        operator: Exists
        effect: NoSchedule
      # Tolerate the taint on the Arm virtual node.
      - key: kubernetes.io/arch
        operator: Equal
        value: arm64
        effect: NoSchedule     
      # Preferentially schedule the workload to a node that uses the x86 architecture.
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            preference:
              matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - amd64
      containers:
      - name: my-container
        image: nginx

FAQ

Scheduling priority of Arm and x86 nodes

By default, the cluster scheduler prioritizes scheduling workloads to ECS instances. If ECS instance resources are insufficient, the scheduler then considers virtual nodes. Without modifying the weights of the scheduler scoring plugins, a pod may be scheduled to an x86 ECS instance if sufficient resources are available, even if you configure nodeAffinity to prioritize Arm nodes. Therefore, the nodeAffinity configurations in this topic only guarantee scheduling priority between different virtual node architectures (Arm vs. x86), not between virtual nodes and ECS instances.

Can I use Arm preemptible instances?

Yes, Arm-based preemptible instances are available. For more information, see Use preemptible instances.

Network configuration for Arm virtual nodes

After you create an or ACK Serverless cluster, configure the vSwitchIds field in the eci-profile to select a vSwitch in a zone that supports Arm-based instances. This ensures that the virtual nodes support the Arm architecture.

What are the limitations of using Arm architecture nodes in ACK Serverless cluster?

Currently, components from the App Marketplace are not supported on the Arm architecture. In the Component Center, only components in the following categories are supported:

  • Core components

  • Logging and monitoring

  • Storage

  • Network

Related documents

Previous：Scheduling overviewNext：Enable pod scheduling policies on virtual nodes