Schedule pods to ECI on virtual nodes

更新时间:
复制 MD 格式

ACK virtual nodes let you quickly schedule pods to run on Elastic Container Instance (ECI) without purchasing or managing ECS nodes. This topic describes how to schedule pods to run on ECI in an ACK One registered cluster.

How it works

Alibaba Cloud Elastic Container Instance (ECI) is a serverless elastic compute service designed for containers. It provides a maintenance-free, strongly isolated, and quick-to-start container runtime environment. When you use ECI, you do not need to purchase or manage underlying ECS nodes, allowing you to focus on your applications instead of infrastructure. You can create ECIs on demand and pay only for the resources configured for your containers on a pay-as-you-go, per-second basis.

Typically, your cluster has at least one group of IDC nodes. When you create a pod, it is scheduled to run on one of these nodes. This architecture works well for services with stable traffic. If your business has unpredictable traffic spikes, you can use virtual nodes to schedule pods directly to ECI. This approach eliminates node creation time, prevents idle node resources, and reduces costs.

Prerequisites

Step 1: Configure RAM permissions

Use onectl

  1. Install and configure onectl on your local machine. For more information, see Manage registered clusters using onectl.

  2. Run the following command to configure RAM permissions for the ack-virtual-node component.

    onectl ram-user grant --addon ack-virtual-node

    Expected output:

    Ram policy ack-one-registered-cluster-policy-ack-virtual-node granted to ram user ack-one-user-ce313528c3 successfully.

Use the console

Before you install the component, you must grant it permissions to access cloud services. This requires creating a RAM user with the necessary permissions and then storing its AccessKey in a Kubernetes secret within your cluster.

  1. Create a RAM user.

  2. Optional: Create a custom policy with the following content.

    Custom policy template

    {
        "Version": "1",
        "Statement": [
            {
                "Action": [
                    "vpc:DescribeVSwitches",
                    "vpc:DescribeVpcs"
                ],
                "Resource": "*",
                "Effect": "Allow"
            },
            {
                "Action": [
                    "eci:CreateContainerGroup",
                    "eci:DeleteContainerGroup",
                    "eci:DescribeContainerGroups",
                    "eci:DescribeContainerGroupStatus",
                    "eci:DescribeContainerGroupEvents",
                    "eci:DescribeContainerLog",
                    "eci:UpdateContainerGroup",
                    "eci:UpdateContainerGroupByTemplate",
                    "eci:CreateContainerGroupFromTemplate",
                    "eci:RestartContainerGroup",
                    "eci:ExportContainerGroupTemplate",
                    "eci:DescribeContainerGroupMetric",
                    "eci:DescribeMultiContainerGroupMetric",
                    "eci:ExecContainerCommand",
                    "eci:CreateImageCache",
                    "eci:DescribeImageCaches",
                    "eci:DeleteImageCache",
                    "eci:DescribeContainerGroupMetaInfos",
                    "eci:UpdateImageCache",
                    "eci:RestartContainer",
                    "eci:RestartContainers"
                ],
                "Resource": [
                    "*"
                ],
                "Effect": "Allow"
            },
            {
                "Action": [
                    "acc:DescribeZones",
                    "acc:CreateInstance",
                    "acc:UpdateInstance",
                    "acc:DeleteInstance",
                    "acc:RestartInstance",
                    "acc:DescribeInstances",
                    "acc:DescribeInstanceStatus",
                    "acc:DescribeInstanceEvents",
                    "acc:DescribeInstanceDetail",
                    "acc:DescribeMultiInstanceMetric",
                    "acc:DescribeContainerLog",
                    "acc:ResizeInstanceVolume",
                    "acc:CreateCustomResource",
                    "acc:UpdateCustomResource",
                    "acc:DeleteCustomResource",
                    "acc:DescribeCustomResources",
                    "acc:DescribeCustomResourceDetail"
                ],
                "Resource": "*",
                "Effect": "Allow"
            }
        ]
    }
  3. Attach a policy to the RAM user.

    You can attach the AliyunECIFullAccess, AliyunVPCReadOnlyAccess, and AliyunAccFullAccess system policies or the custom policy to the RAM user.

  4. Create an AccessKey for the RAM user.

    Warning

    For enhanced security, configure a network access control policy for the AccessKey to restrict access to trusted network environments and improve security. For more information, see AccessKey-based network access restriction policies.

  5. Use the AccessKey to create a secret named alibaba-addon-secret in the registered cluster.

    When you install the ack-virtual-node component, it automatically uses the credentials in this secret to access the required cloud services.

    kubectl -n kube-system create secret generic alibaba-addon-secret --from-literal='access-key-id=<your access key id>' --from-literal='access-key-secret=<your access key secret>'
    Note

    Replace <your access key id> and <your access key secret> with the AccessKey created in the previous step.

Step 2: Install the ack-virtual-node component

Use onectl

Run the following command to install the ack-virtual-node component:

onectl addon install ack-virtual-node

Expected output:

Addon ack-virtual-node, version **** installed.

Use the console

  1. Log on to the ACK console. In the left navigation pane, click Clusters.

  2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Components and Add-ons.

  3. On the Add-ons page, find the ack-virtual-node component and click Install in the component card.

    In the Message dialog box that appears, click OK. The ack-virtual-node component uses the cluster's default vSwitch and security group as the initial ECI configuration parameters. For more information, see Related operations.

Step 3: Schedule pods to run on ECI

After you deploy the ack-virtual-node component, you can use virtual nodes to schedule pods to ECI. This section describes common ways to schedule pods to ECI in a registered cluster. Before you schedule pods, make sure the virtual node is in the Ready state.

  1. Run the following command to check the status of the virtual node:

    kubectl get no | grep virtual-kubelet

    Expected output:

    virtual-kubelet-cn-hangzhou-b   Ready    agent                  18d   v1.20.11-aliyun.1

    The output indicates that the virtual node is in the Ready state.

  2. You can use one of the following three methods to schedule pods to run on ECI.

    Method 1: Configure pod labels (For clusters of version 1.16 or later)

    Add the label alibabacloud.com/eci=true to a pod to schedule it as an ECI pod on a virtual node. The following example shows how to do this:

    1. Run the following command to add the label to the pod:

    2. kubectl run nginx --image nginx -l alibabacloud.com/eci=true
    3. Run the following command to view the pod:

    4. kubectl get pod -o wide | grep virtual-kubelet
    5. Expected output:

    6. nginx-7fc9f746b6-r4xgx     0/1     ContainerCreating   0          20s   192.168.XX.XX   virtual-kubelet        <none>           <none>

    Method 2: Configure namespace labels

    Add the label alibabacloud.com/eci=true to a namespace. All pods subsequently created in that namespace will run as ECI pods on a virtual node. The following example shows how to do this:

    1. Run the following command to create a namespace:

    2. kubectl create ns vk
    3. Run the following command to add the label to the namespace:

    4. kubectl label namespace vk alibabacloud.com/eci=true
    5. Run the following command to schedule a pod to the virtual node in the namespace:

    6. kubectl -n vk run nginx --image nginx
    7. Run the following command to view the pod:

    8. kubectl -n vk get pod -o wide | grep virtual-kubelet
    9. Expected output:

    10. nginx-6f489b847d-vgj4d      1/1     Running             0          1m   192.168.XX.XX   virtual-kubelet        <none>           <none>

    Method 3: Specify a node name

    You can schedule a pod to a specific virtual node by adding nodeName: <YOUR_VIRTUAL_NODE_NAME> to the pod's spec. The pod then runs as an ECI pod on that virtual node. The following example shows how to do this:

    1. Create a file named nginx-deployment.yaml with the following content:

    2. apiVersion: apps/v1 # for versions before 1.8.0 use apps/v1beta1
      kind: Deployment
      metadata:
        name: nginx-deployment-basic
        labels:
          app: nginx
      spec:
        replicas: 2
        selector:
          matchLabels:
            app: nginx
        template:
          metadata:
            labels:
              app: nginx
          spec:
            nodeName: <YOUR_VIRTUAL_NODE_NAME>             # Replace with the name of your virtual node. Run 'kubectl get no' to find it.
            containers:
            - name: nginx
              image: nginx:1.7.9 # Replace with your image and tag.
              ports:
              - containerPort: 80
              resources:
                limits:
                  cpu: "500m"
    3. Run the following command to create the application:

    4. kubectl apply -f nginx-deployment.yaml
    5. Run the following command to view the pods:

    6. kubectl get pod -o wide | grep virtual-kubelet
    7. Expected output:

    8. nginx-6f489b847d-XXX      1/1     Running             0          1m   192.168.XX.XX   virtual-kubelet        <none>           <none>
      nginx-6f489b847d-XXX      1/1     Running             0          1m   192.168.XX.XX   virtual-kubelet        <none>           <none>

Related operations

Modify the ECI vSwitch configuration

You can modify the vSwitch for ECI pods. We recommend configuring multiple vSwitches across different zones. If one zone runs out of resources, the controller can create ECI pods in another zone.

Run the following command to modify the ECI vSwitch information:

kubectl edit configmap eci-profile -n kube-system

The following is a sample configuration:

data:
  enableClusterIp: "true"
  enableHybridMode: "false"
  enablePrivateZone: "false"
  securityGroupId: sg-2zeeyaaxlkq9sppl****
  selectors: ""
  vSwitchIds: vsw-2ze23nqzig8inprou****,vsw-2ze94pjtfuj9vaymf****     # The vSwitches for ECI pods. You can specify multiple vSwitch IDs, separated by commas (,).
  vpcId: vpc-2zeghwzptn5zii0w7****

For more information about eci-profile configurations, see Configure eci-profile.

Delete a virtual node

  1. Uninstall the ack-virtual-node component from the registered cluster.

    Use onectl

    Run the following command to uninstall the component:

    onectl addon uninstall ack-virtual-node

    Expected output:

    Addon ack-virtual-node uninstalled.

    Use the console

    On the Add-ons page, uninstall the ack-virtual-node component.

  2. Run the kubectl delete node <node name> command to delete the virtual node.

    Note

    After you uninstall the ack-virtual-node component, existing ECI pods in the cluster are not deleted.