Serverless elasticity in virtual nodes

What is a virtual node?

Nodes are the basic units that provide compute and storage resources to run workloads in ACK clusters. Most ACK clusters have at least one ECS node pool. After a pod is created, the kubelet schedules the pod to an ECS node in the node pool. This scheduling mode is suitable for applications with stable traffic. However, this mode cannot handle traffic spikes effectively because creating and starting ECS instances is time-consuming, even with ACK's auto scaling capabilities. Virtual nodes allow you to directly schedule pods to run on Alibaba Cloud Elastic Container Instance (ECI). This simplifies node operations and maintenance (O&M), eliminates idle nodes, and reduces costs.

Virtual nodes use the ack-virtual-node component to abstract computing resources. You can deploy workloads directly without managing the underlying infrastructure. The ack-virtual-node component automatically schedules application pods to run on ECI. ECI is a serverless container service where one ECI instance is equivalent to one pod. To deploy container applications on ECI, you only need to provide a container image. You pay only for the resources that your containers consume.

Benefits

Virtual nodes provide the following benefits.

• O&M-free: You do not need to manage the underlying resource pool, which reduces your O&M workload. As managed resources, virtual nodes eliminate routine Kubernetes node O&M tasks, such as system upgrades and security patching.

• Ultra-large capacity: Scale out to a maximum of 50,000 pods without capacity planning in advance.

  Important

  If many pods are associated with services, we recommend that you keep the number of pods below 20,000.

• Elasticity in seconds: Create thousands of pods in seconds. This ensures that sudden traffic spikes do not affect your services due to pod creation delays.

• Security isolation: Pods are created based on ECI. Each container instance is strongly isolated from others using lightweight virtualization and sandboxed container technology. Container instances do not affect each other.

• Cost-effective: Applications are created on demand and use the pay-as-you-go billing method. You are not charged for idle resources, which eliminates waste. The serverless architecture also leads to lower O&M costs.

Scenarios

Based on their features and benefits, virtual nodes are ideal for the following scenarios.

• Online services

  For online services that often experience traffic spikes, such as online education and e-commerce, virtual nodes support scaling in seconds. This prevents system failures caused by delayed scaling during traffic surges and avoids resource waste.

• Data processing

  When you process many concurrent online data tasks, such as Spark and Presto, you are no longer limited by the cost of underlying resources, which can restrict the concurrency of your tasks. You can quickly launch thousands of pods in a short time to meet the demands of online big data processing.

• AI tasks

  For AI tasks that do not need to run continuously and require a large amount of computing resources, such as model training and model inference, you do not need to reserve resources. You can use resources on demand and pay by the second to reduce AI inference costs. Virtual nodes also support elasticity in seconds to respond quickly to sudden task demands.

• CI/CD test environments

  For batch testing tasks in your CI/CD process, such as CI packaging, stress tests, and simulation tests, you can use virtual nodes to create and release container instances at any time. This supports on-demand usage and per-second billing, which provides low-cost, large-scale resource provisioning.

• Jobs and CronJobs

  Tasks such as Jobs and CronJobs do not need to run continuously. After a Job is complete, it automatically terminates, and the corresponding pod is deleted. Virtual nodes support automatic billing suspension and resource release after a task is complete to avoid resource waste.