Node Resource Reservation Policy

更新时间:
复制 MD 格式

ACK reserves node CPU and memory for Kubernetes and system components, which creates a gap between total node capacity (Capacity) and schedulable resources (Allocatable). ACK applies a default reservation policy and also supports custom reservations through kubelet configuration.

Limits

Custom resource reservation requires Kubernetes 1.20 or later. To upgrade, see Manually upgrade a cluster.

Scope of impact

Custom resource reservation and its scope of impact

To change reservation values, configure kubelet for a node pool. Changes apply immediately to existing nodes. New nodes—including those from scaling or Add Existing Node—also use the updated configuration.

Important
  • Do not manually edit the kubelet configuration file on the command line. Doing so may cause configuration conflicts and unexpected behavior during node pool O&M.

  • Increasing resource reservation reduces allocatable resources. On nodes with high resource usage, this may trigger pod eviction. Set values carefully.

Default resource reservation scope of impact

ACK may update default reservation values over time. Updated values apply automatically when you perform node-level changes—for example, upgrading the cluster or node pool, or modifying kubelet parameters. Without such changes, existing nodes retain previous values for stability.

View node allocatable resources

View a node’s total capacity and allocatable resources:

kubectl describe node [NODE_NAME] | grep Allocatable -B 7 -A 6

Expected output:

Capacity:
  cpu:                4                 # Total CPU cores on the node.
  ephemeral-storage:  123722704Ki       # Total ephemeral storage on the node, in KiB.
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  memory:             7925980Ki         # Total memory on the node, in KiB.
  pods:               64
Allocatable:
  cpu:                3900m             # Allocatable CPU cores.
  ephemeral-storage:  114022843818      # Allocatable ephemeral storage, in bytes.
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  memory:             5824732Ki         # Allocatable memory, in KiB.
  pods:               64

Calculate allocatable resources

Use this formula to calculate allocatable resources:Allocatable = Capacity − Reserved − Eviction threshold

Formula details:

Resource reservation policy details

Reserved resources depend on several factors:

  • Higher-spec ECS instances run more pods, so ACK reserves more resources for Kubernetes components.

  • Windows nodes reserve more resources than Linux nodes for the OS and Windows Server components. See Create and manage Windows node pools.

ACK calculates reserved resources based on CPU and memory ranges—the total equals the sum across all ranges. In Kubernetes 1.28, ACK optimized the algorithm to reduce reservations. See Manually upgrade a cluster.

Reserved resources split equally between Kubernetes components (kubeReserved) and system processes (systemReserved). On a 4-core node, ACK reserves 80 millicores in Kubernetes 1.28+ (40 kubeReserved + 40 systemReserved), or 100 millicores in 1.20 through 1.27 (50 + 50).

CPU resource reservation policy

Kubernetes 1.28 and later

Total CPU reservation for compute nodes:

image

For a 32-core node, total CPU reservation is calculated as follows:

1000 × 6% + 1000 × 1% + 1000 × 2 × 0.5% + (32000 − 4000) × 0.25% = 150 millicores

Kubernetes 1.20 through 1.27

Total CPU reservation for compute nodes:

image

For a 32-core node, total CPU reservation is calculated as follows:

100 + (32000 − 4000) × 2.5% = 800 millicores

Memory resource reservation policy

Kubernetes 1.28 and later

Total memory reservation formula:

Total memory reservation = min(11 × ($max_num_pods) + 255, 25% × node memory). The final value is the smaller of 11 × ($max_num_pods) + 255 and 25% of node memory.

  • $max_num_pods: Maximum number of pods supported on the node.

    Note

    Maximum pod counts vary by network plug-in. See Maximum pods per node for details.

    You can log on to the Container Service Management Console and view the maximum pod count on the Nodes page.
    • Terway: Maximum pods per node = Maximum container-network pods + Host-network pods.

    • Flannel: Set by you when creating the cluster.

  • Node memory: Actual usable memory, in MiB.

For example, with Terway (shared ENI, multiple IPs) on an ecs.g7.16xlarge (256 GiB memory): maximum pods = (8 − 1) × 30 + 3 = 213. Total memory reservation = min(11 × (210 + 3) + 255, 256 × 1024 × 25%) = 2598 MiB.

Kubernetes 1.20 through 1.27

Total memory reservation for a compute node:

image

For a node with 256 GiB of memory, total memory reservation is calculated as follows:

4 × 25% + (8 − 4) × 20% + (16 − 8) × 10% + (128 − 16) × 6% + (256 − 128) × 2% = 11.88 GiB

Example default resource reservations for ACK nodes

Note

See Instance families for ECS instance type details.

Total node resources

Reserved resources (Kubernetes 1.28 and later)

Reserved resources (Kubernetes 1.20 through 1.27)

Sample instance type

CPU (cores)

Memory

(GiB)

Maximum Pods per Node

(using the Terway shared ENI multi-IP mode as an example)

CPU

(milicore)

Memory

(MiB)

CPU (millicores)

Memory (MiB)

ECS c7 instance types

2

4

15

70

420

100

1024

4

8

48

80

783

100

1843

8

16

48

90

783

200

2662

16

32

213

110

2598

400

3645

32

64

213

150

2598

800

5611

64

128

213

230

2598

1600

9543

128

256

423

390

4908

2400

12164

ECS ebmc7a instance types

256

512

453

710

5238

3040

17407

FAQ

How do I view total CPU and memory on a node?

CPU

View total CPU cores:

cat /proc/cpuinfo | grep processor

Expected output:

processor       : 0
processor       : 1
processor       : 2
processor       : 3

Memory

View total memory:

cat /proc/meminfo | grep MemTotal

Expected output:

MemTotal:        7660952 kB

Why is available memory less than the instance type specification?

Instance type specs include total memory (including OS usage), so available memory is always less. See After purchasing an instance, why does the displayed memory differ from the instance type specification?.

References