ACK Pro Preset Control Plane Observability

更新时间:
复制 MD 格式

ACK Pro preset control plane allocates dedicated baseline resources to control plane components for consistent performance. You can monitor core capacity metrics and resource utilization to evaluate whether your cluster requires a tier upgrade.

Capacity planning monitoring

Core capacity metrics

The tier specifications of the preset control plane are defined by the following three core capacity metrics. Each metric reflects the processing capability of the control plane in a different dimension.

Metric

Unit

PromQL

Description

API Request Concurrency

Seats

sum(apiserver_flowcontrol_current_executing_seats)

The tier specification defines the maximum API request concurrency in terms of Seats. The preset control plane ensures request processing capability in high concurrency scenarios by allocating dedicated baseline resources to the API server. This monitoring metric is based on the Kubernetes API Priority and Fairness (APF) mechanism and reflects the number of concurrent requests currently being processed.

Pod Scheduling Rate

Pods/sec

rate(scheduler_schedule_attempts_total{result="scheduled"}[2m])

The rate at which the cluster scheduler schedules Pods to nodes per second. The preset control plane ensures deterministic scheduling rates through fixed resource quota, preventing scheduling performance fluctuations caused by control plane elastic scaling.

etcd Database Size

GB

max(etcd_mvcc_db_total_size_in_use_in_bytes)

The capacity for storing full data of the cluster. When the database usage approaches the upper limit of the current tier, upgrade the tier or clean up unused resources in a timely manner.

View core capacity dashboard

  1. On the Cluster Information page of the target cluster, click the Cluster Monitoring tab.

  2. Compare the following capacity metrics with the ACK Pro preset control plane tier specifications for tier selection.

    image

    The dashboard displays the following panels:

    • API Request Concurrency: Corresponds to the Seats specification of the tier, reflecting the concurrent processing pressure on the API server.

    • Pod Scheduling Rate: Shows the number of Pod scheduling requests processed per second to determine whether the scheduler has a bottleneck.

    • etcd Database Size (Cluster Database Size): Shows etcd storage usage to verify that the data volume stays within the safe limit of the current tier.

Tier switching reference criteria

Before you evaluate whether to upgrade the tier, review the tier specifications to obtain the specific capacity values for your current tier.

When any of the following conditions occurs in the cluster, evaluate whether to upgrade the tier:

Dimension

Upgrade reference condition

Description

API Request Concurrency

The current concurrent Seats continuously reaches 80% or more of the current tier upper limit.

The API server concurrent processing capability is approaching saturation. Request queuing or HTTP 429 throttling may occur.

Pod Scheduling Rate

The scheduling rate continuously approaches the current tier upper limit, and Pending Pods are accumulating.

The scheduler has become a bottleneck and workload scaling is constrained.

etcd Database Size

The database usage exceeds 80% of the current tier capacity.

Insufficient storage space may cause etcd to reject writes, affecting cluster state changes.

Control Plane Component Resource Utilization

The CPU or memory utilization of any control plane component (such as kube-apiserver, etcd, and kube-scheduler) continuously stays in high status (utilization ≥ 80%).

The resource allocation of the current tier is under pressure, posing a risk of being unable to support the actual workload. Upgrade the tier to obtain a larger resource quota.

The above conditions are reference recommendations. Upgrade decisions should be based on a comprehensive evaluation of your business scenarios, peak load periods, and duration. Observe metric trends rather than momentary spikes, and plan upgrades in advance before peak load periods.

Resource utilization monitoring and alerting

View resource utilization

  1. On the product page of the target cluster, in the left-side navigation pane, choose Operations > Prometheus Monitoring.

  2. Click the Key Component Monitoring tab, and then switch between the component tabs. Focus on the memory and CPU resource utilization of the kube-apiserver, etcd, and kube-scheduler components.

    image

    The resource utilization metrics are displayed for each component.

Taking kube-apiserver as an example, resource utilization is measured by the following metrics:

Resource type

Metric

Description

Memory Utilization

resource_utilization_level{resource="memory", container="kube-apiserver", utilization_level="high"}

A value of 1 indicates that memory utilization ≥ 80%.

Memory Utilization

resource_utilization_level{resource="memory", container="kube-apiserver", utilization_level="normal"}

A value of 1 indicates that memory utilization < 80%.

CPU Utilization

resource_utilization_level{resource="cpu", container="kube-apiserver", utilization_level="high"}

A value of 1 indicates that CPU utilization ≥ 80%.

CPU Utilization

resource_utilization_level{resource="cpu", container="kube-apiserver", utilization_level="normal"}

A value of 1 indicates that CPU utilization < 80%.

When the utilization metrics continuously stay in high status (utilization ≥ 80%), the current control plane resources are under pressure. Upgrade the tier in a timely manner.

The above table uses kube-apiserver as an example. The container field also applies to other control plane components such as etcd, kube-scheduler, and kube-controller-manager.

Control plane component resource utilization alerting

ACK Pro preset control plane is preconfigured with control plane component resource utilization alerts. When the CPU or memory utilization of any control plane component continuously exceeds the threshold, the system automatically triggers alert notifications.

Property

Description

Alert Name

High Memory Utilization of Control Plane Managed Component / High CPU Utilization of Control Plane Managed Component.

Trigger Condition

The CPU or memory utilization of a control plane managed component (such as kube-apiserver, etcd, and kube-scheduler) continuously remains ≥ 80%.

Notification Method

Notifications are sent through the alert contact group configured for the cluster. Supported channels include SMS, email, and DingTalk.

Recommended Action

After receiving an alert, log on to the console to check the resource utilization status. Upgrade the tier in a timely manner to prevent control plane performance degradation or service unavailability.

Alert example:

Alert Name: High Memory Utilization of Control Plane Managed Component_<cluster-name>
Alert Level: P1
Cluster Name: <cluster-name>
Alert Content: High memory utilization of control plane managed component

References