System health

Historical health details

Use the time selector to choose a specific time range. The historical health details section displays the cluster's health and anomaly status during that period.

• Anomaly event analysis

  The Anomaly Event Analysis panel shows all anomaly check items that affect the health of the cluster or its instances.

  • You can use the drop-down list to filter anomaly events by level. The anomaly levels are described below.

    • Notice: Certain monitoring or system behavior metrics have reached a questionable state, which may indicate a potential risk. O&M personnel should closely monitor these metrics to take action before the situation worsens. Common examples include CPU usage approaching a threshold or a rising trend in memory usage.

    • Warning: Monitoring metrics have crossed a configured alert threshold, which may affect the normal operation of services. O&M personnel should promptly review and address the related issues.

    • Critical: A check item shows clear signs of an anomaly or failure, which could directly impact system availability and business continuity. O&M personnel must intervene immediately to investigate and resolve the problem.

  • Use the Instance ID drop-down list to filter for anomaly events on a specific instance.

  • Diagnosis Status

    • No Diagnosis: No diagnosis has been run for the current anomaly item. This can happen for the following reasons:

      • No diagnosis tool is available for the current anomaly item.

      • The automatic diagnosis feature is in a cooldown period.

      • The anomaly event ended before you viewed the page, which means the anomaly has already resolved.

    • Diagnosing: A diagnosis is in progress for the current anomaly item.

    • Diagnosis Completed: The diagnosis for the current anomaly item is complete. Click View Diagnosis Report in the Actions column to view the report.

    • Diagnosis Failed: The diagnosis call failed. A common reason is that multiple diagnosis requests were initiated on the node simultaneously.

  • Anomaly Feedback

    You can use the feedback mechanism to adjust the anomaly detection algorithm and optimize the sensitivity of an anomaly item. For example, if the system continuously reports node I/O traffic anomalies that you believe are false positives, you can click the Feedback button in the Actions column for one of the anomaly items and adjust the scope and sensitivity.

    • If you choose to apply the change only to the current node, the new sensitivity setting will apply to the node I/O traffic detection for the current instance.

    • If you choose to apply the change to all nodes in the cluster, the new sensitivity setting will apply to the I/O traffic detection for all nodes in the current cluster.

  • OS Copilot intelligent analysis

    The node CPU usage detection, node kernel-mode CPU usage detection, node soft interrupt CPU usage detection, node memory usage detection, node kernel memory usage detection, and node OOM stall prediction and detection check items integrate with OS Copilot. You can click the OS Copilot icon next to an anomaly item to ask the assistant to analyze the root cause. In the Anomaly Event Analysis table, supported check items display an OS Copilot icon. Click the icon to open a chat with predefined prompts. After sending a prompt, OS Copilot analyzes the problem and generates an analysis report in the chat window. The report includes a Problem Overview (instance ID, region, analysis time, and symptoms), Core Metric Analysis (such as a CPU usage distribution table), and Key Findings (such as analysis conclusions on kernel-mode CPU percentage and idle time).

• Top 10 node health list

  The Top 10 Node Health List panel displays information for the 10 instances with the lowest health scores in the cluster, sorted from lowest to highest. Click Node Health in the Actions column to navigate to the corresponding instance's health page.

• Health score trend

  Use the time selector to view the historical trends of the overall health score and the scores for the four metric types. This helps you identify historical health issues in the cluster.

  

• Node health proportion and node issue proportion

  • The Node Health Proportion chart shows the distribution of all nodes across four health levels.

    • Healthy: health score ≥ 90

    • Sub-healthy: 80 ≤ health score < 90

    • Unhealthy: 60 ≤ health score < 80

    • Severely Unhealthy: health score < 60

  • The Node Issue Proportion chart shows the distribution of issue types, including saturation, latency, traffic, and errors.

    

Access the node health page

1. Go to the operating system console.

2. In the left-side navigation pane, click System Management.

3. On the Managed tab, find the target instance by its Instance ID/Name, and then click Node Health in the Actions column.

Node health

The following panels display the real-time health score and resource data for the selected instance.

• Node health score

  This panel displays the real-time health score of the selected instance. Health scores are categorized into the following levels:

  • Healthy: The instance's system or its Pods are in a healthy state. This level indicates that the instance may have Notice-level anomaly events, which have minimal or no impact on your workloads.

  • Sub-healthy: The instance's system or its Pods are in a relatively healthy state. This level indicates that the instance has Notice-level anomalies or contains sub-healthy or unhealthy Pods, which may have a minor impact on your workloads.

  • Unhealthy: The node's system or its Pods are in an unhealthy state. This level indicates that the instance has Warning-level anomalies or contains unhealthy or severely unhealthy Pods. You must address these anomalies immediately. The impact on your workloads is noticeable.

  • Severely Unhealthy: The node's system or its Pods are in a very unhealthy state. This level indicates that the instance has Critical-level anomalies or contains severely unhealthy Pods. You must address these anomalies immediately to prevent a severe impact on your workloads.

• Node health metrics

  This panel displays scores for four types of metrics for the selected instance. You can use this data to identify which metric type is affecting the instance's health score. The metrics are categorized as follows:

  • Saturation: Measures the load capacity of the operating system, typically expressed as the utilization rate of system resources, such as memory, CPU, and disk.

  • Latency: The time required for the operating system to process a specific request, such as task scheduling, memory allocation, I/O, or network operations.

  • Traffic: Statistics on the inbound and outbound data flowing through the system. This metric indicates the demand placed on the service. For an operating system, traffic can refer to network and I/O traffic.

  • Errors: The number of error requests or error events occurring in the operating system. It is typically a count, such as the number of OOM events, packet drop events, or system stalls.

• Pod count panel

  The number of Pods deployed on the current instance that are in the Running state.

• Node resource overview

  • CPU: The CPU usage of the node.

  • Memory: The memory usage of the node.

  • Disk: The root file system usage of the node.

  • Network: The sum of the upload and download rates for all networks on the node.

Historical health details

• Node anomaly event analysis

  • The Node Anomaly Event Analysis panel displays all anomaly check items affecting the health of the instance or its Pods. You can use the drop-down list to filter for anomaly events of a specific level.

  • If Pods are deployed on the instance, you can use the Pod drop-down list to view anomaly events for Pods on the ECS instance. You can filter by Pod name and namespace.

  • You can use the drop-down list to filter anomaly events by level. The anomaly levels are described below.

    • Notice: Certain monitoring or system behavior metrics have reached a questionable state, which may indicate a potential risk. O&M personnel should closely monitor these metrics to take action before the situation worsens. Common examples include CPU usage approaching a threshold or a rising trend in memory usage.

    • Warning: Monitoring metrics have crossed a configured alert threshold, which may affect the normal operation of services. O&M personnel should promptly review and address the related issues.

    • Critical: A check item shows clear signs of an anomaly or failure, which could directly impact system availability and business continuity. O&M personnel must intervene immediately to investigate and resolve the problem.

  • Diagnosis Status

    • No Diagnosis: No diagnosis has been run for the current anomaly item. This can happen for the following reasons:

      • No diagnosis tool is available for the current anomaly item.

      • The automatic diagnosis feature is in a cooldown period.

      • The anomaly event ended before you viewed the page, which means the anomaly has already resolved.

    • Diagnosing: A diagnosis is in progress for the current anomaly item.

    • Diagnosis Completed: The diagnosis for the current anomaly item is complete. Click View Diagnosis Report in the Actions column to view the report.

    • Diagnosis Failed: The diagnosis call failed. A common reason is that multiple diagnosis requests were initiated on the node simultaneously.

  • Anomaly Feedback

    You can use the feedback mechanism to adjust the anomaly detection algorithm and optimize the sensitivity of an anomaly item. For example, if the system continuously reports node I/O traffic anomalies that you believe are false positives, you can click the Feedback button in the Actions column for one of the anomaly items and adjust the scope and sensitivity.

    • If you choose to apply the change only to the current node, the new sensitivity setting will apply to the node I/O traffic detection for the current instance.

    • If you choose to apply the change to all nodes in the cluster, the new sensitivity setting will apply to the I/O traffic detection for all nodes in the current cluster.

  • OS Copilot intelligent analysis

    The node CPU usage detection, node kernel-mode CPU usage detection, node soft interrupt CPU usage detection, node memory usage detection, node kernel memory usage detection, and node OOM stall prediction and detection check items integrate with OS Copilot. You can click the OS Copilot icon next to an anomaly item to ask the assistant to analyze the root cause. In the Anomaly Event Analysis table, supported check items display an OS Copilot icon. Click the icon to open a chat with predefined prompts. After sending a prompt, OS Copilot analyzes the problem and generates an analysis report in the chat window. The report includes a Problem Overview (instance ID, region, analysis time, and symptoms), Core Metric Analysis (such as a CPU usage distribution table), and Key Findings (such as analysis conclusions on kernel-mode CPU percentage and idle time).

• Top 10 Pod health list

  • If Pods exist on the ECS instance, the Top 10 Pod Health List panel displays information for the 10 Pods with the lowest health scores, sorted from lowest to highest.

  • Details about the Pods are provided.

    • The Pod health score represents the lowest score of the Pod during the time range selected in the time selector.

    • Image shows the container image used by the Pod. A maximum of five container images are displayed. Status shows the current Status of the Pod (Running/Offline).

• Health score trend

  Use the time selector to view the historical trends of the overall health score and the scores for the four metric types. This helps you identify historical health issues in the instance.

  

• Pod health proportion and Pod issue proportion

  • The Pod health proportion chart shows the distribution of all Pods on the current ECS instance across four health levels.

    • Healthy: health score ≥ 90

    • Sub-healthy: 80 ≤ health score < 90

    • Unhealthy: 60 ≤ health score < 80

    • Severely Unhealthy: health score < 60

  • The Pod issue proportion chart shows the distribution of issue types, including saturation, latency, traffic, and errors.