OOM diagnosis
The out-of-memory (OOM) diagnosis feature helps you identify the causes of OOM events. This topic describes how to use this feature.
Limitations
Regional limitations
This feature is currently available only in the Chinese mainland and China (Hong Kong).
Operating system limitations
Architecture
Operating system
x86 architecture
Rocky Linux 9.5
Rocky Linux 9.1
Ubuntu 20.04
Alibaba Cloud Linux 3 Container-Optimized Edition
Rocky Linux 8.8
Ubuntu 22.04
Alibaba Cloud Linux 3 Pro
Alibaba Cloud Linux 2/3
CentOS 7.6 or later, or CentOS 8
Anolis OS 7/8
Ubuntu 24.04
ARM architecture
Alibaba Cloud Linux 3 Pro
Alibaba Cloud Linux 3
Use cases
Scenario | Description |
Insufficient system-wide memory | Excessive memory usage on the host exhausts system memory, triggering an OOM event. |
cgroup memory limit exceeded | An OOM event occurs when a process within a cgroup exceeds its configured memory limit, causing the cgroup's total memory usage to reach its maximum. |
Parent cgroup memory limit exceeded | When the total memory usage of a parent cgroup reaches its limit, the system may trigger an OOM event and terminate a process in a child cgroup. |
Insufficient memory on a memory node | In a NUMA architecture, an operating system has multiple memory nodes. You can run the |
cgroup memory limit exceeded due to excessive shared memory usage | In a cgroup OOM scenario, the diagnosis may find that shared memory consumption exceeds 30% of the cgroup's total user-space memory. In this case, the root cause of the OOM event is likely excessive shared memory usage. Further analysis is required to identify the main consumers. |
Prerequisites
If you are a RAM user, ensure that the primary Alibaba Cloud account has granted the
AliyunECSReadOnlyAccessandAliyunSysomFullAccesssystem policies to the RAM user.The Operating System Console is enabled.
When you log on to the Operating System Console for the first time, click Enable Service to enable the service.
Procedure
Go to the Operating System Console.
In the left-side navigation pane, click System Diagnosis.
In the upper-left corner of the page, select the region where your target instance is located.
From the Diagnosis Type list, select Memory Diagnosis. From the Diagnosis Item list, select OOM Diagnosis. Configure the parameters and click Run Diagnosis.
NoteIf SysOM is not installed, clicking Run Diagnosis automatically installs it on the target ECS instance and then starts the diagnosis.
The following table describes the parameters.
Parameter
Description
Required
Instance ID
The ID of the target ECS instance.
Yes
OOM time
Select time: Analyze the OOM event that occurred closest to the specified time.
Do not specify time: Analyze the last OOM event that occurred on the system.
No
In the Diagnosis records section, click View Report.
Diagnosis report
Basic information
This section contains basic information about the diagnosis, including the instance ID, diagnosis item, diagnosis report ID, and the diagnosis initiation time.
Diagnostic conclusion
This section provides the diagnostic conclusion and a detailed description of the root cause of the OOM event.
Diagnostic suggestions
This section provides recommendations based on the diagnosis to help you prevent similar OOM events.
Diagnosis details
OOM diagnosis details
This section shows the task that triggered the OOM event, the task's cgroup, the cgroup that triggered the OOM, and the OOM type.
(Host OOM) Memory usage analysis
When an OOM event is caused by insufficient host memory, the diagnosis report shows the host's memory usage at the time of the event. This includes the proportions of system, application, and free memory, and a breakdown of system and application memory usage.
(cgroup OOM) Shared memory file usage
When an OOM event is caused by excessive shared memory usage within a cgroup, the diagnosis report provides a table of shared memory file usage sorted by size.
Task list at the time of OOM
This section lists the tasks running on the system at the time of the OOM event, including the task name, process ID, and user-space memory usage (RSS).
(Unreclaimable Slab) Unreclaimable Slab list
When an OOM event is caused by excessive unreclaimable Slab memory usage, and the kernel version is 5.10 or later (for example, Alibaba Cloud Linux 3), the diagnosis report provides a breakdown of unreclaimable Slab usage.