Process monitoring

Troubleshoot scheduled CPU alerts

When a host triggers a CPU alert during a fixed time period, use process monitoring to retrospectively review process CPU consumption during that historical time period and identify the process that is consuming CPU:

1. Log on to the Cloud Monitor console. In the left-side navigation pane, choose Cloud Resource Monitoring > Host Monitoring. In the instance list, click the name of the target ECS instance to go to the host monitoring details page.

2. View the CPU utilization trend chart to confirm the CPU changes during the alert period.

3. Switch to the Process Monitoring tab. In the time range selector, select the time range that corresponds to the alert period (1 Hour/6 Hours/12 Hours/1 Day/3 Days/7 Days/14 Days/Custom). If the alert occurred during an unattended period, select Custom, specify the start and end time of the abnormal period (for example, 01:00 to 02:00), and then click OK. In the Process TopN table, view the top five processes by CPU utilization during that period. The table shows the process name/CMD, process ID, user, CPU utilization (%), memory usage (%), open file count, and timestamp.

4. Identify the cause based on the process name:

   • ids.exe / hbrclient: Cloud Backup (HBR) client processes that handle data transfer and task scheduling. Log on to the Cloud Backup console to check whether the backup plan execution time matches the alert time.

   • crontab / Windows Task Scheduler: Check the crontab -l output on Linux instances or Task Scheduler on Windows instances to see whether a scheduled script is consuming CPU.

   • AliYunDun / AliYunDunMonitor: Security Center processes. A brief CPU increase during scheduled security scans is expected behavior.

Use the process name to determine whether the process is an expected business process. If no abnormal process appears in the top five, check scheduled tasks on the instance (run crontab -l on Linux, or open Task Scheduler on Windows) and system logs (Event Viewer on Windows). For the note about kernel threads that do not rank in the top five, see How process metrics are collected.

Add process monitoring

Monitor the instance count of critical processes to track their operational status.

1. Log on to the Cloud Monitor console.

2. In the left-side navigation pane, choose Cloud Resource Monitoring > Host Monitoring.

3. On the Host Monitoring page, click the instance name of the target host or click Monitoring Charts in the Actions column. Then, select the Process Monitoring tab.

4. At the bottom of the page, expand the Process Count Monitoring panel and click Add Process Monitor in the upper-right corner.

5. In the Add Process Monitoring panel, enter the process name, click Add, and then click the icon in the upper-right corner.

   Note

   It may take a few minutes for monitoring data to appear for the newly added process.

Create an alert rule for a process

After adding process monitoring, create an alert rule to receive notifications when the process count changes.

Recommendation: setting the alert threshold for total process count

The total process count on a Linux host is limited by the kernel parameter /proc/sys/kernel/pid_max. To avoid false positives or missed alerts, follow these guidelines:

• Set the alert threshold to 70%–80% of the pid_max value to maintain a safety buffer.

• Run cat /proc/sys/kernel/pid_max to check the current system limit.

• Alternatively, obtain the actual process count during a stable business period from Cloud Monitor historical data or by running ps -eLf | wc -l, then add 20%–30% as the alert threshold.

Avoid setting the threshold to a value close to pid_max.

1. Log on to the Cloud Monitor console.

2. In the left-side navigation pane, choose Cloud Resource Monitoring > Host Monitoring.

3. On the Host Monitoring page, click the instance name of the target host or click Monitoring Charts in the Actions column. Then, select the Process Monitoring tab.

4. Expand the Process Count Monitoring panel. In the target process chart, click the alert icon in the upper-right corner to open the alert rule panel.

5. In the Configure Rule Description panel, set a rule name, define the threshold and alert level for the Process / Number of Current Processes metric, and then click OK.

6. In the Create Alert Rule panel, configure the alert rule parameters and click Confirm.

   Alert rule parameters are described in Create an alert rule.

7. View the process alert rule.

   1. In the left-side navigation pane, choose Alerts > Alert Rules.

   2. On the Alert Rules page, you can view the alert rules for the process dimension in the Alert Rule column.

Delete process monitoring

1. Log on to the Cloud Monitor console.

2. In the left-side navigation pane, choose Cloud Resource Monitoring > Host Monitoring.

3. On the Host Monitoring page, click the instance name of the target host or click Monitoring Charts in the Actions column. Then, select the Process Monitoring tab.

4. Expand the Process Count Monitoring panel and click Add Process Monitor in the upper-right corner. This panel lists all currently monitored processes.

5. In the Add Process Monitoring panel, click Delete in the Actions column for the target process.

6. In the Delete confirmation dialog box, click OK.

7. Click the icon in the upper-right corner.

FAQ

Does Cloud Monitor support viewing the network bandwidth or traffic of a single process (such as Nginx or Java)?

Cloud Monitor does not support viewing outbound bandwidth or downlink traffic of individual processes over a specified time period. To monitor per-process network traffic, use a third-party tool such as nethogs on the ECS instance.

How do I monitor the Java thread count?

Cloud Monitor process monitoring tracks only process counts. You can add java as a process keyword to track running Java processes. Java thread count monitoring is not supported. To monitor Java thread count, use ARMS Application Monitoring.

Troubleshoot processes with high CPU usage in a historical time range

After you receive a CPU alert, use process monitoring in Cloud Monitor to review the CPU utilization of processes in a historical time range and identify the processes that consumed CPU during the abnormal period.

1. Log on to the Cloud Monitor console.

2. In the left-side navigation pane, choose Cloud Resource Monitoring > Host Monitoring.

3. In the instance list on the Host Monitoring page, click the name of the target ECS instance to go to the host monitoring details page.

4. Click the Process Monitoring tab.

5. In the time range selector in the upper-right corner, select Custom, specify the abnormal period, such as 01:00 to 02:00, and then click OK.

6. In the Process TopN table, view the top five processes by CPU utilization, including the process name, process ID, CPU utilization, and memory usage.

Use the process name to determine whether the process that consumes CPU is an expected business process. For further troubleshooting, check the scheduled tasks and the Event Viewer logs of the operating system.