Write monitoring expressions

更新时间:
复制 MD 格式

In the monitoring settings of Node.js Performance Platform, you need to write two types of expressions: threshold expressions and alert expressions.

The monitoring system determines whether to generate an alert based on the threshold expression that you specify. When the monitoring system sends an alert notification, the monitoring system uses the alert expression to describe the details of the alert.

Threshold expressions

A threshold expression is a domain-specific language defined by Node.js Performance Platform to determine whether an alert needs to be generated. For the sake of flexibility and security, only a limited number of expressions are defined.

When you develop an application, you can enter a threshold expression, which will be compiled into an equivalent JavaScript function. When the monitoring system receives the monitoring data of a specific type, the monitoring system uses the monitoring data as the context and substitutes it into the threshold expression. If the result of the expression is true, the monitoring system determines that an alert notification needs to be sent.

A basic expression consists of three parts: an attribute, a comparison operator, and a literal.

Attributes

An attribute identifier is used to represent the attribute of an object in a specific context. An attribute identifier starts with an at sign (@) and follows the @<Attribute name> format. Examples: @load1, @cpu, and @freemem.

Comparison operators

The comparison operators are the same as in common programming languages: ==, >=, >, <=, <, and !=.

Operators

The following operators are supported: +,-, *, /, and %. Example: @heap_used / @heap_limit.

Literals

A literal can be a string or numeric value. For example, a basic expression can be @load1 > 5.

include

The include keyword is used to include a string in an attribute value. Example: @message include "TypeError".

Others

Operators such as &&, ||, and () are also supported. These operators can be used the same way as in common programming languages. Example:

@cpu > 0.10 || @load1 > 5
@cpu > 0.10 && @load1 > 5
@cpu > 0.10 && (@load1 > 5 && @load5 > 5)

Alert expressions

Alert expressions are similar to threshold expressions but are not used for evaluation. They function more like a template language. The following is a simple example:

I am ${@name}. I am ${@age} years old.

The format consists of two main parts: strings outside ${} and expressions inside ${}. Assume the following context:

{name: "Jackson Tian", age: 18}

The final result is:

I am Jackson Tian. I am 18 years old.

If the context of an alert is:

{load1: 5, load5: 2, load15: 2.2}

The alert expression is:

Load1 is high, above 3. Current value: ${load1}. Please investigate.

The execution result of the alert expression is: The average load within the last minute is 5, which is higher than 3. Troubleshoot the issue.

You can specify the @xxx attribute, operators such as +, -, *, %, and /, numeric values, and strings in ${}.

For example, the memory usage percentage is calculated as follows:

Memory usage is too high: ${@freemem / @total * 100}%.

Monitoring types and attributes

Each monitoring type applies to a different context.

system

  • @load1: the average load within the last minute.

  • @load5: the average load within the last 5 minutes.

  • @load15: the average load within the last 15 minutes.

  • @cpu: the CPU utilization. Valid values: 0 to 1.

  • @uptime: the uptime of the operating system. Unit: seconds.

  • @freemem: the free memory. Unit: bytes.

  • @totalmem: the total memory. Unit: bytes.

  • @cpu_count: the number of CPU cores.

disk_usage

  • @used_percent: the percentage of used disk space. Valid values: 0 to 100.

node_log

The data of the node_log type is the kernel data of a process.

Process ID.:

  • @pid: the process ID.

Memory:

  • @rss: the total size of memory used by the process. Unit: bytes.

V8 heap data:

  • @heap_limit: the upper limit of the heap memory that can be allocated. Unit: bytes.

  • @heap_total: the size of the heap memory that is allocated. Unit: bytes.

  • @heap_used: the size of the heap memory that is used. Unit: bytes.

Details of spaces in the V8 heap:

Note: The heap memeory of V8 is divided into multiple spaces.

  • @new_space_used: the size of memory used in the new generation space. Unit: bytes.

  • @old_space_used: the size of memory used in the old generation space. Unit: bytes.

  • @map_space_used: the size of memory used in the map space. Unit: bytes.

  • @code_space_used: the size of memory used in the code space. Unit: bytes.

  • @lo_space_used: the size of memory used in the large object space. Unit: bytes.

For versions earlier than Node.js v4.0, the following four spaces are available:

  • @cell_space_used: the size of memory used in the old generation cell space. Unit: bytes.

  • @property_cell_space_used: the size of memory used in the old generation property cell space. Unit: bytes.

  • @old_data_space_used: the size of memory used in the old generation data space. Unit: bytes.

  • @old_pointer_space_used: the size of memory used in the old generation pointer space. Unit: bytes.

The preceding four spaces are merged into the old generation space (old_space) in Node.js v4.0 and later.

For example, to determine whether a memory leak has occurred, you can configure the following expression: @heap_used / @heap_limit > 0.8. The alert expression can be Oops, a memory leak has occurred. The upper limit of the heap memory is ${@heap_limit} bytes. ${@heap_used} bytes have been used.

Garbage collection(GC) data:

  • @gc_time_during_last_min: the amount of time spent in GC within the last minute. Unit: ms.

  • @total: the total amount of time spent in GC. Unit: ms.

libuv handle data:

  • @active_handles: the number of active handles.

Timer data:

  • @total_timer: the number of timers.

HTTP data:

  • @live_http_request: The number of HTTP requests that are being processed.

  • @http_response_sent: The number of HTTP responses sent within the last minute.

  • @http_request_handled: The number of HTTP requests handled within the last minute.

CPU data:

  • The current CPU usage of the Node process.

  • @cpu_15: the average number of CPU cores that are occupied by the Node.js process within the last 15 seconds.

  • @cpu_30: the average number of CPU cores that are occupied by the Node.js process within the last 30 seconds.

  • @cpu_60: the average number of CPU cores that are occupied by the Node.js process within the last 60 seconds.

Note: The values of the preceding four fields are measured in CPU cores. For example, 0.25 indicates that 0.25 CPU cores are occupied. The value divided by the total number of CPU cores is the percentage of occupied CPU cores.

error_log

The exception log.

  • @type: the type of the exception.

  • @timestamp: the timestamp that is generated when the exception occurs. It is the number of milliseconds that have elapsed since 00:00:00 Thursday, January 1, 1970.

  • @stack: the error stack.

  • @extra: the extra data.

For example, to check whether an uncaughtException exception has occurred, you can run the following Node.js code:

process.on('uncaughtException', function(err) {
  err.name = "UncaughtExceptionError";
  console.log('Caught exception: ' + err);
});

Then, configure the following expression: @type include 'UncaughtExceptionError'. The alert expression is Oops, an UncaughtExceptionError exception has occurred. The stack is ${@stack}.

processes_count

The number of processes.

  • @node_count: the number of Node.js processes.

If the number of processes is 5 in most cases, you can configure the following expression to generate an alert when the number of processes becomes less than 5: @node_count < 5. The alert expression can be The number of Node.js processes is ${@node_count}, which is less than 5. Please check.

version

The version number.

  • @node: the Node.js version number of the system. Format: vx.x.x.

Conclusion

If you have questions about expressions and attributes, you can join the DingTalk group 11794270 to contact us.