Log Monitoring

Prerequisites

• An Alibaba Cloud Elasticsearch instance is created in a region that supports the advanced monitoring and alert service.

  • The advanced monitoring and alert service is available in the following regions: Hangzhou, Beijing, Shanghai, Shenzhen, Qingdao, Zhangjiakou, Chengdu, Guangzhou, Ulanqab, Beijing Finance Cloud, US (Virginia), US (Silicon Valley), Japan (Tokyo), Indonesia (Jakarta), and China (Hong Kong). The regions available in the console are definitive.

  • For more information about how to create an instance, see Create an Alibaba Cloud Elasticsearch instance.

• Learn how to use Grafana monitoring dashboards. For more information, see Grafana dashboards.

Limits

• The advanced monitoring and alert feature provides monitoring and alerts for infrastructure metrics, engine metrics, and log data. Only Elasticsearch 6.7.0 or 7.10.0 with a kernel version later than 1.2.0 supports engine metric monitoring. To upgrade the kernel version, see Upgrade versions.

• You cannot access the advanced monitoring and alert service when you use a RAM role for single sign-on (SSO) to the Alibaba Cloud Management Console. For more information, see Log on to the Alibaba Cloud Management Console using a RAM role for SSO. To access the service, use a Resource Access Management (RAM) user for SSO to the Alibaba Cloud Management Console. For more information, see Log on to the Alibaba Cloud Management Console using a RAM user for SSO.

View default log metrics

1. Log on to the Alibaba Cloud Elasticsearch console.

2. In the left navigation pane, click Advanced Monitoring Alarms.

3. Navigate to Monitoring Visualization > Log Monitoring to view the log monitoring data for all integrated instances.

4. View monitoring data for a specific instance.

   • Method 1: Filter by instance ID to view monitoring data

     1. In the monitoring window, click the blank area to the right of the Log Monitoring Dashboard and press the Esc key to open the Grafana menu and the filter bar.

     2. In the filter bar, select instanceId, regionId, and level to view monitoring data for the instance, such as the Redis Slow Log, slow index logs, access logs, and cluster logs.

   • Method 2: Navigate from the instance list

     1. On the Elasticsearch Instances page, click the ID of the target instance.

     2. In the navigation pane on the left, select Monitoring and Logging > Cluster Monitoring.

     3. Click the Infrastructure Monitoring and Advanced Monitoring and Alerts tabs to view the monitoring data for the current instance.

   Note

   • The Grafana dashboards in the Advanced Monitoring and Alerting service function in the same way as open source Grafana dashboards. For more information, see the Grafana documentation.

   • The default monitoring dashboards provided by the Advanced Monitoring and Alerting service cannot be modified. To create a dashboard that better suits your business needs, you can configure a custom monitoring dashboard.

Query logs

Log Monitoring supports filtered queries for slow query logs, slow index logs, access logs, and main logs. Different log types are retrieved from different data sources, as shown in the following table.

The procedure for querying different types of logs is the same. The following steps use a slow query log as an example.

1. Log on to the Alibaba Cloud Elasticsearch console.

2. In the navigation pane on the left, click Advanced Monitoring Alarms.

3. Select Monitoring Visualization > Log Monitoring.

4. On the Log Monitoring page, click the Slow Query Log tab.

5. Specify the Query.

   For example, to retrieve slow query logs from an instance whose ID starts with `es-cn-n6w24ma4`, for the `geonames` index, with content that includes `aggregations`, and a query time of 240 ms or less, use the following query statement:

   instanceId:es-cn-n6w24ma4* AND index_name: geonames AND content: aggregations AND search_time_ms: <=240

   Note

   • A Query can include operators such as >, <, =, >=, <=, AND, and OR. For more information, see Query string.

   • Different log types use different data sources, and the available query fields vary by data source. For example, on the slow query log page, es-slow-search is the slow query data source, and search_time_ms is a field that represents the query time. For more information, see Built-in fields for log queries.

   • On the Log Monitoring page, the only available feature is Query, which can only be used to retrieve log data. Other search settings cannot be modified. For example, you cannot change the search scope from the default value of logs to sum or another value.

6. View the logs that match the query.

Built-in fields for log queries

• Independent field

  Type	Independent field	Note
  Slow query	search_time_ms	Query duration.
  	search_total_hits	The number of documents that the query hit.
  	search_type	Query type.
  	shard_id	The ID of the shard that executed the query.
  	slow_search_log_type	Slow log type.
  	total_shards	Total number of shards.
  	Content	The query body.
  Slow write	index_time_ms	Write duration.
  	content	The query body.
  Main log	Content	The query body.
  Access log	node	The Elasticsearch node that generates the access log.
  	query	The query body to execute. For filtering, use the source field instead of the `query` field.
  	remote	The IP address of the remote server.
  	bodySize	Request size. Unit: bytes.
  	uri	Access path.

  Note

  The unique fields table lists only the fields that are specific to each log type. For fields that are common to all log types, see the following common fields table.

• Common fields

  By field	Note
  es_available_zone	The zone of the instance.
  es_region	The region where the instance is located.
  es_resourceUid	The UID of the instance.
  es_resource_group	The resource group that contains the instance.
  host	The IP address of the node.
  instanceId	Instance ID.
  Level	The log level, including trace, debug, info, warn, or error.

  Note

  Fields that start with _ are part of the Elasticsearch metadata.