The Alert Center dashboards—the Global Alert Pipeline Center, the Global Alert Rule Center, and the Global Alert Troubleshooting Center—visualize alert-related information.
Background information
When you first use the alerting feature of Simple Log Service, the system prompts you to select a region. It then automatically creates a Project named sls-alert-<Alibaba Cloud account ID>-<region ID> and a Logstore named internal-alert-center-log to store alert logs.
The internal-alert-center-log Logstore is free to use and includes a default index.
Based on the internal-alert-center-log Logstore, Simple Log Service generates built-in dashboards that provide statistics on alert triggers and notifications.
Entry point
Log on to the Simple Log Service console.
In the Projects section, click a project.
In the left-side navigation pane, click Alerts.
On the Alert Center page, click the tab.
Global Alert Pipeline Center
The Global Alert Pipeline Center dashboard displays the historical information about the alerts that are triggered within a specified period of time for the current Alibaba Cloud account. The historical information includes how alerts are triggered, how alerts are denoised, and how alert notifications are sent. The dashboard also displays all historical data. The data includes the number of enabled alert rules, the number of alerts by severity, the number of merged alerts, the number of deduplicated alerts, the number of silenced alerts, the number of alert notifications that are sent, and the number of alert notifications for each notification method.
At the top of the dashboard, you can filter by Business, region, Project, severity, and rule name. Supported channels include voice call, email, Webhook, SMS, DingTalk, and Notification Center.
Global Alert Rule Center
The Global Alert Rule Center dashboard displays all alert rules within a specified period of time for the current Alibaba Cloud account. The dashboard also displays the number of enabled rules, the distribution of rules by status, and the details of rules.
At the top of this dashboard, you can filter by Business, region, Project, and rule name. The overview section shows the current number of evaluations and their 24-hour trend. A horizontal bar chart displays the status distribution: Not Triggered, Error, and Triggered. The main area contains the Latest Evaluation Status of Alert Rules table (with columns for time, rule name, status, details, and actions) and the Latest Status of Alerts in Each Merged Collection table (with columns for stage, merged collection, severity, and alert tags). You can further filter the results by severity and pipeline stage.
Global Alert Troubleshooting Center
The Global Alert Troubleshooting Center dashboard displays information about alert configuration errors that occur within a specified period of time for the current Alibaba Cloud account. The information includes the number and details of global configuration errors, the number and details of configuration errors for each notification method, and the number and details of configuration errors for alert rules.
At the top of the dashboard, you can filter by Business, region, Project, and rule name. Each section uses statistical cards to show the total number of errors and their period-over-period trends. The global configuration failures section includes a bar chart for failure level distribution and a pie chart for failure type distribution. The notification channel errors section displays failure statistics for channels like Webhook and DingTalk.
You can find logs related to configuration errors in the internal-alert-center-log Logstore by running the query __topic__: system_config. The following table lists common errors.
Level | Type | Details | Description | Solution |
error | AlertPolicyNotConfigured | Alert xxx has no alert policy configured | No alert policy is configured for the alert rule and alerts cannot be merged based on routes. Therefore, Simple Log Service ignores the related alerts. | Modify the alert rule and configure a valid alert policy. |
AlertPolicyNotExist | alert policy xxx does not exist | The alert policy that is associated with the alert rule does not exist. Therefore, Simple Log Service ignores the related alerts. | Create an alert policy with the same name or specify another alert policy. | |
AlertPolicyInvalid | alert policy xxx format is error | The format of the alert policy is invalid and related alerts cannot be parsed. Therefore, Simple Log Service ignores the related alerts. | Modify the alert policy, or delete the alert policy and create an alert policy with the same name. | |
GroupPolicyEmpty | Group policy of alert policy xxx is empty | The route consolidation policy of the alert policy is not configured. Therefore, Simple Log Service ignores the related alerts. | Configure the route consolidation policy of the alert policy and make sure that the route consolidation policy is not empty. | |
ActionPolicyNotExist | action policy xxx does not exist | The action policy does not exist. Therefore, Simple Log Service does not send alert notifications for the alerts that use the action policy. | Create an action policy or specify another action policy. | |
ActionPolicyInvalid | action policy xxx is invalid | The format of the action policy is invalid. Therefore, Simple Log Service does not send alert notifications for the alerts that use the action policy. | Modify the action policy, or delete the action policy and create an action policy with the same name. | |
ActionPolicyEmpty | Primary(Secondary) policy for action policy xxx is empty | The primary and secondary action policies of the action policy are empty. Therefore, Simple Log Service does not send alert notifications for the alerts that use the action policy. | Modify the action policy and make sure that the primary and secondary action policies are not empty. | |
warn | AlertPolicyInheritanceLoop | alert policy xxx has inheritance loop with yyy | An inheritance loop exists. For example, Alert Policy A inherits Alert Policy B and Alert Policy B inherits Alert Policy A.
The inheritance loop is not parsed. | Modify the alert policy to prevent inheritance loops. |
UserNotExist | user xxx does not exist | The specified user does not exist. Therefore, Simple Log Service does not send alert notifications to the user. | Create a user or configure the action policy to specify another user. | |
UserGroupNotExist | group xxx does not exist | The specified user group does not exist. Therefore, Simple Log Service does not send alert notifications to the user group. | Create a user group or configure the action policy to specify another user group. | |
UserGroupEmpty | group xxx is empty | The specified user group does not contain users. Therefore, Simple Log Service does not send alert notifications to the user group. | Add users to the user group. | |
OncallGroupNotExist | oncall group xxx does not exist | The specified on-duty group does not exist. Therefore, Simple Log Service does not send alert notifications to the on-duty group. | Create an on-duty group or configure the action policy to specify another on-duty group. | |
ContentTemplateNotExist | content template xxx does not exist | The specified alert template does not exist. Therefore, Simple Log Service ignores the notification methods that use the alert template. | Create an alert template or configure the action policy to specify another alert template. | |
CalendarNotExist | calendar xxx does not exist | The specified calendar does not exist. Therefore, Simple Log Service ignores the on-duty group that uses the calendar. | Create a calendar or configure the on-duty group to specify another calendar. | |
ChannelQuotaNotExist | channel quota does not exist | The quota for the specified notification method does not exist. Therefore, the number of alert notifications that can be sent by using the notification method is unlimited. | Configure a quota for the specified notification method. | |
ChannelQuotaInvalid | channel quota format is invalid | The quota for the specified notification method is left empty. Therefore, the number of alert notifications that can be sent by using the notification method is unlimited. | Change the quota for the notification method and make sure that the quota is not empty. |
Example
The following video introduces the alert troubleshooting process and the asset information required for it.