Configure monitoring and alerting for fully managed ECS

How it works

• Managed Service for Prometheus uses its built-in multi-tenancy to apply tenant tags to ECS instances. When you enable ECS tag injection in host monitoring, the agent automatically forwards tenant metrics to the corresponding service instance.

• By configuring the service provider's Remote Write endpoint, you can send a copy of tenant monitoring data to the service provider's account. This allows the service provider to view data from all tenants, while each tenant can only view their own data.

Procedure

Integrate host monitoring with your VPC

• Important

  If you have an existing VPC, the service provider must integrate the required components from the Integration Center on the Application Real-Time Monitoring Service (ARMS) console. This example uses host monitoring. Configure the parameters as described below to ensure that Node-exporter is automatically installed on ECS instances created by Compute Nest in the VPC, and that monitoring metrics are correctly forwarded to the respective tenant accounts.

  • Node-exporter installation mode: Select Automatic Installation.

  • Host service discovery mode: Select Select Tags.

  • ECS instance tag: Set the key to acs:computenest:serviceId and the value to service-xxx.

  • Custom tag injection: tenant_clusterid, tenant_token, tenant_userid, tenant_cloudproductcode. Leave the other settings at their defaults: enable Collect process status metrics, set Node-exporter service port to 9100, set Metric collection interval to 15 seconds, and enable Automatically configure security group. Then, click OK.

• If you are creating a new VPC, you must orchestrate the resources for host monitoring in the ROS template. For more information, see ALIYUN::ARMS::Environment.

    Environment:
      Type: 'ALIYUN::ARMS::Environment'
      Properties:
        ManagedType:
          Ref: ManagedType
        EnvironmentSubType: ECS
        EnvironmentName:
          Ref: EnvironmentName
        Tags:
          Ref: Tags
        FeePackage:
          Ref: FeePackage
        PrometheusInstanceId:
          Ref: PrometheusInstanceId
        ResourceGroupId:
          Ref: ResourceGroupId
        GrafanaWorkspaceId:
          Ref: GrafanaWorkspaceId
        BindResourceId:
          Ref: BindResourceId
        EnvironmentType:
          Ref: EnvironmentType
    EnvironmentFeature:
      Type: 'ALIYUN::ARMS::EnvironmentFeature'
      Properties:
        EnvironmentId:
          Ref: Environment
        FeatureVersion: 1.1.17
        FeatureName: metric-agent

Configure a Grafana dashboard

Each service instance supports only one dashboard. If a service provider has multiple dashboards, you must combine them into a single, consolidated dashboard. Create this dashboard in Managed Service for Grafana and obtain its URL.

Configure the service identifier and dashboard URL

Set the service identifier and monitoring dashboard information:

1. For testing, you can use the shared service identifier cn-mariadb. This identifier is available for testing in the China (Hangzhou) and China (Hong Kong) regions.

   Note

   A dedicated service identifier is required for production use. To obtain one, join the technical support group mentioned at the beginning of this topic for assistance.

2. Dashboard settings include the dashboard title and the Grafana dashboard URL you obtained earlier. These settings are specific to each service identifier, and the title must match your configured dashboard.

   Note

   This step requires manual configuration. Join the technical support group for assistance.

This example shows the configuration for a service using the cn-mariadb service identifier. In the Managed Service for Prometheus section, set the Service identifier to cn-mariadb. In the Dashboard settings section, you can use the default configuration for the cn-mariadb service identifier. The default dashboard title is MySQL Exporter Quickstart and Dashboard.

Configure alerting

Configure provider-side alerting

• Integrating a common component automatically generates default alert rules based on best practices.

  On the Application Real-Time Monitoring Service (ARMS) console, go to Integration Management. On the Integrated Addons page, find the component and click Alerts to view its default alert rules. For example, the Nginx component generates alert rules such as NginxHigh5xxErrors and NginxDown. You can view, edit, or disable these rules on the Alerts tab of the component's integration details page.

• For custom alert rules:

  • On the ARMS console, choose Prometheus Service > Prometheus Alert Rules to create and manage rules. For more information, see Create a Prometheus alert rule.

  • Create alert rules by importing existing Prometheus rules using a PrometheusRule. For more information, see Create an alert rule by using a PrometheusRule. For example, you can create P1-level custom PromQL alert rules for ECS, to monitor metrics such as public egress bandwidth usage (>80%), egress packet drop rate (>0), ingress packet drop rate (>0), and disk usage (>80%), with a 5-minute duration.