Enable control plane log collection and alerting (new version)

更新时间:
复制 MD 格式

Alibaba Cloud Service Mesh (ASM) provides Log Center, which you can use to view control plane and data plane logs from the ASM console. After you enable log collection, you can use Log Center to view control plane logs, configure log-based alerting rules, view data plane logs, and access related dashboards. This topic describes how to enable control plane log collection and log-based alerting.

Prerequisites

Enable control plane log collection

  1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

  2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Observability Management Center > Log Center.

  3. On the Log Center page, click the Control-Plane Logs tab. Select New project or Use existed project, configure the relevant parameters, and then click Enable Control-Plane Log Collection. In the Submit dialog box, click OK.

    After logs are collected to Simple Log Service (SLS), you can perform the following operations:

Note

If you no longer need this feature, click Disable Control-Plane Log Collection in the upper-right corner of the Control-Plane Logs tab. In the Submit dialog box, click OK.

View control plane logs

After enabling control plane log collection, logs from the managed control plane are collected to Simple Log Service (SLS). You can view or query control plane logs in the SLS console or on the ASM console's integrated log browser.

Note

It may take a few moments for SLS to establish log distribution. You might not see logs on the console immediately after you enable log collection.

Method 1: Use the ASM console

  1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

  2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Observability Management Center > Log Center.

  3. On the Log Center page, click the Control-Plane Logs tab to view log information.

Method 2: Use the SLS console

  1. Log on to the Simple Log Service console.

  2. In the Projects section, click the name of the log service project that corresponds to your target cluster.

  3. In the left-side navigation pane, click the target Logstores. On the page that appears, view the log information.

Configure control plane log alerts

After enabling control plane log collection, you can configure log-based alerting for the control plane.

  1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

  2. On the Mesh Management page, click the name of the target ASM instance. In the left-side navigation pane, choose Observability Management Center > Log Center.

  3. On the Log Center page, click the Control-Plane Logs tab, and then click Alert Setting in the upper-left corner.

  4. In the Control-Plane Alert Setting dialog box, select an Action Policy as needed and click Enable Alert. In the Note dialog box, click Submit.

    An Action Policy defines how to send notifications when an alert is triggered. Two types are supported: ASM Built-in Action Strategy and Custom action strategy. For more information, see action policy. By default, the ASM built-in action policy does not have any notification channels configured. You can click the 设置.png icon and follow the instructions to open the Simple Log Service alert center and configure the channels. If an alert is triggered, refer to Solutions.

Configure alert contacts

You can configure the built-in action policy for the SLS service gateway to set alert notification recipients, notification templates, and other settings.

  1. Log on to the Simple Log Service console.

  2. In the Project List area, click the target project name, and then in the navigation pane on the left, click Alerting.

  3. On the Alert Center page, click Notification Recipient > User Group Management.

  4. On the User Group Management tab, click Modify in the Actions column of the SLS Service Mesh Built-in User Group.

  5. In the Modify User Group dialog box, select the target member in the Available Members area, click the Add icon to add the member to the Added Members area, and then click Confirm.Modify user group

Triggered alert notification example

Note

This topic cannot cover every alert item. The following example shows how to trigger the "Configuration push from the mesh control plane fails" alert by using an incorrect configuration.

  1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

  2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose ASM Gateways > Gateway. On the page that appears, click Create from YAML.

  3. On the Create page, select the target namespace and a scenario template, and then configure the YAML.

    This example uses the default namespace to configure an ASM Gateway. The following is a sample YAML configuration:

    apiVersion: networking.istio.io/v1beta1
    kind: Gateway
    metadata:
      name:  gateway-test
      namespace: default
    spec:
      selector:
        istio: ingressgateway
      servers:
        - hosts:
            - '*console.aliyun.com'
          port:
            name: https
            number: 443
            protocol: HTTPS
          tls:
            credentialName: not-existing-credential
            mode: SIMPLE
  4. View the alert notification.

    1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

    2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Observability Management Center > Log Center.

    3. On the Log Center page, click the Control-Plane Logs tab and search for ACK ERROR to view the alert information.

      If you have configured an email address for your alert contacts, you can also check your email for the alert notification.

Solutions

Warnings for mesh configuration

ASM Mesh Diagnosis has found potentially risky mesh configurations in your cluster. These configurations may cause ASM to produce unexpected results. You can go to the Mesh Diagnosis page to view the alert details and follow the suggestions on the Mesh Diagnosis page to correct the misconfigurations. For more information, see Use ASM Mesh Diagnosis.

Mesh configuration errors

ASM Mesh Diagnosis has found incorrect mesh configurations in your cluster. These configurations pose a high risk of causing unexpected behavior. You should check the alert details on the Mesh Diagnosis page as soon as possible and follow the guidance on the Mesh Diagnosis page to fix the configurations. For more information, see Use ASM Mesh Diagnosis.

Configuration push failure from invalid mesh configuration

The following table lists common error messages for data plane synchronization failures and provides the corresponding solutions. If you cannot find your error message, submit a ticket.

Error message

Solution

Internal:Error adding/updating listener(s) 0.0.0.0_443: Failed to load certificate chain from <inline>, only P-256 ECDSA certificates are supported

This alert indicates that the data plane cluster does not support the certificate configured for the data plane. You must reconfigure the certificate. Only P-256 ECDSA certificates are supported. For more information, see Enable HTTPS on an ASM gateway.

Internal:Error adding/updating listener(s) 0.0.0.0_443: Invalid path: ****

This alert indicates that the certificate path configured for the data plane is incorrect or the certificate does not exist. Verify that the certificate mount path matches the path configured in the gateway. For more information, see Enable HTTPS on an ASM gateway.

Internal:Error adding/updating listener(s) 0.0.0.0_xx: duplicate listener 0.0.0.0_xx found

This alert indicates that a duplicate listener port is configured for the gateway. Check your gateway configuration and remove the duplicate port.

Internal:Error adding/updating listener(s) 192.168.33.189_15021: Didn't find a registered implementation for name: '***'

This alert indicates that the implementation for ***, referenced in the EnvoyFilter patch configuration for listener 15021, cannot be found in the sidecar proxies or ingress gateway. You must remove this reference.

Internal:Error adding/updating listener(s) 0.0.0.0_80: V2 (and AUTO) xDS transport protocol versions are deprecated in grpc_service ***

This alert indicates that the xDS v2 protocol used by your data plane is deprecated. This issue usually occurs when the sidecar proxy version on the data plane is incompatible with the control plane version. To resolve this issue, upgrade the data plane sidecar proxies by deleting their pods. The latest sidecar proxy version is automatically injected when the pods are recreated.

Expired certificate causes gateway failure

This alert indicates that a certificate in Certificate Management has expired and requires immediate action.

Upcoming certificate expiration

This alert indicates that a certificate in Certificate Management is about to expire and should be renewed soon.

Related operations

Modify the control plane log project

  1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

  2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Observability Management Center > Log Center.

  3. On the Log Center page, below the Control-Plane Logs tab, click Change Log Project. In the Change Log Project dialog box, modify the settings as needed and click Submit.