DocsICP FilingConsole
Documentation
Home Page Platform For AI EAS Model Serving Logging, monitoring, and alerting Configure log collection

Configure Log Service for a resource group

Updated at:
Copy as MD
Product
My Favorites

This topic describes three methods for collecting EAS service logs into Simple Log Service (SLS) and provides step-by-step instructions for each method. Choose the method that best fits your use case.

Overview

EAS supports three methods for collecting service logs into SLS. The following table compares the methods to help you choose.

Criterion

EAS Insights tab

CMS 2.0 Onboarding Center

SLS Logtail manual configuration

Use case

Quick onboarding of logs for a single service

Batch onboarding of logs for multiple services

Advanced scenarios such as file log collection and custom parsing

Log type

stdout logs only

stdout logs only

stdout logs + file logs

Complexity

Lowest — enable with one click on the EAS service details page

Medium — configure in the CMS 2.0 console

Highest — manually create machine groups and configure Logtail

Scenario 1: Quick onboarding for a single service (EAS Insights tab)

To quickly onboard stdout logs for a single EAS service, go to the Insights tab on the service details page and enable the feature. Logs are then collected into SLS automatically.

  1. Log on to the PAI console and select the target region at the top of the page.

  2. In the left-side navigation pane, choose Elastic Algorithm Service (EAS). If you have multiple workspaces, select the target workspace.

  3. On the Inference Service tab, click the name of the target service to go to the service details page.

  4. On the service details page, click the Insights tab.

  5. The steps vary depending on whether the region has been onboarded:

    • First time in this region: In Step 2 Initialize Resources, click Enable Insights. Resource initialization and data onboarding are completed automatically.

    • Region already onboarded: The current service instance is onboarded automatically. No further action is required.

Note

Services onboarded through the Insights tab use the single instance mode in CMS 2.0 Integration Management. This mode collects logs only for the current service. To onboard multiple services at a time, see Scenario 2.

Scenario 2: Batch onboarding for multiple services (CMS 2.0 Integration Center)

To onboard stdout logs of multiple EAS services in a region at a time, use the Onboarding Center of CMS 2.0.

  1. Log on to the CMS 2.0 console. Select a workspace in the same region as the EAS service, and then click Integration Center in the left-side navigation pane.

    If no workspace exists in the same region, click the back arrow next to CMS 2.0 in the left-side navigation pane to go to workspace management, and then create a workspace.

  2. On the Onboarding Center page, enter Platform for AI (PAI-EAS) in the search box and click the search result card.

  3. In the Start Onboarding panel, configure the following settings:

    • Account: Select the current account or another account.

    • Policy Name: Select an existing policy or click Create to create a new policy.

    • Region: Read-only. Defaults to the workspace region.

    • Entitys: Select one of the following options: All Entities, By Resource Group, By Tags, or Manual Selection.

  4. Click Determine. In the Addon Config section, configure the following settings:

    • Name: Optional. A name is generated automatically. We recommend that you enter a custom name to distinguish between onboarding configurations.

    • Select the data to be accessed: Select PAI-EAS Service Logs.

  5. Click Determine. Switch to the Integration Management tab to view the onboarding configuration. In the Actions column, click the data source to view the corresponding SLS Logstore information.

Scenario 3: Custom log collection (SLS Logtail manual configuration)

To collect file logs, ship logs to a specific Project and Logstore, or perform advanced operations such as custom log parsing or shipping logs to OSS, manually configure Logtail in SLS.

The procedure varies by resource type. Select the section below that matches your resource type:

  • Public resource group: You must manually create a machine group by using a custom ID and manually configure Logtail.

  • Dedicated resource group: After you configure SLS, the system automatically creates the machine group and Logtail. If you need to collect custom logs, configure them manually.

  • Resource quota: You must manually create a machine group by using a custom ID that differs from the one used for public resource groups, and manually configure Logtail.

Prerequisites

Simple Log Service (SLS) is activated, and a Project and a Logstore are created. For more information, see Manage projects and Create a basic Logstore.

Public resource group

Step 1: Create a machine group

For services deployed in a public resource group, you must manually create a machine group.

  1. Log on to the SLS console. In the Project list, click the target Project.

  2. In the left-side navigation pane, chooseResourcesMachine Groups.

  3. Next to Machine Groups, click the 机器组Create Machine Group.

  4. On the Create Machine Group page, create a machine group by using a custom ID:

    • Machine Group Identifier: Select Custom Identifier.

    • Machine Group Topic: Leave this field empty.

    • Name and Custom Identifier: Enter eas-log-group-{region_id} for both fields. Replace {region_id} with the region ID where the service is deployed. For example, if the service is deployed in the Zhangjiakou region, enter eas-log-group-cn-zhangjiakou.

  5. After you deploy a service in the public resource group, check the heartbeat status of instances on the Machine Groups page. OK indicates that the machine group is working properly.

    Note

    If no service is deployed, the instance list is empty.

Step 2: Configure Logtail

Configure Logtail based on the type of logs you want to collect. The following tabs describe how to collect container stdout logs and container file logs. For more complex configurations, see Create and configure a log collection rule.

Collect container stdout logs

  1. Log on to the SLS console.

  2. In the Quick Data Import section on the right, click Import Data, then select K8S - Stdout and Stderr - Old Version.

  3. Select the target Project and Logstore, and then click Next.

  4. Click Use Existing Machine Groups, select the machine group that you created for the public resource group in Step 1, and then click Next.

  5. (Optional) If you deployed the service with a custom image and want to exclude EAS engine logs, turn on Container Filtering in the Input Configurations section. Then add a container label blacklist: set label name to io.kubernetes.container.name and label value to easworker.

  6. Click Next until the configuration is complete.

Collect container file logs

  1. In the Quick Data Import section on the right, click Import Data, then select Kubernetes - File.

  2. Select the target Project and Logstore, and then click Next.

  3. Select the machine group that you created for the public resource group in Step 1, and then click Next.

  4. In the Input Configurations section, configure the File Path (the path of the files to collect), and then click Next.

  5. Click Next until the configuration is complete.

Resource quota

Step 1: Create a machine group

The procedure is the same as for a public resource group, except that the custom ID is different.

Important

The custom ID for a resource quota is eas-log-group-{region_id}-for-resource-quota. Replace {region_id} with the region ID where the service is deployed. For example, if the service is deployed in the Zhangjiakou region, enter eas-log-group-cn-zhangjiakou-for-resource-quota.

Step 2: Configure Logtail

Configure Logtail based on the type of logs you want to collect. The following tabs describe how to collect container stdout logs and container file logs. For more complex configurations, see Continuously collect text logs from hosts.

Collect container stdout logs

  1. Log on to the SLS console.

  2. In the Quick Data Import section on the right, click Import Data, then select Single Line - Text Logs.

    Note

    You can change the import method based on your requirements. For example, to match collected logs by using regular expressions, select Regex - Text Logs. To parse logs in JSON format, select JSON - Text Logs. You can also modify the configuration after it is created.

  3. Select the target Project and Logstore, and then click Next.

  4. Configure the machine group settings as follows, and then click Next:

    • Scenario: Select Servers.

    • Installation Environment: Select ECS.

    • Machine Group: Select the machine group that you created for the resource quota in Step 1.

  5. In the Logtail configuration, configure the following settings:

    • Set File Path to /var/log/containers-stdout/{container_name}/*.log. Replace {container_name} with the container name. For example, to collect logs from all custom containers, enter /var/log/containers-stdout/worker*/*.log.

  6. Click Next until the configuration is complete.

Collect container file logs

  1. When you deploy the service, specify the log directories in the service configuration JSON:

    {
  "cloud": {
    "logging": {
      "paths": [
        "/home/admin/logs",
        "/home/workspace/logs",
        "/var/log"
      ]
    },
    // Other configurations omitted
    "networking": {
      "security_group_id": "sg-xxxxx",
      "vswitch_id": "vsw-xxxxx",
      "vpc_id": "vpc-xxxxx",
    }
  }
}

  2. In the Quick Data Import section on the right, click Import Data, then select Single Line - Text Logs.

  3. Configure the machine group settings as follows, and then click Next:

    • Scenario: Select Servers.

    • Installation Environment: Select ECS.

    • Machine Group: Select the machine group that you created for the resource quota in Step 1.

  4. In the Logtail configuration, configure the following settings:

    • Type: Select Text Log Collection.

    • File Path: Enter the path of the log files to collect. The path must be one of the paths declared in the service configuration JSON. Example: /home/admin/logs/*.log.

    • Maximum Directory Monitoring Depth: Set this parameter to 0.

  5. Click Next until the configuration is complete.

Dedicated resource group

Step 1: Create a machine group

For dedicated resource groups, you do not need to create a machine group manually. After you configure SLS in the PAI console, the following resources are created automatically:

  • A machine group is created in SLS with the name format eas-sls-{resource-id}-mg (resource-id is the resource group ID).

  • A Logtail configuration is created in the specified Logstore to collect EAS framework logs.

Perform the following steps:

  1. Log on to the PAI console and select the target region at the top of the page.

  2. In the left-side navigation pane, choose AI Computing Resources > Resource Pool.

  3. On the EAS Inference Resources > Dedicated Resource Group tab, click the name of the resource group to go to the dedicated resource group details page.

  4. On the dedicated resource group details page, click Configure SLS.

  5. In the Configure Log Service Settings dialog box, select the SLS Project and Logstore, and then click OK.

Step 2: Configure Logtail

After you configure SLS for a dedicated resource group, a Logtail configuration is automatically created to collect EAS framework logs.

To collect custom logs, see Public resource group - Configure Logtail. When you reach the machine group selection step, select the machine group that was automatically created for the dedicated resource group in Step 1.

FAQ

Q: How do I disable SLS log collection for a PAI-EAS service?

To disable log collection, use the method that matches how you onboarded:

  • Onboarded through the Insights tab or CMS 2.0 Onboarding Center (Scenario 1 or Scenario 2): Log on to the CMS 2.0 console. Find the onboarding configuration in Onboarding Management and delete it.

  • Dedicated resource group (Scenario 3): Go to the dedicated resource group details page in the PAI console, click Configure SLS, and then select Disable SLS from the drop-down list.

  • Public resource group or resource quota (Scenario 3): Log on to the SLS console and delete the corresponding Logtail configuration.

Q: What do I do if log collection is incomplete or does not match traffic after I configure SLS for a dedicated resource group?

If the number of collected logs is noticeably lower than the actual request traffic, or the logs do not match the expected service, troubleshoot in the following order:

  1. Check whether the Container Filtering rule in the Logtail configuration is set correctly. An incorrect filtering rule can prevent logs from the target container from being collected.

  2. Confirm that the service-linked role AliyunServiceRoleForEAS associated with the EAS service has sufficient SLS write permissions. Insufficient permissions can cause some logs to fail to be written.

  3. Refer to this topic to recheck your log collection configuration and the permissions of the service-linked role.

Q: How do I query which PAI-EAS services are writing SLS logs and how much log volume each service has?

In the query and analysis page of the target Logstore, run the following query statement:

* | select service_name, count(1) as log_count where service_name != '' group by service_name order by log_count desc

This statement aggregates the log volume (log_count) by service name (service_name) and sorts the results in descending order, helping you identify which services are writing logs so that you can manage or disable collection for them in bulk. You can use this together with the "How do I disable SLS log collection for a PAI-EAS service?" entry in this section to first identify, and then disable, collection.

Previous：Logging, monitoring, and alertingNext：View service monitoring metrics