Route logs to multiple destinations with LoongCollector-Simple Log Service(SLS)-阿里云帮助中心

A single LoongCollector collection configuration can send logs to multiple Logstores, avoiding the configuration bloat and resource waste of maintaining separate configurations for each destination.

Use cases

In complex system environments, you often need to distribute logs from a single source to different destinations based on their content or attributes to meet various storage, analysis, or compliance requirements. For example:

Log categorization and storage: Business services may write both regular operational logs and audit logs to the same file. Audit logs often require long-term archival for compliance, while operational logs are needed only for short-term troubleshooting. Sending them to logstores with different lifecycle policies can significantly reduce storage costs.
Separating logs from multiple containers in a pod: In Kubernetes, a single pod can run multiple containers, such as an application container and a sidecar. You may need to collect their standard output logs into separate logstores for analysis.

Traditionally, you create a separate collection configuration for each destination Logstore, which leads to configuration sprawl, complicates management, and runs duplicate collection tasks that waste CPU and memory. The multi-destination feature solves these problems with a single configuration.

How it works

The core mechanism is to route logs based on a tag. The entire data lifecycle, from collection to final delivery, is handled within a single collection task.

Duplication: If a log entry matches the routing rules for multiple output destinations, it is duplicated and sent to all matching destination logstores.
Discard on no match: If a log entry does not match the routing rules of any output destination, it is discarded and not sent anywhere.
Default delivery: If an output destination has no routing rules configured, all collected logs are sent to that destination.

Procedure

The following two scenarios show how to configure the multi-destination feature of LoongCollector.

Important

This feature is available only for LoongCollector 3.0.0 and later. Logtail does not support this feature.

Access the global collection configuration

Unlike standard configurations, a collection configuration for multi-destination delivery is not created under a specific logstore. Instead, you create it from the Resources > Configurations page of your project.

Log on to the Simple Log Service console, and then click the name of your project.
In the left-side navigation pane, choose Resources > Configurations.

Note
This page centralizes the management of all collection configurations within the project, including orphan configurations that remain after you delete their associated logstores.

Scenario 1: Categorize logs from a single file

Send regular service logs and audit logs from the same log file to two separate Logstores with different storage lifecycles.

On the Configurations page, click Create Logtail Configuration.
In the Quick Data Import dialog box, find the Single Line - Text Logs card and click Integrate Now.
Configure the Machine Group Configurations and click Next.
- Scenario: Servers
- Installation Environment: ECS
- Select Machine Group: Select the machine group that contains the target host and add it to Applied Machine Group. If LoongCollector is not installed on the host, see Configure a machine group (install LoongCollector) to complete the setup.
Configure the Logtail Configurations settings and click Next.
- - Configuration Name: Specify a name for the collection configuration. The name must be unique within the project and cannot be changed after creation.
- Input Configurations
  - File Path: Specify the log collection path.
    - Linux: The path must start with a forward slash (/). For example, /data/mylogs/**/*.log matches all files that have the .log extension in the /data/mylogs directory and its subdirectories.
    - Windows: The path must start with a drive letter. For example, C:\Program Files\Intel\**\*.Log.
  - Maximum Directory Monitoring Depth: Specify the maximum directory depth that the wildcard ** in the File Path can match. The default value is 0, which limits monitoring to the current directory.
- Processor Configurations
  - Log Sample: Click Add Sample Log and enter the following content. In this sample, entries with the action field are audit logs.
```
2025-10-28 11:22:21 INFO User test deleted a record | action=DELETE
2025-10-28 11:22:21 INFO Cache refreshed
2025-10-28 11:22:22 INFO User guest attempted unauthorized access | action=ACCESS_DENIED
2025-10-28 11:22:23 INFO Connected to database
2025-10-28 11:22:23 INFO User admin logged in | action=LOGIN
2025-10-28 11:22:24 INFO Connected to database
2025-10-28 11:22:25 INFO User guest attempted unauthorized access | action=ACCESS_DENIED
2025-10-28 11:22:26 INFO User test deleted a record | action=DELETE
2025-10-28 11:22:26 INFO Cache refreshed
2025-10-28 11:22:29 INFO Processed 200 requests in the last minute
```
  - Processing Method: Select SPL.
  - SPL Statement: Enter the following SPL statement.
```
* | parse-regexp content, '^(\d+-\d+-\d+\s\d+:\d+:\d+)\s+(\w+)\s+(.*?)\s*(?:\|\s*action=(\w+))?$' as time, level, message, action | project-away content | extend "__tag__:type" = case when action = '' then 'service_log' else 'audit_log' end
```
    This SPL statement performs the following actions:
    - It uses a regular expression to extract the time, level, message, and action fields from the log content, and then removes the original content field.
    - It adds a new tag named type and sets its value to audit_log if the action field is not empty, or to service_log otherwise.
      
      Note
      Tag naming conventions: When you create or reference a Tag in SPL, the field name must be prefixed with __tag__: , such as __tag__:type. However, when you configure routing rules, you only need to enter the Tag name type.
- Output Configurations: Click to expand the output settings and add two output destinations.
  - Configure Output Target 1: Store standard logs with a type tag value of service_log.
    - Logstore: Select the logstore for service logs, such as a logstore named service_log.
    - Compression Method: The supported methods are lz4 and zstd. The default is lz4.
    - Route Settings:
      - Tag Name: Enter the field name directly. Do not add the __tag__: prefix. In this example, the tag name is set to type.
      - Tag Value: Set to service_log.
      - Discard this tag?: Turn on this switch if you do not want to include this tag in the uploaded logs.
  - Configure output destination 2 to store audit logs. These are logs where the value of the type tag is audit_log.
    - Logstore: Select the logstore for audit logs, such as a logstore named audit_log.
    - Compression Method: The supported methods are lz4 and zstd. The default is lz4.
    - Route Settings:
      - Tag Name: type.
      - Tag Value: audit_log.
      - Discard this tag?: Turn on this switch if you do not want to include this tag in the uploaded logs.
Query and Analysis Configurations:
- By default, full-text index is enabled, which allows you to search for keywords in the raw log content.
- To query by field, click Automatic Index Generation after the preview data loads. Simple Log Service then creates an index based on the first log entry in the preview.
After you complete the configuration, click Next to complete the setup.
After you complete and save the configuration, LoongCollector automatically distributes logs to the corresponding logstores based on the action field in the logs.
- The audit_log logstore should only contain log records with an action field.
  
  Query the raw logs in the audit_log logstore. The results show that all log entries contain the action field, with values such as ACCESS_DENIED, DELETE, and LOGIN. This confirms that the logs are correctly categorized and stored.
- The service_log logstore should contain only general service logs.
  
  Query the raw logs in the service_log logstore. The results show only general INFO-level service logs, such as Processed 200 requests in the last minute, Cache refreshed, and Connected to database, and contain no error logs or other types.

Scenario 2: Separate container logs in a pod

Collect standard output logs from two containers (app1-container and app2-container) within the same Pod in an ACK cluster and send them to separate Logstores, enabling fine-grained log separation without multiple collection configurations.

On the Configurations page, click Create Logtail Configuration.
In the Quick Data Import dialog box, find the K8s - Stdout and Stderr - New Version card and click Integrate Now.
Configure the Machine Group Configurations and click Next.
- Scenario: Kubernetes Clusters
- Deployment Method: ACK Daemonset
- Select Machine Group: From Source Machine Group, add the default machine group k8s-group-${cluster_id} to Applied Machine Group on the right.
Configure the Logtail Configurations settings and click Next.
- - Configuration Name: Specify a name for the collection configuration. The name must be unique within the project and cannot be changed after creation.
- Input Configurations
  - Enable the switches for Standard Output or Standard Error as needed. By default, both are enabled.
  - Container Filtering: Set filter conditions to limit the scope of log collection.
    - Enable container filtering, click Add > K8s Namespace Regular Matching, and enter ^(app)$ to collect container data only from the app namespace.
- Output Configurations: Click to expand the output settings and add two output destinations.
  
  K8s standard output collection automatically attaches a series of tags that contain container metadata to logs, such as _container_name_. This scenario uses this built-in tag for routing.
  - Configure output destination 1: Store app1 container logs.
    - Logstore: Select the logstore for app1 logs, such as a logstore named app1-log.
    - Compression Method: The supported methods are lz4 and zstd. The default is lz4.
    - Route Settings:
      - Tag Name: Enter _container_name_. The __tag__: prefix is not required.
      - Tag Value: app1-container.
      - Discard this tag?: Turn on this switch if you do not want to include this tag in the uploaded logs.
  - Configure Output Destination 2: Store app2 container logs.
    - Logstore: Select the logstore for app2 logs, such as a logstore named app2-log.
    - Compression Method: The supported methods are lz4 and zstd. The default is lz4.
    - Route Settings:
      - Tag Name: _container_name_.
      - Tag Value: app2-container.
      - Discard this tag?: Turn on this switch if you do not want to include this tag in the uploaded logs.
Query and Analysis Configurations:
- By default, full-text index is enabled, which allows you to search for keywords in the raw log content.
- To query by field, click Automatic Index Generation after the preview data loads. Simple Log Service then creates an index based on the first log entry in the preview.
After you complete the configuration, click Next to complete the setup.
After you save the configuration, standard output logs from the different containers are automatically routed to their specified logstores.
- The app1-log logstore:
  
  In the Simple Log Service console, set the time range to the last 3 minutes. The query returns 1,454 logs. The source is app1-container, the output type is stdout, and the content is the Hello from app1 message that the container prints every second. This confirms that logs from the app1 container are collected to this logstore.
- The app2-log logstore:
  
  The query returns 1,454 log entries. The source is the standard output of the app2-container, and the content is the periodically printed Hello from app2 message. This confirms that logs from the app2 container are collected and sent to this logstore.