Configure application log collection using Pod environment variables-Container Service for Kubernetes(ACK)-阿里云帮助中心

ACK One register clusters are integrated with Simple Log Service. You can enable Simple Log Service for an Alibaba Cloud Container Compute Service (ACS) cluster to collect stdout and text logs from containers in the ACS cluster.

Pod environment variables let you configure log collection per application directly in the pod spec—without modifying cluster-level DaemonSet configurations or restarting the log agent. This topic describes how to configure log collection for pods in an ACS cluster by using pod environment variables.

Prerequisites

Before you begin, ensure that you have:

An ACS cluster with Simple Log Service enabled
A Simple Log Service project associated with the cluster (default name: k8s-log-{ACS cluster ID})

Step 1: Configure log collection when you create an application

YAML templates comply with Kubernetes syntax. Use the env field to define Collection Configuration and Custom Tag parameters.

The following example shows a pod with two log collection configurations and one custom tag:

apiVersion: v1
kind: Pod
metadata:
  name: my-demo
  labels:
    alibabacloud.com/acs: "true"
    alibabacloud.com/compute-class: general-purpose
    alibabacloud.com/compute-qos: default
spec:
  containers:
  - name: my-demo-app
    image: 'registry.cn-hangzhou.aliyuncs.com/log-service/docker-log-test:latest'
    env:
    - name: aliyun_logs_log-stdout   # Collect stdout to Logstore "log-stdout"
      value: stdout
    - name: aliyun_logs_log-varlog   # Collect /var/log/*.log to Logstore "log-varlog"
      value: /var/log/*.log
    - name: aliyun_logs_mytag1_tags  # Attach custom tag tag1=v1 to all logs
      value: tag1=v1
    command: ["sh", "-c"]
    args: ["echo 'Starting my demo app'; sleep 3600"]

All log collection environment variables use aliyun_logs_ as the prefix. The format is:

Collection configuration: aliyun_logs_{key} — creates a Logstore named {key} and routes logs there
Custom tag: aliyun_logs_{key}_tags — appends a tag to all log data collected from the container

In the example above:

aliyun_logs_log-stdout creates a Logstore named log-stdout and collects stdout from the container into it.
aliyun_logs_log-varlog creates a Logstore named log-varlog and collects /var/log/*.log files into it.
aliyun_logs_mytag1_tags appends tag1=v1 to all collected log data. mytag1 is the tag name (without underscores).

Note If you collect log files from a path other than stdout, set volumeMounts in the pod spec. Set mountPath to /var/log to allow Logtail to read from /var/log/*.log.

Note If you have additional requirements—such as routing logs to a specific project or Logstore, configuring retention, or setting the Logstore type—see Step 2: Configure advanced settings in the env field.

Step 2: Configure advanced settings in the env field

Use the following environment variables to customize log collection beyond the defaults.

Required variable

Variable	Description	Example
`aliyun_logs_{key}`	Specifies what to collect. Set to `stdout` for container stdout, or a file path for log files. `{key}` must contain only lowercase letters, digits, and hyphens (-). If `aliyun_logs_{key}_logstore` is not set, a Logstore named `{key}` is created automatically.	`- name: aliyun_logs_access-log`<br> `value: /var/log/nginx/access.log`

Note The default log collection mode is simple mode. To parse log data, use the Simple Log Service console.

Important

{key} is the name of the log collection configuration and must be unique in the ACS cluster.

Optional variables

Variable	Description	Default	Example
`aliyun_logs_{key}_tags`	Adds a custom tag to log data. Format: `{tag-key}={tag-value}`.	None	`- name: aliyun_logs_catalina_tags`<br> `value: app=catalina`
`aliyun_logs_{key}_project`	Specifies the Simple Log Service project to store logs. The project must be in the same region as the `alibaba-log-controller` component.	Project specified at cluster creation	`- name: aliyun_logs_catalina_project`<br> `value: my-k8s-project`
`aliyun_logs_{key}_logstore`	Specifies the Logstore to store logs.	`{key}`	`- name: aliyun_logs_catalina_logstore`<br> `value: my-logstore`
`aliyun_logs_{key}_shard`	Specifies the number of shards for the Logstore. Valid values: 1–10.	2	`- name: aliyun_logs_catalina_shard`<br> `value: 4`
`aliyun_logs_{key}_ttl`	Specifies the log retention period in days. Valid values: 1–3650. Set to 3650 for permanent retention.	90	`- name: aliyun_logs_catalina_ttl`<br> `value: 3650`
`aliyun_logs_{key}_machinegroup`	Specifies the node group where the application is deployed.	Node group where `alibaba-log-controller` is deployed	`- name: aliyun_logs_catalina_machinegroup`<br> `value: my-machine-group`
`aliyun_logs_{key}_logstoremode`	Specifies the Logstore type. Valid values: `standard`, `query`. See below for when to use each type.	`standard`	`- name: aliyun_logs_catalina_logstoremode`<br> `value: query`

Important

_shard, _ttl, and _logstoremode only take effect when the Logstore does not already exist. If the specified Logstore exists, these variables are ignored.

Choosing a Logstore type

Type	Supports log analysis	Index traffic cost	Use when
`standard`	Yes (including SQL)	Standard	Real-time monitoring, interactive analysis, building observability systems
`query`	Query only (no SQL)	~50% of standard	Large data volumes, long retention (weeks or months), or when log analysis is not required

Scenario 1: Collect logs from multiple applications to the same Logstore

Set aliyun_logs_{key}_logstore to the same Logstore name across applications.

The following example collects stdout from two applications into stdout-logstore. The {key} values (app1-stdout and app2-stdout) are different to keep log collection configuration names unique, while both point to the same Logstore.

Configure the following environment variables for Application 1:

env:
- name: aliyun_logs_app1-stdout
  value: stdout
- name: aliyun_logs_app1-stdout_logstore
  value: stdout-logstore

Configure the following environment variables for Application 2:

env:
- name: aliyun_logs_app2-stdout
  value: stdout
- name: aliyun_logs_app2-stdout_logstore
  value: stdout-logstore

Scenario 2: Collect logs from different applications to different projects

In each Simple Log Service project, create a machine group and set its custom identifier to k8s-group-{cluster-id}, where {cluster-id} is the ID of the cluster. You can use any name for the machine group.
In the pod spec for each application, specify the project, Logstore, and machine group.

The following example routes Application 1 and Application 2 to separate projects. If both applications run in the same cluster, they can share the same machine group.

Application 1:

env:
- name: aliyun_logs_app1-stdout
  value: stdout
- name: aliyun_logs_app1-stdout_project
  value: app1-project
- name: aliyun_logs_app1-stdout_logstore
  value: app1-logstore
- name: aliyun_logs_app1-stdout_machinegroup
  value: app1-machine-group

Application 2:

env:
- name: aliyun_logs_app2-stdout
  value: stdout
- name: aliyun_logs_app2-stdout_project
  value: app2-project
- name: aliyun_logs_app2-stdout_logstore
  value: app2-logstore
- name: aliyun_logs_app2-stdout_machinegroup
  value: app1-machine-group

Step 3: View logs in the Simple Log Service console

After configuration is complete, application logs are collected and stored in Simple Log Service. To view the logs:

Log on to the Simple Log Service console.
In the Projects section, select the project corresponding to the ACS cluster. The default project name is k8s-log-{ACS cluster ID}.
Go to the Logstore tab. In the Logstore list, find the Logstore configured for log collection. Move the pointer over the Logstore name and click the icon. Then click Search & Analysis.

On the log query page, you can view the stdout and text logs collected from the container, and see the custom tags appended to the log fields.