Collect logs from Knative Services

CRD - AliyunPipelineConfig (recommended)

Create a Custom Resource (CR) from the AliyunPipelineConfig CRD. The configuration applies automatically. To modify it, update the CR.

Key constraints:

• The configName value must be unique within the SLS project.

• Create one CR per Logtail configuration. If multiple CRs reference the same configuration, only the first CR takes effect.

Steps:

1. Get the kubeconfig file and connect to your cluster using kubectl.

2. Create a YAML file:

   vim cube.yaml

3. Add the configuration to the YAML file. See the examples below.

4. Apply the configuration:

   kubectl apply -f cube.yaml

   Logtail starts collecting logs from the specified containers and sending them to SLS.

apiVersion: telemetry.alibabacloud.com/v1alpha1
# Create a CR from the ClusterAliyunPipelineConfig CRD.
kind: ClusterAliyunPipelineConfig
metadata:
  # The name must be unique in the Kubernetes cluster. It is also used as the Logtail configuration name.
  name: example-k8s-file
spec:
  # Specify the project to collect logs into.
  project:
    name: k8s-log-test
  # Create a Logstore to store logs.
  logstores:
    - name: k8s-file
  config:
    inputs:
      # Use input_file to collect text logs from containers.
      - Type: input_file
        # Log file path inside the containers.
        FilePaths:
          - /data/logs/app_1/**/test.LOG
        # Enable container discovery.
        EnableContainerDiscovery: true
        # Filter containers. Multiple conditions use logical AND.
        ContainerFilters:
          # Match containers in the default namespace.
          K8sNamespaceRegex: default
          # Match containers whose name contains "app".
          K8sContainerRegex: ^(.*app.*)$
    flushers:
      # Use flusher_sls to send logs to a Logstore.
      - Type: flusher_sls
        Logstore: k8s-file
        # For valid endpoint and region values, see https://www.alibabacloud.com/help/en/sls/developer-reference/service-entrance
        Endpoint: cn-hangzhou.log.aliyuncs.com
        Region: cn-hangzhou
        TelemetryType: logs

apiVersion: telemetry.alibabacloud.com/v1alpha1
# Create a CR from the ClusterAliyunPipelineConfig CRD.
kind: ClusterAliyunPipelineConfig
metadata:
  name: example-k8s-file
spec:
  project:
    name: k8s-log-test
  logstores:
    - name: k8s-file
  config:
    # Optional: add a sample log for reference.
    sample: |
      2024-06-19 16:35:00 INFO test log
      line-1
      line-2
      end
    inputs:
      - Type: input_file
        FilePaths:
          - /data/logs/app_1/**/test.LOG
        EnableContainerDiscovery: true
        # Enable multi-line log collection.
        Multiline:
          # Use a custom regex to identify the first line of each log entry.
          Mode: custom
          StartPattern: \d+-\d+-\d+.*
    processors:
      # Parse logs using a regular expression. Capturing groups define extracted fields.
      - Type: processor_parse_regex_native
        SourceKey: content
        Regex: (\d+-\d+-\d+\s*\d+:\d+:\d+)\s*(\S+)\s*(.*)
        Keys: ["time", "level", "msg"]
    flushers:
      - Type: flusher_sls
        Logstore: k8s-file
        Endpoint: cn-hangzhou.log.aliyuncs.com
        Region: cn-hangzhou
        TelemetryType: logs

CRD - AliyunLogConfig

Create a CR from the AliyunLogConfig CRD. The configuration is applied automatically. To modify it, update the CR.

Steps:

1. Get the kubeconfig file and connect to your cluster using kubectl.

2. Create a YAML file:

   vim cube.yaml

3. Add the configuration. See the example below. The configName value must be unique in the SLS project.

4. Apply the configuration:

   kubectl apply -f cube.yaml

apiVersion: log.alibabacloud.com/v1alpha1
kind: AliyunLogConfig
metadata:
  name: example-k8s-file
  namespace: kube-system
spec:
  # If left blank, the project named k8s-log-<your_cluster_id> is used.
  project: k8s-log-test
  # If the specified Logstore does not exist, SLS creates it automatically.
  logstore: k8s-file
  logtailConfig:
    # Set to "file" for text logs.
    inputType: file
    configName: example-k8s-file
    inputDetail:
      # Simple mode for text logs.
      logType: common_reg_log
      logPath: /data/logs/app_1
      # Supports wildcard characters (* and ?). Example: log_*.log
      filePattern: test.LOG
      # Required for container log collection.
      dockerFile: true
      advanced:
        k8s:
          K8sPodRegex: '^(app.*)$'

SLS console

1. Log on to the Simple Log Service console.

2. In the Quick Data Import section, click Import Data. In the Import Data dialog box, click the Kubernetes - File card.

   

3. Select the project and Logstore, then click Next. Use the project created during Logtail installation.

4. In the Machine Group Configurations step, configure the following:

   1. Select a machine group option based on your setup:

      • Kubernetes Clusters > ACK Daemonset

      • Kubernetes Clusters > Self-managed Cluster in DaemonSet Mode

      Important

      Subsequent settings vary based on which option you select.

   2. Confirm the machine groups appear in Applied Server Groups, then click Next. SLS automatically creates a machine group named k8s-group-${your_k8s_cluster_id} during Logtail installation.

      Important

      • To create a new machine group, click Create Machine Group and configure the parameters. See Collect container logs from ACK clusters.

      • If a machine group's heartbeat status is FAIL, click Automatic Retry. If the issue persists, see How do I troubleshoot an error related to a Logtail machine group?

5. Create a Logtail configuration and click Next. Log collection begins after creation. Container filtering options Available for Logtail 1.0.34+. Earlier versions: environment variables and container labels only.

   • K8s Pod Name Regular Matching: Match containers by pod name using a regular expression. Example: ^(nginx-log-demo.*)$ matches all containers in pods whose name starts with nginx-log-demo.

   • K8s Namespace Regular Matching: Match containers by namespace. Example: ^(default|nginx)$ matches containers in the nginx and default namespaces.

   • K8s Container Name Regular Matching: Match containers by container name (as defined in spec.containers). Example: ^(container-test)$ matches containers named container-test.

   • Container Label Whitelist / Blacklist: Filter containers by container labels (key-value pairs). Kubernetes namespaces and container names map to the labels io.kubernetes.pod.namespace and io.kubernetes.container.name. Key-value pairs are evaluated with OR logic. String matching is the default; prefix with ^ and suffix with $ for regex matching.

   • Environment Variable Whitelist / Blacklist: Filter containers by environment variable key-value pairs. OR logic applies. Example: set Environment Variable Name to NGINX_SERVICE_PORT and Environment Variable Value to ^(80|6379)$ to match containers on port 80 or 6379.

   • Kubernetes Pod Label Whitelist / Blacklist: Filter containers by Kubernetes pod labels (distinct from container labels). OR logic applies. Example: set Label Name to environment and Label Value to ^(dev|pre)$ to match containers with pod labels environment:dev or environment:pre.

   Note A Logtail configuration takes up to 3 minutes to take effect.
   Important

   Container labels (retrieved via docker inspect) differ from Kubernetes labels. See Obtain labels and Obtain environment variables.

   Global configurations

   Parameter	Description
   Configuration name	Unique name within the project. Cannot be changed later.
   Log topic type	Log topic source. Machine Group Topic: by machine group. File Path Extraction: regex on file path. Custom: user-defined. See Log topics.
   Advanced parameters	Optional. See CreateLogtailPipelineConfig.

   Input configurations

   Parameter	Description
   Logtail deployment mode	Select Daemonset for this use case.
   File path type	Select Path in Container in most cases. Select Host Path only for hostPath-mounted volumes.
   File path	Specify the log file path. Linux paths start with / (example: /apsara/nuwa//app.Log`). Windows paths start with a drive letter (example: `C:\Program Files\Intel\\*.Log). Use * and ? as wildcard characters only. Examples: /apsara/nuwa/**/*.log collects all .log files under /apsara/nuwa recursively. /var/logs/app_*/**/*.log collects .log files in subdirectories matching app_* under /var/logs. /var/log/nginx/**/access* collects files starting with access under /var/log/nginx recursively.
   Maximum directory monitoring depth	Maximum subdirectory depth for the ** wildcard. 0 monitors only the specified directory. Keep low — larger values increase resource consumption and may cause latency.
   Enable container metadata preview	Shows matched container metadata after configuration creation.
   Container filtering	Filter containers for log collection. Logtail earlier than 1.0.34: environment variables and container labels only. Logtail 1.0.34+: Kubernetes-level filtering (K8s Pod Name, Namespace, Container Name, Pod Label).
   Log tag enrichment	Add log tags from environment variables and pod labels.
   File encoding	Encoding format of the log files.
   First collection size	Data size for initial collection. Default: 1,024 KB. Range: 0–10,485,760 KB. Files smaller than 1,024 KB are read from the start; larger files are read from the last 1,024 KB.
   Collection blacklist	Exclude directories or files by exact path or wildcards (* and ?). Maximum 10 entries. Paths cannot end with /. Types: File Path, File, and Directory.
   Allow file to be collected multiple times	By default, one Logtail configuration per file. Enable to allow multiple configurations per file.
   Advanced parameters	Optional. See Create a Logtail pipeline configuration.

   Processor configurations

   Parameter	Description
   Log sample	Sample logs for processing configuration. Maximum: 1,500 characters.
   Multi-line mode	Multi-line log detection. Custom: use Regex to Match First Line for log boundaries. Multi-line JSON: each JSON object spans multiple lines. Processing Method If Splitting Fails: Discard (drop unmatched logs) or Retain Single Line (one line per entry).
   Processing method	Add processors for data processing. Logtail V2.0: freely combine native and extended processors (extended must follow native). Earlier versions: no mixing.

6. Create indexes and preview data, then click Next. Full-text indexing is enabled by default. To create field indexes automatically, click Automatic Index Generation. See Create indexes.

   Important

   Use full-text indexes to query all fields. Use field indexes to query specific fields and reduce index traffic. Field indexes are required for log analysis (queries must include a SELECT statement).

7. Click Query Log. Wait about 1 minute for indexes to take effect, then view logs on the Raw Logs tab. See Guide to log query and analysis.

Environment variables

Configure log collection in your Knative Service YAML using environment variables. Names must use the aliyun_logs_ prefix. The {key} part allows only lowercase letters, digits, and hyphens (-).

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: helloworld-go-log
spec:
  template:
    spec:
      containers:
      - name: my-demo-app
        image: 'registry.cn-hangzhou.aliyuncs.com/log-service/docker-log-test:latest'
        env:
        # Collect stdout to Logstore "log-stdout".
        - name: aliyun_logs_log-stdout
          value: stdout
        # Collect /var/demo/*.log files to Logstore "log-varlog".
        - name: aliyun_logs_log-varlog
          value: /var/demo/*.log
        # Add a custom tag to all logs collected by this container.
        - name: aliyun_logs_mytag1_tags
          value: tag1=v1
        volumeMounts:
        - name: volumn-sls-mydemo
          mountPath: /var/demo
        # Optional: add a sleep command if the pod restarts repeatedly.
        command: ["sh", "-c"]
        args: ["sleep 3600"]
      volumes:
      - name: volumn-sls-mydemo
        emptyDir: {}