inject sidecar containers into virtual node pods

Step 1: Create the ConfigMap

Create a ConfigMap in the kube-system namespace to hold the filebeat configuration.

kubectl apply -f filebeat-config.yaml

The following filebeat-config.yaml mounts the configuration file to the sidecar container. The placeholder variables (such as ${NODE_NAME} and ${ELASTIC_CLOUD_ID}) are not evaluated in this example and do not need to be replaced.

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  namespace: kube-system
  labels:
    k8s-app: filebeat
data:
  filebeat.yml: |-
    filebeat.inputs:
    - type: log
      paths:
        - /var/log/std/*.log
      processors:
        - add_kubernetes_metadata:
            host: ${NODE_NAME} # Not evaluated in this example. Use as-is.
            matchers:
            - logs_path:
                logs_path: "/var/log/std/"

    # To enable hints-based autodiscover, remove filebeat.inputs and uncomment:
    #filebeat.autodiscover:
    #  providers:
    #    - type: kubernetes
    #      node: ${NODE_NAME}
    #      hints.enabled: true
    #      hints.default_config:
    #        type: container
    #        paths:
    #          - /var/log/containers/*${data.kubernetes.container.id}.log

    processors:
      - add_cloud_metadata:
      - add_host_metadata:

    cloud.id: ${ELASTIC_CLOUD_ID}  # Not evaluated in this example. Use as-is.
    cloud.auth: ${ELASTIC_CLOUD_AUTH}  # Not evaluated in this example. Use as-is.

    output.elasticsearch:
      hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
      username: ${ELASTICSEARCH_USERNAME}  # Not evaluated in this example. Use as-is.
      password: ${ELASTICSEARCH_PASSWORD}  # Not evaluated in this example. Use as-is.

Step 2: Deploy the SidecarSet

Create a SidecarSet to describe the filebeat sidecar container. The SidecarSet selects pods with the serverless.alibabacloud.com/virtual-node: "true" label and injects the sidecar automatically.

kubectl apply -f sidecarset.yaml

apiVersion: apps.kruise.io/v1alpha1
kind: SidecarSet
metadata:
  name: filebeat-sidecarset
spec:
  selector:
    matchLabels:
      serverless.alibabacloud.com/virtual-node: "true" # Selects all pods scheduled to virtual nodes.
  updateStrategy:
    type: NotUpdate
  containers:
  # This example uses busybox instead of the actual filebeat image.
  # To use filebeat, replace the container spec with:
  #- name: filebeat
  #  image: docker.elastic.co/beats/filebeat:8.6.1
  #  args: ["-c", "/etc/filebeat.yml", "-e"]
  - name: filebeat
    image: busybox
    imagePullPolicy: IfNotPresent
    args:
      - "/bin/sh"
      - "-c"
      - "cat /etc/filebeat.yml && sleep 36000"
    env:
    - name: ECI_SIDECAR_CONTAINER  # Causes the sidecar to exit after the application container exits.
      value: "true"
    volumeMounts:
    - name: config
      mountPath: /etc/filebeat.yml
      readOnly: true
      subPath: filebeat.yml
    - name: stdlog                 # Mounts the pod's standard output logs for the sidecar to read.
      mountPath: /var/log/std
      readOnly: true
  volumes:
  - name: config
    configMap:
      name: kube-system/filebeat-config  # Namespace/Name format for cross-namespace reference.
  - name: stdlog
    csi:
      driver: stdlogplugin.csi.alibabacloud.com

apiVersion: apps.kruise.io/v1alpha1
kind: SidecarSet
metadata:
  name: filebeat-sidecarset
spec:
  selector:
    matchLabels:      
      serverless.alibabacloud.com/virtual-node: "true" # Matches all pods scheduled to virtual nodes.
  updateStrategy:
    type: NotUpdate
  containers:
  - name: filebeat
    image: busybox
    imagePullPolicy: IfNotPresent    
    args: [
      "/bin/sh",
      "-c",
      "cat /etc/filebeat.yml && sleep 36000", # This example only prints the filebeat configuration content.
    ]    
    volumeMounts:
    - name: config
      mountPath: /etc/filebeat.yml
      readOnly: true
      subPath: filebeat.yml    
  volumes:
  - name: config
    configMap:
      name: kube-system/filebeat-config # Use the namespace/name format to reference a ConfigMap in another namespace.

Step 3: Authorize cross-namespace ConfigMap access

The application pod is in the default namespace, but the ConfigMap is in kube-system. Create a SidecarSetResourceBinding to grant the SidecarSet read-only access to the ConfigMap.

kubectl apply -f sidecarset-resourcebinding.yaml

# Grants filebeat-sidecarset read-only access to filebeat-config in the kube-system namespace.
apiVersion: sidecarset.alibabacloud.com/v1alpha1
kind: SidecarSetResourceBinding
metadata:
  name: filebeat-sidecarset-resourcebinding
  namespace: kube-system # Grants permissions only on resources in this namespace.
  labels:
spec:
  subjects:
    - kind: SidecarSet
      name: filebeat-sidecarset
  resourceRefs:
    - kind: ConfigMap # Read-only: Get, List, and Watch.
      name: filebeat-config

# Authorizes filebeat-sidecarset. Pods matching the SidecarSet can access the filebeat-config ConfigMap in the kube-system namespace.
apiVersion: sidecarset.alibabacloud.com/v1alpha1
kind: SidecarSetResourceBinding
metadata:
  name: filebeat-sidecarset-resourcebinding
  namespace: kube-system # This SidecarSetResourceBinding only authorizes resources in the kube-system namespace.
  labels:
spec:
  subjects:
    - kind: SidecarSet
      name: filebeat-sidecarset
  resourceRefs:
    - kind: ConfigMap # Only grants read-only permission (Get, List, Watch).
      name: filebeat-config

Step 4: Deploy the application pod

Deploy the echo-server application. Adding the alibabacloud.com/eci: "true" label schedules the pod to a virtual node, which also adds the serverless.alibabacloud.com/virtual-node: "true" label—triggering sidecar injection.

kubectl apply -f echo-server.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo-server
  labels:
    app: echo-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: echo-server
  template:
    metadata:
      labels:
        app: echo-server
        alibabacloud.com/eci: "true"  # Schedules the pod to a virtual node.
    spec:
      containers:
        - name: echo-server
          image: hashicorp/http-echo
          imagePullPolicy: IfNotPresent
          args:
            - -listen=:8080
            - -text="hello world"

Verify injection

1. Check that the sidecar was injected. Before injection, the pod shows 1/1 READY (application container only). After the SidecarSet injects the sidecar, the count becomes 2/2.

   kubectl get pod

   Expected output:

   NAME                          READY   STATUS    RESTARTS   AGE
   echo-server-f8bdc5844-r44nj   2/2     Running   0          14m

2. Verify the sidecar can read the application pod's standard output logs.

   kubectl exec echo-server-f8bdc5844-r44nj -c filebeat -- cat /var/log/std/echo-server/0.log

   Expected output:

   2025-04-29T11:26:06.783205694+08:00 stderr F 2025/04/29 03:26:06 Server is listening on :8080

3. Verify the sidecar can read the ConfigMap mounted from the kube-system namespace.

   kubectl exec echo-server-f8bdc5844-r44nj -c filebeat -- cat /etc/filebeat.yml

   Expand to view example output

   filebeat.inputs:
   - type: log
     paths:
       - /var/log/std/*.log
     processors:
       - add_kubernetes_metadata:
           host: ${NODE_NAME} # Not effective. Do not modify. Use directly.
           matchers:
           - logs_path:
               logs_path: "/var/log/std/"
   
   # To enable hints based autodiscover, remove `filebeat.inputs` configuration and uncomment this:
   #filebeat.autodiscover:
   #  providers:
   #    - type: kubernetes
   #      node: ${NODE_NAME}
   #      hints.enabled: true
   #      hints.default_config:
   #        type: container
   #        paths:
   #          - /var/log/containers/*${data.kubernetes.container.id}.log
   
   processors:
     - add_cloud_metadata:
     - add_host_metadata:
   
   cloud.id: ${ELASTIC_CLOUD_ID}  # Not effective. Do not modify. Use directly.
   cloud.auth: ${ELASTIC_CLOUD_AUTH}  # Not effective. Do not modify. Use directly.
   
   output.elasticsearch:
     hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
     username: ${ELASTICSEARCH_USERNAME}  # Not effective. Do not modify. Use directly.
     password: ${ELASTICSEARCH_PASSWORD}  # Not effective. Do not modify. Use directly. 

   The output shows the full contents of the filebeat.yml ConfigMap, confirming the cross-namespace mount is working.

Control container startup and exit order

Sidecar containers often need to start before application containers and exit after them. Configure this behavior using container startup and exit priorities.

The example SidecarSet in this topic also demonstrates an inline approach: setting the ECI_SIDECAR_CONTAINER: "true" environment variable causes the sidecar container to exit automatically after the application container exits. This is the recommended approach for virtual node scenarios.

Handle sidecar containers in Job pods

For Job workloads, a long-running sidecar container can block the Job from completing after the application container finishes. To prevent this, set ECI_SIDECAR_CONTAINER: "true" in the sidecar container environment:

env:
- name: ECI_SIDECAR_CONTAINER  # Causes the sidecar to exit after the application container exits.
  value: "true"

This causes the sidecar to exit automatically when the application container finishes, so the Job can complete normally. For full details, including how to ignore the sidecar's exit code, see Forcibly terminate sidecar containers and ignore exit codes.

Upgrade sidecar containers

To update a sidecar container already injected into running pods, use the OpenKruise sidecar hot upgrade feature. Hot upgrade updates the sidecar in-place without restarting the pod or affecting application availability, and is fully compatible with virtual node pods.

After triggering an upgrade, monitor progress by checking the SidecarSet status:

kubectl get sidecarset filebeat-sidecarset -o yaml

The status section reports the following counters:

Collect standard output logs

Mount the application pod's standard output logs into the sidecar container using a stdlog CSI volume. The sidecar can then forward logs to your logging backend without any changes to the application container.

The example SidecarSet in this topic already includes the stdlog volume configuration. For more details on the stdlog volume type, see Mount stdlog to a pod.

The stdlog volume mounts logs at /var/log/std/<container-name>/0.log inside the sidecar container:

volumes:
- name: stdlog
  csi:
    driver: stdlogplugin.csi.alibabacloud.com

apiVersion: apps.kruise.io/v1alpha1
kind: SidecarSet
metadata:
  name: filebeat-sidecarset
spec:
  selector:
    matchLabels:
      serverless.alibabacloud.com/virtual-node: "true" # Matches all pods scheduled to virtual nodes.
  updateStrategy:
    type: NotUpdate
  containers:
  # This example only prints the sidecar container log content.
  - name: filebeat
    image: busybox
    imagePullPolicy: IfNotPresent
    args: [
      "/bin/sh",
      "-c",
      "cat /var/log/std/filebeat/0.log && sleep 36000",
    ]    
    volumeMounts:    
    - name: stdlog # Mounts the pod's standard output log volume /var/log/std directory for the sidecar container to read.
      mountPath: /var/log/std
      readOnly: true
  volumes:  
  - name: stdlog
    csi:
      driver: stdlogplugin.csi.alibabacloud.com