Configure sidecar startup and shutdown

Feature description

Alibaba Cloud Container Compute Service (ACS) clusters do not support DaemonSets due to the limits of virtual nodes. To achieve a DaemonSet-like effect, you can add sidecar containers to pods in ACS clusters. The lifecycle of sidecar containers added to a pod is not independent of the lifecycle of the pod. To achieve a DaemonSet-like effect, you must configure the startup and exit priorities of the sidecar containers. For example, you may need the sidecar containers to start before the main application containers during pod creation. If you use pods of the Job type, you may need the sidecar containers to be forcefully terminated after the main application containers are terminated.

To meet the preceding requirements, ACS provides the following methods to configure the startup and exit priorities of sidecar containers.

• Kubernetes-native sidecar container configuration

  In Kubernetes 1.29 and later, support for native sidecar configuration is enabled by default. You can configure a sidecar by defining it as an init container and setting its restartPolicy to Always.

  Note

  The native approach implements the sidecar container as a special init container. During Pod startup, application containers wait until the sidecar container has started. The restartPolicy: Always setting allows the sidecar container to start, stop, and restart without affecting the main application container or other init containers.

• ACS-optimized sidecar container configuration

  For Kubernetes versions 1.28 and earlier, ACS allows you to set a special environment variable, __IS_SIDECAR__, to mark a regular container as a sidecar container.

  Important

  The ACS-optimized configuration allows you to declare a regular container as a sidecar. It starts before other regular containers and is automatically retried on failure, as if its restartPolicy were Always. Additionally, ACS clusters on earlier Kubernetes versions (1.28 and below) include compatibility features to ensure the container status is updated correctly.

  However, in newer Kubernetes versions or other cluster types, this method is limited by Kubernetes restrictions on container status updates. After a sidecar configured using the ACS-optimized method fails and restarts, its containerStatus and the Pod's status will not be updated to Running. For an accurate status, rely on the actual Pod state. We recommend that you upgrade your cluster to version 1.29 or later and use the Kubernetes-native sidecar container configuration.

Sidecar container configuration methods

Example

1. Create a file named test-sidecar.yaml and copy the following content to the file. The file is used to create a Job that provisions two containers named app and sidecar. The app container is an application container. The sidecar container is a sidecar container.

   ACS-optimized sidecar container configuration

   apiVersion: batch/v1
   kind: Job
   metadata:
     name: test
   spec:
     template:
       metadata:
         labels:
           app: test
       spec:
         containers:
         - name: app
           image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/busybox:1.28
           command: ['sh', '-c', 'for i in $(seq 1 10);do echo "logging" >> /var/logs.txt; sleep 1; done']
         - name: sidecar
           image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/busybox:1.28
           command: ['sh', '-c', 'touch /var/logs.txt && tail -F /var/logs.txt']
           env:
           - name: __IS_SIDECAR__   # Add an environment variable to the container.
             value: "true"          # Specify the container as a sidecar container.
         restartPolicy: Never
     backoffLimit: 2

   Kubernetes-native sidecar container configuration

   apiVersion: batch/v1
   kind: Job
   metadata:
     name: test
   spec:
     template:
       metadata:
         labels:
           app: test
       spec:
         initContainers:
         - name: sidecar
           image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/busybox:1.28
           command: ['sh', '-c', 'touch /var/logs.txt && tail -F /var/logs.txt']
           restartPolicy: Always  # Specify the container as a sidecar container.
         containers:
         - name: app
           image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/busybox:1.28
           command: ['sh', '-c', 'for i in $(seq 1 10);do echo "logging" >> /var/logs.txt; sleep 1; done']
         restartPolicy: Never
     backoffLimit: 2

2. Run the following command to create a Job:

   kubectl apply -f test-sidecar.yaml

3. View the details of the Job and pod and observe the effect of the environment variable.

   • Confirm that the Job has completed and its status is Succeeded.

     kubectl describe job <job-name>

     Example:

     Parallelism:    1
     Completions:    1
     Completion Mode:  NonIndexed
     Start Time:     Tue, 29 Oct 2024 11:24:18 +0800
     Completed At:   Tue, 29 Oct 2024 11:25:17 +0800
     Duration:       59s
     Pods Statuses:  0 Active (0 Ready) / 1 Succeeded / 0 Failed
     Pod Template:

4. View the details of the sidecar container and check the start sequence and actual exit code of containers

   kubectl describe pod <pod-name>

   • The following figure shows that the sidecar container was started before the app container. This ensures that the dependent functions required by the main container such as traffic routing are ready in advance. The following figure also shows that the sidecar container was terminated 10 seconds after the app container stopped running. This ensures that the pod provisioned by the Job can be terminated as normal.

     Events:
       Type    Reason     Age    From                Message
       ----    ------     ----   ----                -------
       Normal  Scheduled  5m11s  default-scheduler   Successfully assigned default/test-729nd to virtual-kubelet-cn-shenzhen-f
       Normal  Pulling    5m13s  kubelet             Pulling image "registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/busybox:1.28"
       Normal  Pulled     5m12s  kubelet             Successfully pulled image "registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/busybox:1.28" in 384ms
       Normal  Created    5m12s  kubelet             Created container sidecar
       Normal  Started    5m12s  kubelet             Started container sidecar
       Normal  Pulling    5m12s  kubelet             Pulling image "registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/busybox:1.28"
       Normal  Pulled     5m12s  kubelet             Successfully pulled image "registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/busybox:1.28" in 342ms
       Normal  Created    5m12s  kubelet             Created container app
       Normal  Started    5m12s  kubelet             Started container app
       Normal  Killing    5m1s   kubelet             Stopping container sidecar

   • The following figure shows that the sidecar container was forcefully terminated and the exit code is 0. This ensures that the pod can be terminated with success or failure as normal, without being affected by the sidecar container.

     sidecar:
         Container ID:  containerd://bd85de5ad7ee1cb5a4807b094f2d41fa90881916857172 0e73ef0dfbd26f1dfb
         Image:         registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/busybox:1.28
         Image ID:      registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/busybox@sha256:141c253bc4c3fd0a20
         Port:          <none>
         Host Port:     <none>
         Command:
           sh
           -c
           touch /var/logs.txt && tail -F /var/logs.txt
         State:          Terminated
           Reason:       Completed
           Message:      Force this container to be success(137, Error, )
           Exit Code:    0
           Started:      Tue, 29 Oct 2024 11:24:32 +0800
           Finished:     Tue, 29 Oct 2024 11:25:13 +0800
         Ready:          False
         Restart Count:  0