Configure automatic sidecar injection for Agent Sandbox-Container Compute Service-阿里云帮助中心

Configure the spec.runtimes field in your Sandbox resource to automatically inject CSI storage mount and AgentRuntime sidecar configurations, eliminating manual YAML setup.

Prerequisites

An Agent Sandbox environment is created. Create an Agent Sandbox.
In the Add-ons section of your cluster, ensure that the ack-agent-sandbox-controller add-on is version v0.5.12 or later.

When the add-on is installed, a ConfigMap named sandbox-injection-config is automatically created in the sandbox-system namespace. This ConfigMap contains two configurations: agent-runtime and csi.

Before customizing this configuration, contact Alibaba Cloud technical support to validate your changes.

Overview

Manually configuring CSI and sidecar containers for dynamic storage mounts requires complex YAML. With automatic sidecar injection, you declare the spec.runtimes field in your SandboxSet or Sandbox resource, and the system injects the required configurations into new Sandbox instances. Two injection types are supported:

csi: Injects init container and volume configurations for CSI storage mounts, enabling shared storage such as NAS or OSS.

To enable dynamic storage mounting, you must allow privileged containers and access to the hostPath (/var/run/csi). You can submit a ticket to have the security restrictions lifted. However, you must assume responsibility for the associated security risks. For more information about this mechanism, see the shared responsibility model.
agent-runtime: Injects an AgentRuntime init container (such as envd) and adds required environment variables and lifecycle hooks to the business container.

Injecting agent-runtime is required for the Command and Filesystem interfaces. Automatic sidecar injection applies only to newly created Sandbox instances.

Configure sidecar injection

Declare the runtime types in the spec.runtimes field of a SandboxSet or Sandbox resource.

Example SandboxSet resource configuration

apiVersion: agents.kruise.io/v1alpha1
kind: SandboxSet
metadata:
  name: code-interpreter-inject-test
  namespace: default
spec:
  runtimes:
  - name: csi           # Enables CSI mount capabilities. New Sandbox instances are injected with the corresponding sidecar configuration.
  - name: agent-runtime # Injects environment management tools such as envd.
  replicas: 4
  template:
    metadata:
      labels:
        alibabacloud.com/acs: "true"
        alibabacloud.com/compute-class: agent-sandbox # Agent Sandbox instance type. 
        alibabacloud.com/compute-qos: default # Compute QoS: default/best-effort.
    spec:
      automountServiceAccountToken: false
      containers:
      - image: registry-cn-zhangjiakou-vpc.ack.aliyuncs.com/acs/code-interpreter:v1.6 # Replace this with the region of your cluster.
        imagePullPolicy: IfNotPresent
        name: sandbox
        resources:
          limits:
            cpu: "1"
            memory: 1Gi
          requests:
            cpu: "1"
            memory: 1Gi
      terminationGracePeriodSeconds: 30

Example Sandbox resource configuration

apiVersion: agents.kruise.io/v1alpha1
kind: Sandbox
metadata:
  name: code-interpreter-inject-test-xxx
  namespace: default
spec:
  runtimes:
  - name: csi           # Provides CSI mount capabilities.
  - name: agent-runtime # Injects environment management tools such as envd.
  ...

Example of a Pod after injection

# Note: The <region-id> in the image address is automatically replaced with the region ID of your cluster.
apiVersion: v1
kind: Pod
metadata:
  annotations:
    agents.kruise.io/created-by: sandbox
  labels:
    agents.kruise.io/sandbox-pool: code-interpreter-init-xxx
    alibabacloud.com/acs: "true"
  name: code-interpreter-init-xxx
  namespace: default
spec:
  automountServiceAccountToken: false
  containers:
  - env:
    # --- The following configurations are automatically injected ---
    - name: ENVD_DIR
      value: /mnt/envd
    - name: GODEBUG
      value: multipathtcp=0
    - name: POD_UID
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: metadata.uid
    # --- End of automatically injected configurations ---
    image: example:tag
    imagePullPolicy: IfNotPresent
    # --- The following configurations are automatically injected ---
    lifecycle:
      postStart:
        exec:
          command:
          - bash
          - -c
          - /mnt/envd/envd-run.sh
    # --- End of automatically injected configurations ---
    name: sandbox
    resources:
      limits:
        cpu: "2"
        memory: 2Gi
      requests:
        cpu: "2"
        memory: 2Gi
    volumeMounts:
    # --- The following configurations are automatically injected ---
    - mountPath: /mnt/envd
      name: envd-volume
    - mountPath: /run/csi/mount-root
      mountPropagation: HostToContainer
      name: mount-root
    - mountPath: /var/run/csi/sockets/nasplugin.csi.alibabacloud.com
      name: nas-plugin-dir
    - mountPath: /var/run/csi/sockets/ossplugin.csi.alibabacloud.com
      name: oss-plugin-dir
    # --- End of automatically injected configurations ---
  # --- The following configurations are automatically injected ---
  initContainers:
  - command:
    - sh
    - /workspace/entrypoint_inner.sh
    env:
    - name: ENVD_DIR
      value: /mnt/envd
    - name: __IGNORE_RESOURCE__
      value: "true"
    image: registry-<region-id>-vpc.ack.aliyuncs.com/acs/agent-runtime:v0.0.5
    imagePullPolicy: IfNotPresent
    name: init
    resources: {}
    restartPolicy: Always
    volumeMounts:
    - mountPath: /mnt/envd
      name: envd-volume
  - args:
    - --endpoint=unix://var/run/csi/sockets/driverplugin.csi.alibabacloud.com-replace/csi.sock
    - --driver=nas,oss
    - --v=1
    - --run-controller-service=false
    - --run-node-service=true
    - --feature-gates=AlinasMountProxy=true
    env:
    - name: __IGNORE_RESOURCE__
      value: "true"
    - name: KUBELET_ROOT_DIR
      value: /
    - name: ALIBABA_CLOUD_NETWORK_TYPE
      value: vpc
    - name: REGION_ID
      value: cn-hangzhou
    - name: OSS_SKIP_GLOBAL_MOUNT
      value: "true"
    - name: KUBE_NODE_NAME
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: spec.nodeName
    image: registry-<region-id>-vpc.ack.aliyuncs.com/acs/csi-plugin:v1.35.1-2592a4872
    imagePullPolicy: IfNotPresent
    name: csi-sidecar
    resources:
      limits:
        cpu: 500m
        memory: 1Gi
      requests:
        cpu: 100m
        memory: 128Mi
    restartPolicy: Always
    securityContext:
      privileged: true
    volumeMounts:
    - mountPath: /run/csi/mount-root
      mountPropagation: Bidirectional
      name: mount-root
    - mountPath: /var/run/csi/sockets/nasplugin.csi.alibabacloud.com
      name: nas-plugin-dir
    - mountPath: /var/run/csi/sockets/ossplugin.csi.alibabacloud.com
      name: oss-plugin-dir
    - mountPath: /run/cnfs
      name: run-cnfs
    - mountPath: /host/var/run/efc
      name: efc-metrics-dir
    - mountPath: /host/var/run/ossfs
      name: ossfs-metrics-dir
  - args:
    - --socket=/run/cnfs/alinas-mounter.sock
    - --v=4
    env:
    - name: __IGNORE_RESOURCE__
      value: "true"
    image: registry-<region-id>-vpc.ack.aliyuncs.com/acs/csi-agent:v1.35.3-cgroupv1-dport-forbidden
    imagePullPolicy: IfNotPresent
    name: csi-agent-sidecar
    resources:
      limits:
        cpu: 500m
        memory: 1Gi
      requests:
        cpu: 500m
        memory: 1Gi
    restartPolicy: Always
    securityContext:
      privileged: true
    volumeMounts:
    - mountPath: /run/csi/mount-root
      mountPropagation: Bidirectional
      name: mount-root
    - mountPath: /sys/fs/cgroup/net_cls/kubepods
      name: cgroup-net-cls
    - mountPath: /etc/aliyun-defaults/cpfs
      name: csi-agent-config
    - mountPath: /etc/aliyun-defaults/alinas
      name: csi-agent-config
    - mountPath: /run/cnfs
      name: run-cnfs
  # --- End of automatically injected configurations ---
  restartPolicy: Always
  volumes:
  # --- The following configurations are automatically injected ---
  - emptyDir: {}
    name: envd-volume
  - hostPath:
      path: /dev/fuse
      type: CharDevice
    name: fuse-device
  - hostPath:
      path: /sys/fs/cgroup/net_cls/kubepods
      type: Directory
    name: cgroup-net-cls
  - hostPath:
      path: /var/run/csi
      type: DirectoryOrCreate
    name: mount-root
  - emptyDir: {}
    name: nas-plugin-dir
  - emptyDir: {}
    name: oss-plugin-dir
  - emptyDir: {}
    name: run-cnfs
  - emptyDir: {}
    name: efc-metrics-dir
  - emptyDir: {}
    name: ossfs-metrics-dir
  - emptyDir: {}
    name: csi-agent-config
  # --- End of automatically injected configurations ---

Injection configuration

The following default injection configurations apply to v0.5.12 and later. You typically do not need to modify them. The <region-id> placeholder in image addresses is automatically replaced with your cluster's region ID, such as cn-zhangjiakou.

Default injection configuration for agent-runtime

{
  "mainContainer": {
    "name": "",
    "env": [
      {
        "name": "ENVD_DIR",
        "value": "/mnt/envd"
      },
      {
        "name": "GODEBUG",
        "value": "multipathtcp=0"
      },
      {
        "name": "POD_UID",
        "valueFrom": {
          "fieldRef": {
            "fieldPath": "metadata.uid"
          }
        }
      }
    ],
    "resources": {},
    "volumeMounts": [
      {
        "name": "envd-volume",
        "mountPath": "/mnt/envd"
      }
    ],
    "lifecycle": {
      "postStart": {
        "exec": {
          "command": [
            "bash",
            "-c",
            "/mnt/envd/envd-run.sh"
          ]
        }
      }
    }
  },
  "csiSidecar": [
    {
      "name": "init",
      "image": "registry-<region-id>-vpc.ack.aliyuncs.com/acs/agent-runtime:v0.0.5",
      "command": [
        "sh",
        "/workspace/entrypoint_inner.sh"
      ],
      "env": [
        {
          "name": "ENVD_DIR",
          "value": "/mnt/envd"
        },
        {
          "name": "__IGNORE_RESOURCE__",
          "value": "true"
        }
      ],
      "resources": {},
      "restartPolicy": "Always",
      "volumeMounts": [
        {
          "name": "envd-volume",
          "mountPath": "/mnt/envd"
        }
      ],
      "imagePullPolicy": "IfNotPresent"
    }
  ],
  "volume": [
    {
      "name": "envd-volume",
      "emptyDir": {}
    }
  ]
}

Default injection configuration for csi

{
  "mainContainer": {
    "name": "",
    "resources": {},
    "volumeMounts": [
      {
        "name": "mount-root",
        "mountPath": "/run/csi/mount-root",
        "mountPropagation": "HostToContainer"
      },
      {
        "name": "nas-plugin-dir",
        "mountPath": "/var/run/csi/sockets/nasplugin.csi.alibabacloud.com"
      },
      {
        "name": "oss-plugin-dir",
        "mountPath": "/var/run/csi/sockets/ossplugin.csi.alibabacloud.com"
      }
    ]
  },
  "csiSidecar": [
    {
      "name": "csi-sidecar",
      "image": "registry-<region-id>-vpc.ack.aliyuncs.com/acs/csi-plugin:v1.35.1-2592a4872",
      "args": [
        "--endpoint=unix://var/run/csi/sockets/driverplugin.csi.alibabacloud.com-replace/csi.sock",
        "--driver=nas,oss",
        "--v=1",
        "--run-controller-service=false",
        "--run-node-service=true",
        "--feature-gates=AlinasMountProxy=true"
      ],
      "env": [
        {
          "name": "__IGNORE_RESOURCE__",
          "value": "true"
        },
        {
          "name": "KUBELET_ROOT_DIR",
          "value": "/"
        },
        {
          "name": "ALIBABA_CLOUD_NETWORK_TYPE",
          "value": "vpc"
        },
        {
          "name": "REGION_ID",
          "value": "cn-hangzhou"
        },
        {
          "name": "OSS_SKIP_GLOBAL_MOUNT",
          "value": "true"
        },
        {
          "name": "KUBE_NODE_NAME",
          "valueFrom": {
            "fieldRef": {
              "apiVersion": "v1",
              "fieldPath": "spec.nodeName"
            }
          }
        }
      ],
      "resources": {
        "limits": {
          "cpu": "500m",
          "memory": "1Gi"
        },
        "requests": {
          "cpu": "100m",
          "memory": "128Mi"
        }
      },
      "restartPolicy": "Always",
      "volumeMounts": [
        {
          "name": "mount-root",
          "mountPath": "/run/csi/mount-root",
          "mountPropagation": "Bidirectional"
        },
        {
          "name": "nas-plugin-dir",
          "mountPath": "/var/run/csi/sockets/nasplugin.csi.alibabacloud.com"
        },
        {
          "name": "oss-plugin-dir",
          "mountPath": "/var/run/csi/sockets/ossplugin.csi.alibabacloud.com"
        },
        {
          "name": "run-cnfs",
          "mountPath": "/run/cnfs"
        },
        {
          "name": "efc-metrics-dir",
          "mountPath": "/host/var/run/efc"
        },
        {
          "name": "ossfs-metrics-dir",
          "mountPath": "/host/var/run/ossfs"
        }
      ],
      "imagePullPolicy": "IfNotPresent",
      "securityContext": {
        "privileged": true
      }
    },
    {
      "name": "csi-agent-sidecar",
      "image": "registry-<region-id>-vpc.ack.aliyuncs.com/acs/csi-agent:v1.35.3-cgroupv1-dport-forbidden",
      "args": [
        "--socket=/run/cnfs/alinas-mounter.sock",
        "--v=4"
      ],
      "env": [
        {
          "name": "__IGNORE_RESOURCE__",
          "value": "true"
        }
      ],
      "resources": {
        "limits": {
          "cpu": "500m",
          "memory": "1Gi"
        },
        "requests": {
          "cpu": "500m",
          "memory": "1Gi"
        }
      },
      "restartPolicy": "Always",
      "volumeMounts": [
        {
          "name": "mount-root",
          "mountPath": "/run/csi/mount-root",
          "mountPropagation": "Bidirectional"
        },
        {
          "name": "cgroup-net-cls",
          "mountPath": "/sys/fs/cgroup/net_cls/kubepods"
        },
        {
          "name": "csi-agent-config",
          "mountPath": "/etc/aliyun-defaults/cpfs"
        },
        {
          "name": "csi-agent-config",
          "mountPath": "/etc/aliyun-defaults/alinas"
        },
        {
          "name": "run-cnfs",
          "mountPath": "/run/cnfs"
        }
      ],
      "imagePullPolicy": "IfNotPresent",
      "securityContext": {
        "privileged": true
      }
    }
  ],
  "volume": [
    {
      "name": "fuse-device",
      "hostPath": {
        "path": "/dev/fuse",
        "type": "CharDevice"
      }
    },
    {
      "name": "cgroup-net-cls",
      "hostPath": {
        "path": "/sys/fs/cgroup/net_cls/kubepods",
        "type": "Directory"
      }
    },
    {
      "name": "mount-root",
      "hostPath": {
        "path": "/var/run/csi",
        "type": "DirectoryOrCreate"
      }
    },
    {
      "name": "nas-plugin-dir",
      "emptyDir": {}
    },
    {
      "name": "oss-plugin-dir",
      "emptyDir": {}
    },
    {
      "name": "run-cnfs",
      "emptyDir": {}
    },
    {
      "name": "efc-metrics-dir",
      "emptyDir": {}
    },
    {
      "name": "ossfs-metrics-dir",
      "emptyDir": {}
    },
    {
      "name": "csi-agent-config",
      "emptyDir": {}
    }
  ]
}

Each configuration item includes the following fields:

Field	Description
`mainContainer`	Configurations injected into the main container: environment variables (`env`), volume mounts (`volumeMounts`), and lifecycle hooks (`lifecycle`). Format: Kubernetes `corev1.Container`.
`csiSidecar`	Sidecar containers that inject CSI plugins or AgentRuntime init containers. Format: array of `corev1.Container`.
`volume`	Volume configurations injected at the Pod level. Format: array of `corev1.Volume`.