为Agent Sandbox配置Sidecar容器自动注入-容器计算服务(ACS)-阿里云帮助中心

为简化Agent Sandbox的存储挂载及运行时注入的YAML配置，可通过在Sandbox资源中配置 spec.runtimes 字段，实现自动注入 CSI 存储挂载和 AgentRuntime 相关配置，来简化整个部署流程。

准备工作

已完成Agent Sandbox环境搭建，详情请参见创建Agent Sandbox。
在集群组件管理中，确认ack-agent-sandbox-controller组件版本为v0.5.12及以上。
安装组件时，在sandbox-system命名空间下会自动创建名为sandbox-injection-config的ConfigMap，包含agent-runtime和csi两个配置项。
如需特殊定制或配置修改，请在修改前联系阿里云技术支持评估配置的完备性。

功能概述

在Agent Sandbox中配置动态存储挂载时，由于涉及手动配置CSI及Sidecar容器，YAML配置一般较为复杂。通过Sidecar自动注入功能，您只需定义业务容器，并在SandboxSet或Sandbox资源中声明spec.runtimes字段，系统将自动为新建的Sandbox注入所需的CSI和AgentRuntime Sidecar配置。目前支持以下两种注入类型：

csi：注入CSI存储挂载相关的init容器和Volume配置，支持NAS、OSS等共享存储挂载能力。
启用动态存储挂载功能，需要为容器开放特权容器 (Privileged Container) 和宿主机路径 (hostPath，/var/run/csi) 的容器安全验证，可以提交工单放开限制，但由此带来的安全风险需要用户承担一定责任，相关机制请参见安全责任共担模型。
agent-runtime：注入AgentRuntime init容器（如envd等环境管理工具），并为业务容器添加相应的环境变量和生命周期钩子。

Sidecar自动注入仅对新创建的Sandbox实例生效，存量实例不受影响。如需使用Command 和Filesystem 接口，必须注入agent-runtime。

配置Sidecar注入

在SandboxSet或Sandbox资源的spec.runtimes字段中声明需要注入的运行时类型。以下分别展示两种资源的配置示例。

SandboxSet资源配置示例

apiVersion: agents.kruise.io/v1alpha1
kind: SandboxSet
metadata:
  name: code-interpreter-inject-test
  namespace: default
spec:
  runtimes:
  - name: csi           # 支持CSI挂载能力，新建的Sandbox会被注入对应的Sidecar
  - name: agent-runtime # 注入envd等环境管理工具
  replicas: 4
  template:
    metadata:
      labels:
        alibabacloud.com/acs: "true"
        alibabacloud.com/compute-class: agent-sandbox # Agent Sandbox 实例类型 
        alibabacloud.com/compute-qos: default # 算力质量 default/best-effort
    spec:
      automountServiceAccountToken: false
      containers:
      - image: registry-cn-zhangjiakou-vpc.ack.aliyuncs.com/acs/code-interpreter:v1.6 # 替换成实际集群所在的地域
        imagePullPolicy: IfNotPresent
        name: sandbox
        resources:
          limits:
            cpu: "1"
            memory: 1Gi
          requests:
            cpu: "1"
            memory: 1Gi
      terminationGracePeriodSeconds: 30

Sandbox资源配置示例

apiVersion: agents.kruise.io/v1alpha1
kind: Sandbox
metadata:
  name: code-interpreter-inject-test-xxx
  namespace: default
spec:
  runtimes:
  - name: csi           # 提供CSI挂载能力
  - name: agent-runtime # 注入envd等环境管理工具
  ...

注入后的Pod效果示例

# 注意：镜像地址中的<region-id>会根据集群所在地域自动替换
apiVersion: v1
kind: Pod
metadata:
  annotations:
    agents.kruise.io/created-by: sandbox
  labels:
    agents.kruise.io/sandbox-pool: code-interpreter-init-xxx
    alibabacloud.com/acs: "true"
  name: code-interpreter-init-xxx
  namespace: default
spec:
  automountServiceAccountToken: false
  containers:
  - env:
    # --- 以下为自动注入的配置 ---
    - name: ENVD_DIR
      value: /mnt/envd
    - name: GODEBUG
      value: multipathtcp=0
    - name: POD_UID
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: metadata.uid
    # --- 自动注入配置结束 ---
    image: example:tag
    imagePullPolicy: IfNotPresent
    # --- 以下为自动注入的配置 ---
    lifecycle:
      postStart:
        exec:
          command:
          - bash
          - -c
          - /mnt/envd/envd-run.sh
    # --- 自动注入配置结束 ---
    name: sandbox
    resources:
      limits:
        cpu: "2"
        memory: 2Gi
      requests:
        cpu: "2"
        memory: 2Gi
    volumeMounts:
    # --- 以下为自动注入的配置 ---
    - mountPath: /mnt/envd
      name: envd-volume
    - mountPath: /run/csi/mount-root
      mountPropagation: HostToContainer
      name: mount-root
    - mountPath: /var/run/csi/sockets/nasplugin.csi.alibabacloud.com
      name: nas-plugin-dir
    - mountPath: /var/run/csi/sockets/ossplugin.csi.alibabacloud.com
      name: oss-plugin-dir
    # --- 自动注入配置结束 ---
  # --- 以下为自动注入的配置 ---
  initContainers:
  - command:
    - sh
    - /workspace/entrypoint_inner.sh
    env:
    - name: ENVD_DIR
      value: /mnt/envd
    - name: __IGNORE_RESOURCE__
      value: "true"
    image: registry-<region-id>-vpc.ack.aliyuncs.com/acs/agent-runtime:v0.0.5
    imagePullPolicy: IfNotPresent
    name: init
    resources: {}
    restartPolicy: Always
    volumeMounts:
    - mountPath: /mnt/envd
      name: envd-volume
  - args:
    - --endpoint=unix://var/run/csi/sockets/driverplugin.csi.alibabacloud.com-replace/csi.sock
    - --driver=nas,oss
    - --v=1
    - --run-controller-service=false
    - --run-node-service=true
    - --feature-gates=AlinasMountProxy=true
    env:
    - name: __IGNORE_RESOURCE__
      value: "true"
    - name: KUBELET_ROOT_DIR
      value: /
    - name: ALIBABA_CLOUD_NETWORK_TYPE
      value: vpc
    - name: REGION_ID
      value: cn-hangzhou
    - name: OSS_SKIP_GLOBAL_MOUNT
      value: "true"
    - name: KUBE_NODE_NAME
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: spec.nodeName
    image: registry-<region-id>-vpc.ack.aliyuncs.com/acs/csi-plugin:v1.35.1-2592a4872
    imagePullPolicy: IfNotPresent
    name: csi-sidecar
    resources:
      limits:
        cpu: 500m
        memory: 1Gi
      requests:
        cpu: 100m
        memory: 128Mi
    restartPolicy: Always
    securityContext:
      privileged: true
    volumeMounts:
    - mountPath: /run/csi/mount-root
      mountPropagation: Bidirectional
      name: mount-root
    - mountPath: /var/run/csi/sockets/nasplugin.csi.alibabacloud.com
      name: nas-plugin-dir
    - mountPath: /var/run/csi/sockets/ossplugin.csi.alibabacloud.com
      name: oss-plugin-dir
    - mountPath: /run/cnfs
      name: run-cnfs
    - mountPath: /host/var/run/efc
      name: efc-metrics-dir
    - mountPath: /host/var/run/ossfs
      name: ossfs-metrics-dir
  - args:
    - --socket=/run/cnfs/alinas-mounter.sock
    - --v=4
    env:
    - name: __IGNORE_RESOURCE__
      value: "true"
    image: registry-<region-id>-vpc.ack.aliyuncs.com/acs/csi-agent:v1.35.3-cgroupv1-dport-forbidden
    imagePullPolicy: IfNotPresent
    name: csi-agent-sidecar
    resources:
      limits:
        cpu: 500m
        memory: 1Gi
      requests:
        cpu: 500m
        memory: 1Gi
    restartPolicy: Always
    securityContext:
      privileged: true
    volumeMounts:
    - mountPath: /run/csi/mount-root
      mountPropagation: Bidirectional
      name: mount-root
    - mountPath: /sys/fs/cgroup/net_cls/kubepods
      name: cgroup-net-cls
    - mountPath: /etc/aliyun-defaults/cpfs
      name: csi-agent-config
    - mountPath: /etc/aliyun-defaults/alinas
      name: csi-agent-config
    - mountPath: /run/cnfs
      name: run-cnfs
  # --- 自动注入配置结束 ---
  restartPolicy: Always
  volumes:
  # --- 以下为自动注入的配置 ---
  - emptyDir: {}
    name: envd-volume
  - hostPath:
      path: /dev/fuse
      type: CharDevice
    name: fuse-device
  - hostPath:
      path: /sys/fs/cgroup/net_cls/kubepods
      type: Directory
    name: cgroup-net-cls
  - hostPath:
      path: /var/run/csi
      type: DirectoryOrCreate
    name: mount-root
  - emptyDir: {}
    name: nas-plugin-dir
  - emptyDir: {}
    name: oss-plugin-dir
  - emptyDir: {}
    name: run-cnfs
  - emptyDir: {}
    name: efc-metrics-dir
  - emptyDir: {}
    name: ossfs-metrics-dir
  - emptyDir: {}
    name: csi-agent-config
  # --- 自动注入配置结束 ---

注入配置说明

以下为v0.5.11及以上版本默认提供的注入配置，通常无需修改。其中镜像地址中的<region-id>会根据集群所在地域自动替换（如cn-zhangjiakou）。

agent-runtime默认注入配置

{
  "mainContainer": {
    "name": "",
    "env": [
      {
        "name": "ENVD_DIR",
        "value": "/mnt/envd"
      },
      {
        "name": "GODEBUG",
        "value": "multipathtcp=0"
      },
      {
        "name": "POD_UID",
        "valueFrom": {
          "fieldRef": {
            "fieldPath": "metadata.uid"
          }
        }
      }
    ],
    "resources": {},
    "volumeMounts": [
      {
        "name": "envd-volume",
        "mountPath": "/mnt/envd"
      }
    ],
    "lifecycle": {
      "postStart": {
        "exec": {
          "command": [
            "bash",
            "-c",
            "/mnt/envd/envd-run.sh"
          ]
        }
      }
    }
  },
  "csiSidecar": [
    {
      "name": "init",
      "image": "registry-<region-id>-vpc.ack.aliyuncs.com/acs/agent-runtime:v0.0.5",
      "command": [
        "sh",
        "/workspace/entrypoint_inner.sh"
      ],
      "env": [
        {
          "name": "ENVD_DIR",
          "value": "/mnt/envd"
        },
        {
          "name": "__IGNORE_RESOURCE__",
          "value": "true"
        }
      ],
      "resources": {},
      "restartPolicy": "Always",
      "volumeMounts": [
        {
          "name": "envd-volume",
          "mountPath": "/mnt/envd"
        }
      ],
      "imagePullPolicy": "IfNotPresent"
    }
  ],
  "volume": [
    {
      "name": "envd-volume",
      "emptyDir": {}
    }
  ]
}

csi默认注入配置

{
  "mainContainer": {
    "name": "",
    "resources": {},
    "volumeMounts": [
      {
        "name": "mount-root",
        "mountPath": "/run/csi/mount-root",
        "mountPropagation": "HostToContainer"
      },
      {
        "name": "nas-plugin-dir",
        "mountPath": "/var/run/csi/sockets/nasplugin.csi.alibabacloud.com"
      },
      {
        "name": "oss-plugin-dir",
        "mountPath": "/var/run/csi/sockets/ossplugin.csi.alibabacloud.com"
      }
    ]
  },
  "csiSidecar": [
    {
      "name": "csi-sidecar",
      "image": "registry-<region-id>-vpc.ack.aliyuncs.com/acs/csi-plugin:v1.35.1-2592a4872",
      "args": [
        "--endpoint=unix://var/run/csi/sockets/driverplugin.csi.alibabacloud.com-replace/csi.sock",
        "--driver=nas,oss",
        "--v=1",
        "--run-controller-service=false",
        "--run-node-service=true",
        "--feature-gates=AlinasMountProxy=true"
      ],
      "env": [
        {
          "name": "__IGNORE_RESOURCE__",
          "value": "true"
        },
        {
          "name": "KUBELET_ROOT_DIR",
          "value": "/"
        },
        {
          "name": "ALIBABA_CLOUD_NETWORK_TYPE",
          "value": "vpc"
        },
        {
          "name": "REGION_ID",
          "value": "cn-hangzhou"
        },
        {
          "name": "OSS_SKIP_GLOBAL_MOUNT",
          "value": "true"
        },
        {
          "name": "KUBE_NODE_NAME",
          "valueFrom": {
            "fieldRef": {
              "apiVersion": "v1",
              "fieldPath": "spec.nodeName"
            }
          }
        }
      ],
      "resources": {
        "limits": {
          "cpu": "500m",
          "memory": "1Gi"
        },
        "requests": {
          "cpu": "100m",
          "memory": "128Mi"
        }
      },
      "restartPolicy": "Always",
      "volumeMounts": [
        {
          "name": "mount-root",
          "mountPath": "/run/csi/mount-root",
          "mountPropagation": "Bidirectional"
        },
        {
          "name": "nas-plugin-dir",
          "mountPath": "/var/run/csi/sockets/nasplugin.csi.alibabacloud.com"
        },
        {
          "name": "oss-plugin-dir",
          "mountPath": "/var/run/csi/sockets/ossplugin.csi.alibabacloud.com"
        },
        {
          "name": "run-cnfs",
          "mountPath": "/run/cnfs"
        },
        {
          "name": "efc-metrics-dir",
          "mountPath": "/host/var/run/efc"
        },
        {
          "name": "ossfs-metrics-dir",
          "mountPath": "/host/var/run/ossfs"
        }
      ],
      "imagePullPolicy": "IfNotPresent",
      "securityContext": {
        "privileged": true
      }
    },
    {
      "name": "csi-agent-sidecar",
      "image": "registry-<region-id>-vpc.ack.aliyuncs.com/acs/csi-agent:v1.35.3-cgroupv1-dport-forbidden",
      "args": [
        "--socket=/run/cnfs/alinas-mounter.sock",
        "--v=4"
      ],
      "env": [
        {
          "name": "__IGNORE_RESOURCE__",
          "value": "true"
        }
      ],
      "resources": {
        "limits": {
          "cpu": "500m",
          "memory": "1Gi"
        },
        "requests": {
          "cpu": "500m",
          "memory": "1Gi"
        }
      },
      "restartPolicy": "Always",
      "volumeMounts": [
        {
          "name": "mount-root",
          "mountPath": "/run/csi/mount-root",
          "mountPropagation": "Bidirectional"
        },
        {
          "name": "cgroup-net-cls",
          "mountPath": "/sys/fs/cgroup/net_cls/kubepods"
        },
        {
          "name": "csi-agent-config",
          "mountPath": "/etc/aliyun-defaults/cpfs"
        },
        {
          "name": "csi-agent-config",
          "mountPath": "/etc/aliyun-defaults/alinas"
        },
        {
          "name": "run-cnfs",
          "mountPath": "/run/cnfs"
        }
      ],
      "imagePullPolicy": "IfNotPresent",
      "securityContext": {
        "privileged": true
      }
    }
  ],
  "volume": [
    {
      "name": "fuse-device",
      "hostPath": {
        "path": "/dev/fuse",
        "type": "CharDevice"
      }
    },
    {
      "name": "cgroup-net-cls",
      "hostPath": {
        "path": "/sys/fs/cgroup/net_cls/kubepods",
        "type": "Directory"
      }
    },
    {
      "name": "mount-root",
      "hostPath": {
        "path": "/var/run/csi",
        "type": "DirectoryOrCreate"
      }
    },
    {
      "name": "nas-plugin-dir",
      "emptyDir": {}
    },
    {
      "name": "oss-plugin-dir",
      "emptyDir": {}
    },
    {
      "name": "run-cnfs",
      "emptyDir": {}
    },
    {
      "name": "efc-metrics-dir",
      "emptyDir": {}
    },
    {
      "name": "ossfs-metrics-dir",
      "emptyDir": {}
    },
    {
      "name": "csi-agent-config",
      "emptyDir": {}
    }
  ]
}

每个配置项的数据结构包含以下字段：

字段	说明
`mainContainer`	主容器注入配置，包括环境变量（`env`）、卷挂载（`volumeMounts`）和生命周期钩子（`lifecycle`）等。格式与Kubernetes `corev1.Container`一致。
`csiSidecar`	Sidecar容器列表，用于注入CSI插件容器或AgentRuntime初始化容器。格式为`corev1.Container`数组。
`volume`	Volume配置列表，定义注入到Pod级别的卷。格式为`corev1.Volume`数组。