Deploy confidential containers with CAA

How it works

CAA (Cloud API Adaptor), also known as Peer Pods, is a core component of the CNCF Confidential Containers project. By calling cloud platform OpenAPI, CAA seamlessly integrates confidential computing capabilities into a Kubernetes cluster and automatically creates confidential virtual machines. These virtual machines act as special "confidential sandboxes" to run workload pods, using the underlying hardware's Trusted Execution Environment (TEE) technology to protect the workload's runtime data. The advantages are as follows.

• Easy to use: You do not need to manage the underlying bare-metal architecture or nested virtualization stacks. After you connect to an ACK cluster, you can use kubectl commands to deploy confidential computing workloads.

• Secure and confidential: Workloads run in ECS confidential virtual machines, where hardware-level memory encryption and isolation mechanisms ensure the integrity and confidentiality of runtime data.

• Open-source compatibility: ACK collaborates with the Confidential Containers community to deploy confidential computing workloads in clusters based on the CAA solution, and accepts continuous auditing from the community through code transparency (Code Transparency).

You can deploy the CAA DaemonSet (official version v0.19.0) in an ACK cluster and then deploy confidential computing workloads on top of it. The workloads are created as pods and run inside confidential virtual machines. This ensures runtime data security throughout the container lifecycle and helps prevent attacks from outside the virtual machine. The following diagram shows the process.

Prerequisites

• The TDX confidential computing environment has some known limitations. Ensure you understand these limitations before you begin.

• You have created an ACK managed Pro cluster that meets the following requirements. For more information, see Create an ACK managed cluster.

  Parameter	Description
  Region and zone	Only zone I in the China (Beijing) region and zone B in the Singapore region are supported. To create a vSwitch, see Create and manage vSwitches.
  Kubernetes version	1.34 or later. To upgrade your cluster, see Manually upgrade a cluster.
  Network plug-in	Flannel.
  Configure SNAT for VPC	Enable this feature to allow cluster access to the internet. You can also enable this feature for an existing cluster. For more information, see Enable internet access for a cluster.
  RRSA OIDC	Enable this feature. RAM Roles for Service Accounts (RRSA) provides pod-level isolation for OpenAPI permissions, enabling fine-grained control over cloud resource access and reducing security risks. You can also enable this feature for an existing cluster. For more information, see Isolate pod permissions by using RAM Roles for Service Accounts (RRSA). After you enable this feature, see Obtain the URL and ARN of the OIDC provider to obtain the provider ARN. This ARN is required for authentication.

Step 1: Configure RRSA

To allow CAA to manage confidential virtual machines for confidential containers, you must configure a RAM role for identity and a RAM policy for permissions. This section describes how to grant the Cloud API Adaptor ServiceAccount permissions to manage ECS and VPC resources. This enables fine-grained, pod-level permission control.

Step 2: Create a node pool and configure security group

Create a node pool to deploy the controller workloads for the CAA solution. You also need to configure security group rules for the node pool to open the specific ports that CAA requires.

This node pool is used only to run the CAA control plane components and does not host workload pods. The node pool does not require confidential computing instance types. You can use regular nodes.

1. Log on to the ACK console. In the left navigation pane, click Clusters.

2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Nodes > Node Pools.

3. Click Create Node Pool and configure the parameters as prompted.

   The following table describes only the key parameters. For more information, see Create and manage a node pool.

   Parameter	Description
   Managed node pool	Select Disable.
   vSwitch	Select a vSwitch in a supported zone. The node pool adds nodes in this zone.
   Operating system	Ubuntu 24.04 64-bit.
   Expected nodes	The initial number of nodes in the node pool. Set this to 1 or more.
   The following are advanced configurations. Scroll down and expand Advanced Options (Optional).
   Node labels	Add the following node label to facilitate the scheduling of the CAA controller. Key: node.kubernetes.io/worker Value: Leave this field empty.

4. In the node pool list, click the node pool name, and then click the Overview tab. In the Node Pool Information area, click the security group ID to go to the security group details page.

5. On the Inbound tab, click Add Rule and add the following two rules.

   Protocol	Source	Destination	Description
   Custom TCP	The CIDR block of the current VPC	15150	Allows secure communication between CAA components and confidential virtual machines.
   Custom UDP	The CIDR block of the current VPC	4789	Enables network management for confidential containers based on VXLAN.

Step 4: Deploy a sample application

This example deploys a sample application that uses runtimeClassName: kata-remote as the confidential container runtime. When the Pod is scheduled, Kata Remote triggers CAA to dynamically create a TDX confidential virtual machine.

1. Save the following content as pod-caa-demo.yaml. This file is used to deploy a pod that uses the confidential container runtime.

   apiVersion: v1
   kind: Pod
   metadata:
     name: pod-caa-demo
   spec:
     runtimeClassName: kata-remote
     containers:
       - image: alibaba-cloud-linux-3-registry.cn-hangzhou.cr.aliyuncs.com/alinux3/alinux3:latest
         name: hello
         command:
         - sh
         - -c
         - 'echo hello && sleep infinity'

2. Deploy the pod-caa-demo.yaml file.

   kubectl apply -f pod-caa-demo.yaml

   Wait about 3 minutes, and then check whether the application is deployed.

   kubectl get pod pod-caa-demo

   Expected output:

   NAME           READY   STATUS    RESTARTS   AGE
   pod-caa-demo   1/1     Running   0          52s

3. Go to the ECS console, click Instances in the left navigation bar, and view the TDX confidential virtual machines that start with podvm-. This indicates that CAA has successfully created the underlying compute resources.

FAQ

References

• Confidential Containers

• Cloud API Adaptor

Contact us

To ask product questions or provide suggestions, join our DingTalk group (ID: 30521601).