Deploy confidential containers with CAA

1. Create a RAM role

Refer to Create a RAM role for a trusted identity provider to create a RAM role. In this example, the RAM role is named ack-caa-demo, and the key parameters are as follows:

After you create the role, go to its details page. In the Basic Information section, find and record its ARN. This ARN is required for authentication.

2. Create a RAM policy

1. See the script below to create a RAM policy named ack-caa-policy that grants permissions for ECS and VPC operations. For the procedure, see Create a custom permission policy.

   {
     "Version": "1",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "ecs:RunInstances",
           "ecs:DeleteInstance",
           "ecs:DescribeInstanceAttribute",
           "ecs:CreateNetworkInterface",
           "ecs:DeleteNetworkInterface",
           "ecs:AttachNetworkInterface",
           "ecs:ModifyNetworkInterfaceAttribute",
           "ecs:DescribeNetworkInterfaceAttribute"
         ],
         "Resource": "*"
       },
       {
         "Effect": "Allow",
         "Action": [
           "vpc:DescribeVSwitchAttributes",
           "vpc:AllocateEipAddress",
           "vpc:ReleaseEipAddress",
           "vpc:AssociateEipAddress",
           "vpc:UnassociateEipAddress",
           "vpc:DescribeEipAddresses"
         ],
         "Resource": "*"
       }
     ]
   }

2. Grant the permission policy ack-caa-policy to the previously created RAM role. For instructions, see Manage permissions for a RAM role.

Step 3: Deploy CAA using a Helm chart

Deploy the following core components of CAA:

• Cert Manager: A prerequisite for CAA deployment. It handles certificate validation for components such as the CAA webhook.

• CAA (Cloud API Adaptor): Includes components such as the Kata Remote Runtime and the CAA node DaemonSet. Based on pod scheduling requests, they dynamically create confidential virtual machines to serve as runtime nodes for pods.

mkdir cert-manager
cat << EOF > cert-manager/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - https://github.com/jetstack/cert-manager/releases/download/v1.15.3/cert-manager.yaml
images:
  - name: quay.io/jetstack/cert-manager-controller
    newName: registry-cn-hangzhou.ack.aliyuncs.com/dev/cert-manager-controller
  - name: quay.io/jetstack/cert-manager-webhook
    newName: registry-cn-hangzhou.ack.aliyuncs.com/dev/cert-manager-webhook
  - name: quay.io/jetstack/cert-manager-cainjector
    newName: registry-cn-hangzhou.ack.aliyuncs.com/dev/cert-manager-cainjector
EOF
kubectl apply -k cert-manager

mkdir cert-manager
cat << EOF > cert-manager/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - https://github.com/jetstack/cert-manager/releases/download/v1.15.3/cert-manager.yaml
EOF
kubectl apply -k cert-manager

2. Deploy Cloud API Adaptor

1. Create a Helm-managed namespace.

   kubectl apply -f - << EOF
   apiVersion: v1
   kind: Namespace
   metadata:
     name: confidential-containers-system
     labels:
       app.kubernetes.io/managed-by: Helm
     annotations:
       meta.helm.sh/release-name: peerpods
       meta.helm.sh/release-namespace: confidential-containers-system
   EOF

2. Download the project source code.

   git clone https://github.com/confidential-containers/cloud-api-adaptor.git -b v0.22.0
   cd cloud-api-adaptor/src/cloud-api-adaptor/install/charts/peerpods

3. Modify the relevant configurations in the providers/alibabacloud.yaml file.

   Parameter	Description
   providerConfigs.alibabacloud.SECURITY_GROUP_IDS	The ID of the security group for the virtual machine. Set this to the security group ID of the node pool that you created in Step 2.
   providerConfigs.alibabacloud.VSWITCH_ID	The vSwitch ID of the virtual machine. Set this to the vSwitch selected for the node pool created in Step 2.
   providerConfigs.alibabacloud.REGION	The region where the cluster is located. For example, cn-beijing.
   providerConfigs.alibabacloud.IMAGEID	The boot image for the virtual machine. Set this parameter to the image ID of the region where your cluster is deployed. For more information, see the following table.

   The following table lists the image ID for each supported region.

   Region	Region ID	Image ID
   China (Beijing)	cn-beijing	m-2ze2vxvrxsbue3sf8b02
   China (Hangzhou)	cn-hangzhou	m-bp146ws6x3iyuwjvbdw2
   China (Hong Kong)	cn-hongkong	m-j6c3kfz2vhu6ze04wacb
   Singapore	ap-southeast-1	m-t4ng1w8ipua3c0o57hor

4. Create a cluster Secret. The CAA DaemonSet uses this secret to make Alibaba Cloud API calls to create TDX instances.

   Replace ${ALIBABA_CLOUD_ROLE_ARN} with the RAM role ARN, and ${ALIBABA_CLOUD_OIDC_PROVIDER_ARN} with the RRSA OIDC provider ARN.

   kubectl create secret generic my-provider-creds \
       -n confidential-containers-system \
       --from-literal=ALIBABA_CLOUD_ROLE_ARN=${ALIBABA_CLOUD_ROLE_ARN} \
       --from-literal=ALIBABA_CLOUD_OIDC_PROVIDER_ARN=${ALIBABA_CLOUD_OIDC_PROVIDER_ARN} \
       --from-literal=ALIBABA_CLOUD_OIDC_TOKEN_FILE=/var/run/secrets/ack.alibabacloud.com/rrsa-tokens/token

5. Deploy the CAA workload.

   Chinese mainland

   Create a file named ./mirror-values.yaml with the following content to use the Alibaba Cloud image source.

   image:
     name: registry-cn-hangzhou.ack.aliyuncs.com/dev/coco-cloud-api-adaptor
     tag: v0.22.0
   resourceCtrl:
     image:
       repository: registry-cn-hangzhou.ack.aliyuncs.com/dev/peerpod-ctrl
       tag: v0.22.0
     authProxy:
       image: registry-cn-hangzhou.ack.aliyuncs.com/dev/kube-rbac-proxy:v0.14.0
   webhook:
     image:
       repository: registry-cn-hangzhou.ack.aliyuncs.com/dev/peer-pods-webhook
       tag: v0.22.0
     authProxy:
       image: registry-cn-hangzhou.ack.aliyuncs.com/dev/kube-rbac-proxy:v0.14.0
   kata-deploy:
     image:
       reference: confidential-ai-registry.cn-shanghai.cr.aliyuncs.com/product/kata-deploy
       tag: 4.0.0
     kubectlImage:
       reference: registry-cn-hangzhou.ack.aliyuncs.com/dev/kata-containers-kubectl
       tag: latest

   Run the following command to deploy CAA.

   helm dependency update
   helm install peerpods . \
       -f providers/alibabacloud.yaml \
       -f ./mirror-values.yaml \
       --set secrets.mode=reference \
       --set secrets.existingSecretName=my-provider-creds \
       --set alibabacloud.rrsa.enable=true \
       --dependency-update \
       -n confidential-containers-system

   China (Hong Kong) and overseas

   Run the following command to deploy CAA.

   helm dependency update
   helm install peerpods . \
       -f providers/alibabacloud.yaml \
       --set secrets.mode=reference \
       --set secrets.existingSecretName=my-provider-creds \
       --set alibabacloud.rrsa.enable=true \
       --dependency-update \
       -n confidential-containers-system

   Wait about 3 minutes, and then check the deployment status.

   kubectl -n confidential-containers-system get pod 

   Expected output:

   NAME                                             READY   STATUS    RESTARTS   AGE
   cloud-api-adaptor-daemonset-pnmpz                1/1     Running   0          20m
   cloud-api-adaptor-daemonset-qhlqk                1/1     Running   0          20m
   cloud-api-adaptor-daemonset-zpkt5                1/1     Running   0          20m
   kata-deploy-89x8h                                1/1     Running   0          20m
   kata-deploy-fcpb8                                1/1     Running   0          20m
   kata-deploy-jfkjg                                1/1     Running   0          20m
   peerpodctrl-controller-manager-bfdb466fb-xl8xc   2/2     Running   0          20m

   The expected output includes the kata-deploy-*, cloud-api-adaptor-daemonset-*, and peerpodctrl-controller-manager-* components, all with a status of Running, indicating a successful deployment.