Quick access from ACK

更新时间:
复制 MD 格式

When running containerized workloads on Container Service for Kubernetes (ACK), hardcoding AccessKeys in application code poses security risks and makes credential rotation difficult. KMS cloud-native access eliminates this risk by injecting a KMS Agent Sidecar into pods, allowing applications to securely retrieve credentials managed in KMS through local HTTP requests.

How it works

The Key Management Service (KMS) cloud-native access feature provides a guided workflow for component installation and configuration. When a pod in your Container Service for Kubernetes (ACK) cluster sends an HTTP request, the KMS Agent—automatically injected into the pod via the Helm component ack-kms-agent-webhook-injector—receives the request and authenticates using an OpenID Connect (OIDC) JWT or a Resource Access Management (RAM) role. After KMS verifies the permissions, it returns the credentials through the Agent. The Agent caches credentials locally to reduce repeated requests and lower credential retrieval latency.

image

Prerequisites

Install the KMS Agent

Enable during cluster creation

When creating an ACK managed cluster or ACK Edge cluster, go to the Cluster Configurations step, expand Advanced Options (Optional), and click Enable next to RRSA OIDC.

image

Enable for an existing cluster

  1. Log on to the ACK console. In the left navigation pane, click Clusters.

  2. Click the name of your cluster. In the left navigation pane, click Cluster Information.

  3. On the Basic Information tab, scroll to the Security and Auditing section and click Enable next to RRSA OIDC.

    image

  4. In the Enable RRSA dialog box, click Confirm. Wait for the cluster status to change from Updating to Running. RRSA is now enabled.

Step 1: Create a namespace and a service account (optional)

A namespace partitions your ACK cluster into logically isolated virtual spaces for different environments such as development, testing, and production. Applications in different namespaces cannot access each other's resources by default. If you already have a namespace and service account for your business application, skip this step.

  1. Create a namespace.

    1. Create a namespace by using a YAML file. The following example uses app1-namespace.yaml to create a namespace named app1-dev:

      apiVersion: v1
      kind: Namespace
      metadata:
        name: app1-dev
    2. Run the following command to create the namespace:

      kubectl apply -f app1-namespace.yaml
    3. Verify that the namespace is created. If the output includes app1-dev, the namespace is created.

      kubectl get namespaces
  2. Create a service account.

    1. Create a service account by using a YAML file. The following example uses app1-serviceaccount.yaml to create a service account named app1-service in the app1-dev namespace that you created in the previous step:

      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: app1-service
        namespace: app1-dev
    2. Run the following command to create the service account:

      kubectl apply -f app1-serviceaccount.yaml
    3. Verify that the service account is created. If the output includes app1-service, the service account is created.

      kubectl get serviceaccount -n app1-dev

Step 2: Configure permissions

KMS cloud-native access supports the following two authentication methods.

Note

OpenID Connect (OIDC) is recommended for most scenarios due to its simpler configuration. Use RAM role when you need to reuse existing RAM role policies or require cross-account access.

Authentication method

Features

OIDC (ACK)

The ACK cluster authenticates by using OpenID Connect (OIDC) standard JWT. The Agent automatically obtains the service account token to prove the pod identity to KMS. No RAM role is required, and the configuration is the simplest.

RAM role

Accesses KMS by using STS temporary credentials of a RAM role. This method is suitable for scenarios where you want to reuse an existing RAM role setup.

OIDC (ACK)

  1. Log on to the KMS console. In the left-side navigation pane, choose Application Access > Cloud-native Access. In the Elastic Computing section, find your ACK cluster and click Actions.

  2. In the configuration panel that appears, set Authentication Method to OIDC (ACK).

  3. Configure the following parameters:

    Parameter

    Description

    Namespace

    The name of the namespace in which the pod resides, such as app1-dev.

    ServiceAccount

    The service account used by the pod, such as app1-service.

    PodNamePrefix

    The prefix of the pod name. After you configure this parameter, only pods whose names match the prefix are allowed to pass verification. If you do not configure this parameter, only the namespace and service account are verified.

    Scope

    The method to access KMS. Valid values:

    • Specified KMS Instance: Accesses keys and credentials in a specific KMS instance by using the instance endpoint.

    • Shared KMS Gateway: Accesses credentials by using the KMS service endpoint.

    Application Access Point Name

    The custom name of the application access point (access credential) for identification and management.

    Policy Name

    The custom name of the RAM policy. Create a policy directly in this step without pre-creating it in the RAM console.

    RBAC Permissions

    The RBAC (Role-Based Access Control) permission level, which determines the credential operation permissions for the application.

    • If Specified KMS Instance is selected for the scope:

      • CryptoServiceKeyUser: Allows the use of keys in the KMS instance for credential encryption.

      • CryptoServiceSecretUser: Allows the use of credentials in the KMS instance and supports credential APIs for the instance.

    • If Shared KMS Gateway is selected for the scope: Only SecretUser is supported, which allows the use of all credentials under the current account.

    Accessible Resources

    Select the credentials and keys (used for credential encryption and decryption) that the application needs to access.

    Important

    If you select multiple credentials and the total length of credential names exceeds the limit, a "parameter invalid" error is returned. In this case, use a wildcard to specify allowed credentials, such as secret/rds-ibm*, which allows access to credentials with the prefix rds-ibm.

    Description

    Optional. A detailed description of the application access point. Maximum length: 8,192 characters.

  4. Click OK to proceed.

RAM role

  1. Obtain the identity provider information.

    1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

    2. Click the name of the target cluster to go to the details page.

    3. On the Basic Information tab, move the pointer over the Enabled label next to RRSA (RAM Roles for Service Accounts) OIDC in the Security and Auditing section to view the provider URL and ARN information.image

  2. Create a RAM role.

    1. Log on to the RAM console. In the left-side navigation pane, choose Identities > Roles, and then click Create Role.

    2. Select IdP as the trusted entity type, and then click Switch to Editor.

    3. In the Visual Editor section, configure the following items:

      • Basic configuration

        Parameter

        Description

        Effect

        Select Allow.

        Action

        Keep the default value sts:AssumeRole.

        Condition

        Add a condition with the key oidc:sub, the operator StringEquals, and the value system:serviceaccount:<namespace>:<ServiceAccountName>, where namespace and ServiceAccountName correspond to the namespace and service account of the pod running the workload.

      • Principal configuration:

        1. Select Identity Provider as the Principal, and then click Edit below.

        2. On the Identity Provider configuration page, configure the following parameters and click OK.

          Parameter

          Description

          IdP Type

          Select OIDC.

          Identity Provider

          Select the identity provider automatically created by the ACK cluster after RRSA is enabled: ack-rrsa-<cluster_id>.

    4. After the configuration is complete, click OK to set the role name (for example, app1-rrsa), and then click OK.

  3. Create a policy and attach it to the RAM role. For more information, see Create a custom policy and Attach a policy to a RAM role.

    1. In the left-side navigation pane, choose Permissions > Policies.

    2. Click Create Policy, select Script Editor, and then configure the policy by using the following example.

      Note

      In this example, the policy is named dev-role-for-rrsa-kms-policy and allows access only to credentials with the tag env:app1.

      {
          "Version": "1",
          "Statement": [
              {
                  "Effect": "Allow",
                  "Action": [
                      "kms:Decrypt",
                      "kms:GetSecretValue"
                  ],
                  "Resource": "*",
                  "Condition": {
                      "StringEqualsIgnoreCase": {
                          "kms:tag/secret": [
                              "app1"
                          ]
                      }
                  }
              }
          ]
      }
    3. Go back to the policy list, find the target policy, and click Attach to Identity in the Actions column.

    4. In the Principal section, select the RAM role you created, and then click Confirm.

Step 3: Install ack-kms-agent-webhook-injector

  1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

  2. On the Clusters page, click the name of the target cluster to go to the details page.

  3. In the left-side navigation pane of the details page, choose Applications > Helm.

  4. On the Helm page, click Deploy. Configure the Basic Information section, and then click Next.

    Parameter

    Description

    Application Name

    Use the default application name ack-kms-agent-webhook-injector.

    Namespace

    Use the default chart namespace kube-system. Install to one namespace per ACK cluster; no need to install multiple times.

    Source

    Default: Marketplace. This parameter cannot be modified.

    Chart

    Search for and select ack-kms-agent-webhook-injector.

  5. When the confirmation dialog appears, verify the information and click Yes.

  6. On the Parameters page, configure the parameters based on the authentication method you selected in Step 2.

    • OIDC (ACK): Keep the default configuration.

    • RAM role: Leave agent.auth.roleArn empty, and set agent.auth.roleArnMapping to <Namespace>:<ServiceAccountName>:<RAM Role ARN>. The following example uses the data created in Step 2:

      Note

      The Namespace and ServiceAccountName correspond to the namespace and service account of the pod running the workload. The RAM Role ARN can be viewed on the RAM role details page.

      agent:
        auth:
          roleArn:
          roleArnMapping:
            app1-dev:app1-service: acs:ram::190325303126****:role/app1-rrsa
  7. After the configuration is complete, click OK. You are redirected to the application details page.

Step 4: Inject the Agent Sidecar

  1. Add a pod annotation: Set the annotation key to kms-agent-webhook-injector/inject and the value to true.

    1. In the left-side navigation pane of the cluster details page, choose Workloads > Deployments.

    2. Switch to the namespace in which your workload runs, and add the pod annotation to the Deployment.

      Important

      For the RAM role authentication method, modify the Deployment YAML configuration and set the ServiceAccountName parameter to the service account name of the pod running the workload (for example, app1-service). For the OIDC (ACK) authentication method, no modification is required.

      • Create a new Deployment

        Create from Image

        1. Click Create from Image above the Deployment list and configure the parameters.

        2. When configuring Advanced, go to the Labels and Annotations section and add the pod annotation: enter kms-agent-webhook-injector/inject in the Name column and true in the Value column.

        3. Click Create to complete.

        Create from YAML

        1. Click Create from YAML above the Deployment list.

        2. Edit the YAML file and add kms-agent-webhook-injector/inject: "true" under spec.template.metadata.annotations (create this section if it does not exist).

        3. Click Create to complete.

      • Modify an existing Deployment

        1. Find the target workload and click Actions > Details.

        2. On the details page, click Edit YAML in the upper-right corner.

        3. Add kms-agent-webhook-injector/inject: "true" under spec.template.metadata.annotations (create this section if it does not exist).

        4. Click Update and wait for the workload to be ready.

  2. Verify the injection

    1. Go back to Workloads > Deployments and click the name of the target Deployment to go to the details page.

    2. On the Pods tab, check the Image column. The KMS Agent appears injected into the pod as a Sidecar.

      Note

      A pod may be injected with the KMS Agent twice. This happens because an init container is used for initialization. The init container terminates (Terminated) after initialization completes, and does not negatively impact the application or continuously consume compute resources.

Retrieve KMS credentials

After your Deployment is injected with the KMS Agent, the application container can retrieve credentials from KMS through HTTP requests to the KMS Agent. No AccessKey needs to be configured in your code. The following examples show how to retrieve credentials. Replace <SecretId> with your actual credential name.

Important
  • The KMS Agent listens only on 127.0.0.1, which means only applications or processes on the same machine can communicate with it. External network devices cannot connect. The access address supports only localhost or 127.0.0.1, not the local IP of the application. The following examples use localhost.

  • In addition to the KMS Agent access method, KMS also supports access via SDK. For specific operations, see Secrets Manager Client.

Using curl

# Read the token from a file
curl -v -H "X-KMS-Token:$(</var/run/kmstoken/token)" 'http://localhost:2025/secretsmanager/get?secretId=<SecretId>'

# Or write the token directly
curl -v -H "X-KMS-Token:<token>" 'http://localhost:2025/secretsmanager/get?secretId=<SecretId>'

Go code example

package main

import (
    "fmt"
    "io"
    "net/http"
    "os"
)

func main() {
    // You can specify versionStage or versionId to retrieve a specific credential version.
    // The following example retrieves a credential by versionId:
    // url := fmt.Sprintf("http://localhost:2025/secretsmanager/get?secretId=%s&versionId=%s", "agent-test", "version-id")
    url := fmt.Sprintf("http://localhost:2025/secretsmanager/get?secretId=%s", "agent-test")

    // Note: In the ACK scenario, the token path is /var/run/kmstoken/token
    token, err := os.ReadFile("/var/run/kmstoken/token")
    if err != nil {
        fmt.Printf("error reading token file: %v\n", err)
        return
    }

    req, err := http.NewRequest("GET", url, nil)
    if err != nil {
        fmt.Printf("error creating request: %v\n", err)
        return
    }

    req.Header.Add("X-KMS-Token", string(token))

    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        fmt.Printf("error sending request: %v \n", err)
        return
    }
    defer resp.Body.Close()

    body, _ := io.ReadAll(resp.Body)
    fmt.Printf("status code %d - %s \n", resp.StatusCode, string(body))
}

Expected response

A successful request returns an HTTP status code 200 and a JSON body containing the credential value. The following is an example response:

{
    "RequestId": "4e5b3c7a-****-****-****-1a2b3c4d5e6f",
    "SecretName": "agent-test",
    "SecretData": {
        "username": "admin",
        "password": "Ex@mpleP@ssw0rd"
    },
    "VersionId": "v1",
    "VersionStage": "ACSCurrent"
}

Billing

  • KMS-side costs:

    Before you use the KMS Agent, you must purchase a KMS instance, which is billed on a subscription basis. The KMS Agent itself does not incur extra charges. For more information, see Product billing.

  • ACK-side costs:

    The ack-kms-agent-webhook-injector component is free. You may incur additional costs for compute resources consumed by the injected Sidecar and Webhook workloads.

    • After you install the ack-kms-agent-webhook-injector component, a Webhook service workload is generated, which consumes compute resources and incurs costs. Limit the CPU and memory usage of this workload in the configuration file.

    • When you create or update a qualifying workload, the ack-kms-agent-webhook-injector injects the KMS Agent as a Sidecar into the container. The KMS Agent consumes compute resources and incurs costs.

Troubleshooting

If you encounter issues when installing or using the KMS Agent, refer to the following common issues for troubleshooting.

Cause

Solution

Failed to install the Agent

Run kubectl get nodes to verify the cluster is in the Running state. Check KMS permissions under the RAM user.

Network unreachable or timeout

Ensure that the ACK cluster can reach the KMS service. If you access KMS through a VPC, make sure the VPC where the ACK cluster resides is network-connected to the KMS instance.

Insufficient permissions

Check the following configurations:

  • OIDC (ACK): Verify the namespace and service account, and check whether a PodNamePrefix filter is configured.

  • RAM role: Verify that the RAM role is correctly attached to the identity provider and that the policy includes the kms:GetSecretValue and kms:Decrypt permissions.

Failed to inject the Agent

Check whether the ack-kms-agent-webhook-injector component is correctly installed and running normally. Verify that the pod annotation kms-agent-webhook-injector/inject: "true" is correctly configured.

RRSA is not enabled

The OIDC (ACK) authentication method depends on the RRSA (RAM Roles for Service Accounts) feature. In the ACK console, go to the cluster's security and audit module to enable RRSA. For detailed steps, see the identity provider setup in the RAM role tab of Step 2.