Use Terraform to create or modify ASM custom resources-Alibaba Cloud Service Mesh-阿里云帮助中心

Service Mesh (ASM) supports the Terraform Kubernetes Provider for managing custom resources as code, starting from version 1.22.6.109. This guide walks through two workflows: creating a VirtualService and modifying an existing ASMMeshConfig resource.

Why use Terraform for ASM resources

kubectl apply works for ad-hoc changes, but Terraform provides additional capabilities for managing service mesh resources at scale:

Unified workflow -- Manage ASM resources alongside cluster infrastructure in a single configuration language.
State tracking -- Track resource state, plan changes, and detect drift without manually inspecting the Kubernetes API.
Dependency management -- Terraform resolves relationships between resources and applies changes in the correct order.

Prerequisites

Before you begin, make sure that you have:

A kubectl client connected to an ACK cluster. For details, see Get a cluster kubeconfig and connect to the cluster using kubectl
(Optional) Terraform installed and configured locally
(Optional) The tfk8s tool for converting Kubernetes YAML to Terraform HCL

Note: All commands in this guide also work in Cloud Shell. Before you start, switch the Terraform version to ensure that the Terraform version is greater than 0.14.

Set up the Terraform project

Create an empty directory for the Terraform project and add a provider.tf file: This tells the Terraform Kubernetes Provider to use your local kubeconfig for cluster access.
```
   provider "kubernetes" {
     config_path = "~/.kube/config"
   }
```
Initialize the project:
```
   terraform init
```

The final directory structure used in this guide looks like this:

terraform-Project/
├── provider.tf           # Provider configuration
├── virtualservice.tf     # Scenario 1: Create a VirtualService
├── asmmeshconfig.tf      # Scenario 2: Modify ASMMeshConfig
└── resources/
    └── demo.yaml         # VirtualService YAML definition

Important

The kubernetes_manifest resource queries the Kubernetes API during terraform plan to validate the resource schema. The cluster must be reachable before you run any plan or apply commands.

Create a VirtualService

This workflow creates an Istio VirtualService that routes traffic to productpage.prod.svc.cluster.local with a 5-second timeout.

Define the resource in YAML

Create a file named resources/demo.yaml:

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: my-productpage-rule
  namespace: istio-system
spec:
  hosts:
  - productpage.prod.svc.cluster.local # ignores rule namespace
  http:
  - timeout: 5s
    route:
    - destination:
        host: productpage.prod.svc.cluster.local

Create the Terraform configuration

Create a file named virtualservice.tf:

resource "kubernetes_manifest" "virtualservice_demo" {
  manifest = yamldecode(file("./resources/demo.yaml"))
}

The yamldecode(file(...)) expression reads the YAML file and converts it to a Terraform-compatible map.

Preview and apply

Preview the changes: The output shows that one resource will be created:
```
   terraform plan
```
```
   Plan: 1 to add, 0 to change, 0 to destroy.
```
Apply the changes:
```
   terraform apply --auto-approve
```

Verify the resource

kubectl get VirtualService -n istio-system

Expected output:

NAME                  GATEWAYS   HOSTS                                    AGE
my-productpage-rule              ["productpage.prod.svc.cluster.local"]   77s

Clean up

To remove the VirtualService:

terraform destroy -target=kubernetes_manifest.virtualservice_demo --auto-approve

Modify an existing ASMMeshConfig resource

This workflow imports an existing ASMMeshConfig resource into Terraform state and then modifies it to disable automatic Pod health check rewriting by setting spec.sidecarInjectorWebhookConfiguration.rewriteAppHTTPProbe to false.

ASMMeshConfig already exists in the cluster, so you must import it into Terraform state before making changes. Two import approaches are available:

Recommended: Use the tfk8s tool to generate the .tf file directly from the live resource.
Alternative: Use built-in Terraform commands if tfk8s is not available.

Import ID syntax

The terraform import command for kubernetes_manifest uses the following ID format:

"apiVersion=<api-version>,kind=<kind>,[namespace=<namespace>,]name=<name>"

The namespace parameter is required only for namespace-scoped resources. ASMMeshConfig is a cluster-scoped resource, so no namespace is needed.

Parameter	Description
`kubernetes_manifest`	Terraform resource type, matching the resource type in the `.tf` file
`asmmeshconfig_default`	Terraform resource name, matching the resource name in the `.tf` file
`apiVersion`	API version registered in the Kubernetes CRD. Check with `kubectl get <resource-type> <resource-name> -o yaml`
`kind`	Resource type registered in the Kubernetes CRD
`name`	Name of the resource to import

Import with tfk8s (recommended)

Generate asmmeshconfig.tf from the live resource: This produces a clean .tf file. The expected content is similar to:

   kubectl get asmmeshconfig default -o yaml | tfk8s --strip -o asmmeshconfig.tf

   resource "kubernetes_manifest" "asmmeshconfig_default" {
     manifest = {
       "apiVersion" = "istio.alibabacloud.com/v1beta1"
       "kind" = "ASMMeshConfig"
       "metadata" = {
         "name" = "default"
       }
       "spec" = {
         "accessLogConfiguration" = {}
         "ambientConfiguration" = {
           "enabled" = false
           "redirectMode" = ""
           "waypoint" = {}
           "ztunnel" = {}
         }
         "cniConfiguration" = {
           "enabled" = true
           "excludeNamespaces" = "istio-system,kube-system"
           "repair" = {}
         }
         "enableGatewayAPI" = true
         "gatewayAPIInferenceExtension" = {}
         "ingressControllerMode" = "OFF"
         "ingressSelector" = "ingressgateway1"
         "ingressService" = "istio-ingressgateway1"
         "sidecarInjectorWebhookConfiguration" = {}
         "smcEnabled" = false
       }
     }
   }

Import the resource into Terraform state:

   terraform import kubernetes_manifest.asmmeshconfig_default "apiVersion=istio.alibabacloud.com/v1beta1,name=default,kind=ASMMeshConfig"

Align the Terraform and Kubernetes resource states:
```
   terraform apply --auto-approve
```

Import with Terraform commands

Use this approach if tfk8s is not installed.

Import the resource into Terraform state:

   terraform import kubernetes_manifest.asmmeshconfig_default "apiVersion=istio.alibabacloud.com/v1beta1,name=default,kind=ASMMeshConfig"

Export the current state to a .tf file:

   terraform show -no-color > asmmeshconfig.tf

Edit asmmeshconfig.tf manually: The cleaned-up file looks like this:

Change object to manifest.
Remove all parameters with null values.

   resource "kubernetes_manifest" "asmmeshconfig_default" {
     manifest = {
       apiVersion = "istio.alibabacloud.com/v1beta1"
       kind       = "ASMMeshConfig"
       metadata   = {
         name = "default"
       }
       spec = {
         accessLogConfiguration         = {}
         adaptiveSchedulerConfiguration = {}
         ambientConfiguration           = {
           redirectMode = ""
           waypoint     = {}
           ztunnel      = {}
         }
         cniConfiguration = {
           enabled = true
           repair  = {}
         }
         localityLbSetting = {
           enabled = true
         }
       }
     }
   }

Align the Terraform and Kubernetes resource states:
```
   terraform apply --auto-approve
```

Apply the configuration change

After the import is complete, edit asmmeshconfig.tf to add rewriteAppHTTPProbe = false under sidecarInjectorWebhookConfiguration:

resource "kubernetes_manifest" "asmmeshconfig_default" {
  manifest = {
    "apiVersion" = "istio.alibabacloud.com/v1beta1"
    "kind" = "ASMMeshConfig"
    "metadata" = {
      "name" = "default"
    }
    "spec" = {
      "accessLogConfiguration" = {}
      "ambientConfiguration" = {
        "enabled" = false
        "redirectMode" = ""
        "waypoint" = {}
        "ztunnel" = {}
      }
      "cniConfiguration" = {
        "enabled" = true
        "excludeNamespaces" = "istio-system,kube-system"
        "repair" = {}
      }
      "enableGatewayAPI" = true
      "gatewayAPIInferenceExtension" = {}
      "ingressControllerMode" = "OFF"
      "ingressSelector" = "ingressgateway1"
      "ingressService" = "istio-ingressgateway1"
      "sidecarInjectorWebhookConfiguration" = {
        "rewriteAppHTTPProbe" = false
      }
      "smcEnabled" = false
    }
  }
}

Preview the changes: The output shows the specific field change:

   terraform plan

   # kubernetes_manifest.asmmeshconfig_default will be updated in-place
   ~ resource "kubernetes_manifest" "asmmeshconfig_default" {
       ~ manifest = {
           ~ spec       = {
               ~ sidecarInjectorWebhookConfiguration = {
                   + rewriteAppHTTPProbe = false
                 }
                 # (9 unchanged attributes hidden)
             }
             # (3 unchanged attributes hidden)
         }
     }
   Plan: 0 to add, 1 to change, 0 to destroy.

Apply the changes: Expected output:

   terraform apply --auto-approve

   kubernetes_manifest.asmmeshconfig_default: Modifying...
   kubernetes_manifest.asmmeshconfig_default: Modifications complete after 1s
   ...
   Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Troubleshooting

State drift between Terraform and Kubernetes

If terraform plan shows inconsistent changes between manifest and object, the Kubernetes resource was modified outside of Terraform (for example, through the ASM console or kubectl). Run terraform refresh to sync Terraform state with the live cluster state before applying further changes.

Plan-time API access errors

The kubernetes_manifest resource queries the Kubernetes API during terraform plan to validate the resource schema. If the cluster is unreachable at plan time, the plan fails. Make sure the cluster is accessible and your kubeconfig is valid before running terraform plan.

Inconsistent result after apply

ASM resources may be modified by mutating admission controllers after Terraform applies a change. If you see Error: Provider produced inconsistent result after apply, add a computed_fields block to your resource definition to tell Terraform to ignore those fields:

resource "kubernetes_manifest" "asmmeshconfig_default" {
  computed_fields = ["metadata.annotations", "metadata.labels"]
  manifest = {
    # ... your manifest
  }
}

List only the fields that the API server or admission controllers modify automatically.

What's next

Install and configure Terraform locally -- Set up the Terraform Kubernetes Provider for managing ASM resources.
Service Mesh (ASM) documentation -- Explore other ASM custom resources you can manage with Terraform.

上一篇: Associate an EIP with the ASM control plane 下一篇: Manage global namespaces