DocsICP FilingConsole
官方文档
首页

Manage multi-cluster applications with ASM and Karmada

更新时间：
复制 MD 格式
产品详情
我的收藏

Service Mesh (ASM) integrates with Karmada (Kubernetes Armada) to deploy and run cloud-native applications across multiple Kubernetes clusters without requiring application changes. Karmada provides plug-and-play automation for multi-cluster application management in cloud-native scenarios, with key features like centralized multi-cloud management, high availability, fault recovery, and traffic scheduling. This topic demonstrates how to use ASM and Karmada for multi-cluster application management.

Background information

Karmada uses Kubernetes-native APIs to define federated resource templates, allowing for easy integration with existing Kubernetes tools. Karmada also provides a standalone PropagationPolicy API to define multi-cluster scheduling requirements.

  • It supports a 1:N policy mapping mechanism, so you do not need to specify scheduling constraints every time you create a federated application.

  • When using default policies, you can interact directly with the Kubernetes API.

Cluster modes

Karmada can manage member clusters in two modes: push and pull. The main difference between the push and pull modes is how member clusters are accessed when manifests are deployed.

Push mode

The Karmada control plane directly accesses the kube-apiserver of a member cluster to get the cluster status and deploy manifests.

Pull mode

The Karmada control plane does not directly access member clusters. Instead, it delegates requests to an agent component named karmada-agent.

Each karmada-agent serves a single cluster and has the following responsibilities:

  • Registers the cluster with Karmada (creates a Cluster object).

  • Maintains and reports the cluster status to Karmada (updates the status of the Cluster object).

  • Watches for manifests from the Karmada execution space (a namespace in the format karmada-es-<cluster name>) and applies them to the cluster it serves.

Key components

The Karmada control plane includes the following components:

  • Karmada API Server

  • Karmada Controller Manager

  • Karmada Scheduler

etcd stores the Karmada API objects. The API Server provides the REST endpoint for all other components. The Karmada Controller Manager acts on the API objects that you create through the API server.

image

Prerequisites

  • You have two ACK clusters (named member1 and member2 in this example) in the same virtual private cloud (VPC). For more information, see Create an ACK managed cluster.

    Note

    • This topic uses two clusters in the same VPC as an example. If your clusters are in different VPCs, you must configure network connectivity between the VPCs.

    • We recommend using an enterprise security group when creating the clusters.

  • You have created an ASM instance (named mesh1 in this example) and configured a sidecar injection policy for the default namespace. For more information, see Create an ASM instance and Configure a sidecar injection policy.

  • You have deployed a Karmada primary cluster instance (named karmada-master in this example) and added the two ACK clusters (member1 and member2) as member clusters. For more information, see Karmada Installation.

  • You have added the two clusters to the ASM instance (mesh1) and created a serverless ingress gateway. For more information, see Add clusters to an ASM instance and create a serverless ingress gateway.

Step 1: Deploy applications with Karmada

This topic uses the Bookinfo sample application, similar to the process described in Use an ASM serverless gateway for multi-cluster ingress. However, instead of manually deploying different reviews versions to separate clusters, this topic shows how to achieve the same result by creating a Karmada PropagationPolicy in the karmada-master cluster.

  1. Create a file named bookinfo-karmada.yaml with the following content.

    View YAML content

    # Details service
apiVersion: v1
kind: Service
metadata:
  name: details
  labels:
    app: details
    service: details
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: details
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: bookinfo-details
  labels:
    account: details
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: details-v1
  labels:
    app: details
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: details
      version: v1
  template:
    metadata:
      labels:
        app: details
        version: v1
    spec:
      serviceAccountName: bookinfo-details
      containers:
      - name: details
        image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/examples-bookinfo-details-v1:1.19.1
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
# Ratings service
apiVersion: v1
kind: Service
metadata:
  name: ratings
  labels:
    app: ratings
    service: ratings
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: ratings
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: bookinfo-ratings
  labels:
    account: ratings
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ratings-v1
  labels:
    app: ratings
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ratings
      version: v1
  template:
    metadata:
      labels:
        app: ratings
        version: v1
    spec:
      serviceAccountName: bookinfo-ratings
      containers:
      - name: ratings
        image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/examples-bookinfo-ratings-v1:1.19.1
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
# Reviews service
apiVersion: v1
kind: Service
metadata:
  name: reviews
  labels:
    app: reviews
    service: reviews
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: reviews
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: bookinfo-reviews
  labels:
    account: reviews
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: reviews-v1
  labels:
    app: reviews
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: reviews
      version: v1
  template:
    metadata:
      labels:
        app: reviews
        version: v1
    spec:
      serviceAccountName: bookinfo-reviews
      containers:
      - name: reviews
        image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/examples-bookinfo-reviews-v1:1.19.1
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: reviews-v2
  labels:
    app: reviews
    version: v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: reviews
      version: v2
  template:
    metadata:
      labels:
        app: reviews
        version: v2
    spec:
      serviceAccountName: bookinfo-reviews
      containers:
      - name: reviews
        image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/examples-bookinfo-reviews-v2:1.19.1
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: reviews-v3
  labels:
    app: reviews
    version: v3
spec:
  replicas: 1
  selector:
    matchLabels:
      app: reviews
      version: v3
  template:
    metadata:
      labels:
        app: reviews
        version: v3
    spec:
      serviceAccountName: bookinfo-reviews
      containers:
      - name: reviews
        image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/examples-bookinfo-reviews-v3:1.19.1
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
# Productpage services
apiVersion: v1
kind: Service
metadata:
  name: productpage
  labels:
    app: productpage
    service: productpage
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: productpage
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: bookinfo-productpage
  labels:
    account: productpage
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: productpage-v1
  labels:
    app: productpage
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: productpage
      version: v1
  template:
    metadata:
      labels:
        app: productpage
        version: v1
    spec:
      serviceAccountName: bookinfo-productpage
      containers:
      - name: productpage
        image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/examples-bookinfo-productpage-v1:1.19.1
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
        volumeMounts:
        - name: tmp
          mountPath: /tmp
        securityContext:
          runAsUser: 1000
      volumes:
      - name: tmp
        emptyDir: {}
---

  2. Run the following command to deploy the Bookinfo application in the Karmada primary cluster:

    kubectl --kubeconfig /etc/karmada/karmada-apiserver.config apply -f bookinfo-karmada.yaml
    Note

    By default, the kubeconfig file for the Karmada primary cluster is saved to /etc/karmada/karmada-apiserver.config. If you chose a different installation path, replace the path after --kubeconfig with the correct path.

  3. Create a file named propagation.yaml with the following content.

    apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: service-propagation
spec:
  resourceSelectors:
    - apiVersion: v1
      kind: Service
      name: productpage
    - apiVersion: v1
      kind: Service
      name: details
    - apiVersion: v1
      kind: Service
      name: reviews
    - apiVersion: v1
      kind: Service
      name: ratings
  placement:
    clusterAffinity:
      clusterNames:
        - member1
        - member2
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: produtpage-propagation
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: productpage-v1
    - apiVersion: v1
      kind: ServiceAccount
      name: bookinfo-productpage
  placement:
    clusterAffinity:
      clusterNames:
        - member1
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: details-propagation
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: details-v1
    - apiVersion: v1
      kind: ServiceAccount
      name: bookinfo-details
  placement:
    clusterAffinity:
      clusterNames:
        - member2
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: reviews-propagation
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: reviews-v1
    - apiVersion: apps/v1
      kind: Deployment
      name: reviews-v2
    - apiVersion: apps/v1
      kind: Deployment
      name: reviews-v3
    - apiVersion: v1
      kind: ServiceAccount
      name: bookinfo-reviews
  placement:
    clusterAffinity:
      clusterNames:
        - member1
        - member2
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: ratings-propagation
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: ratings-v1
    - apiVersion: v1
      kind: ServiceAccount
      name: bookinfo-ratings
  placement:
    clusterAffinity:
      exclude:
        - member1

    The .spec.placement.clusterAffinity field in a PropagationPolicy defines scheduling constraints for a specific cluster. Without these constraints, any cluster can be a scheduling candidate.

    You can configure the following four fields:

    Parameter

    Description

    labelSelector

    A filter to select member clusters by their labels. If this field is not empty, only clusters that match the filter are selected.

    fieldSelector

    A filter to select member clusters by their fields. If this field is not empty, only clusters that match the filter are selected.

    clusterNames

    A list of cluster names to explicitly include in the selection.

    exclude

    A list of cluster names to explicitly exclude from the selection.

    This example uses only the clusterNames and exclude fields. For more information about the .spec.placement.clusterAffinity field, see Resource Propagating in the Karmada documentation.

  4. Run the following command to deploy the PropagationPolicy:

    kubectl --kubeconfig /etc/karmada/karmada-apiserver.config apply -f propagation.yaml

  5. Use the kubeconfig files for the member1 and member2 clusters to view the Deployments in each cluster.

    1. member1

      kubectl --kubeconfig member1 get deployment

      Expected output:

      NAME             READY   UP-TO-DATE   AVAILABLE   AGE
productpage-v1   1/1     1            1           12m
reviews-v1       1/1     1            1           12m
reviews-v2       1/1     1            1           12m
reviews-v3       1/1     1            1           12m

    2. member2

      kubectl --kubeconfig member2 get deployment

      Expected output:

      NAME         READY   UP-TO-DATE   AVAILABLE   AGE
details-v1   1/1     1            1           16m
ratings-v1   1/1     1            1           16m
reviews-v1   1/1     1            1           16m
reviews-v2   1/1     1            1           16m
reviews-v3   1/1     1            1           16m

Step 2: Add virtual service and gateway

  1. In the default namespace, create a virtual service named bookinfo with the following content. For more information, see Manage virtual services.

    apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: bookinfo
spec:
  hosts:
  - "*"
  gateways:
  - bookinfo-gateway
  http:
  - match:
    - uri:
        exact: /productpage
    - uri:
        prefix: /static
    - uri:
        exact: /login
    - uri:
        exact: /logout
    - uri:
        prefix: /api/v1/products
    route:
    - destination:
        host: productpage
        port:
          number: 9080

  2. In the default namespace, create a gateway named bookinfo-gateway with the following content. For more information, see Manage gateways.

    apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: bookinfo-gateway
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"

Step 3: Verify access

  1. Obtain the IP address of the ingress gateway. For more information, see Obtain the IP address of an ingress gateway.

  2. In your browser, enter http://{IP_address_of_the_serverless_ingress_gateway}/productpage and refresh the page multiple times. The page cycles through the three versions of the reviews service, with traffic distributed roughly evenly. Although reviews-v3 is in a different cluster than some of the other services, it works as expected.

    The reviews-v3 page shows red stars in the "Book Reviews" section and displays Reviews served by: reviews-v3-xxx at the bottom, indicating the current service version.

    The reviews-v2 page shows black stars in the "Book Reviews" section and displays Reviews served by: reviews-v2-xxx at the bottom.

    The reviews-v1 page shows only review text with no star ratings and displays the service instance, such as reviews-v1-586d96b89c-6v5p8, at the bottom.

Previous：Use an ASM Serverless gateway to create a multi-cluster ingressNext：Multi-cluster disaster recovery