Use a multi-cluster Ingress

Updated at:

Multi-cluster Ingresses act as Layer-7 load balancers that route external HTTP traffic to Services deployed across multiple Kubernetes clusters. By combining multi-cluster Services (MCS) with Ingress rules, a single domain name resolves to Ingress endpoints in multiple clusters, and each cluster forwards requests to backends across all associated clusters.

Prerequisites

Before you begin, ensure that you have:

How it works

After you configure multi-cluster Services and Ingress rules, requests to your domain name are routed to Services deployed across multiple clusters.

The following diagram shows the overall traffic flow:

Traffic flow diagram showing requests routed from a domain name through Ingress endpoints in multiple clusters to backend Services

In the following example, Ingresses are deployed in ACK Cluster 1 and ACK Cluster 2, while the backend application runs in ACK Cluster 1, ACK Cluster 2, and ACK Cluster 3. The diagram below shows how requests are distributed:

Diagram showing multi-cluster Ingress request distribution: Ingresses in ACK Cluster 1 and ACK Cluster 2 forward requests to backend pods in all three clusters

Key components in the flow:

  • ServiceExport: Exports a cluster-local Service to the multi-cluster Service.

  • ServiceImport: Imports the multi-cluster Service into a cluster so that the local Ingress can reach backends in all associated clusters. Imported Services are named in the amcs-{Service name} format.

  • NGINX Ingress controller: Resolves requests against Ingress rules and forwards them to the imported Service.

Configure multi-cluster Ingresses

Step 1: Map the multi-cluster Service to clusters with Ingresses

For each cluster that will host an Ingress, follow the steps in Use the CLI to configure MCS:

  1. In each cluster, create a ServiceExport to export the local Service to the multi-cluster Service.

  2. In each cluster that hosts an Ingress, create a ServiceImport to import the multi-cluster Service. The imported Service is automatically named amcs-{Service name} (for example, amcs-nginx).

  3. Verify the imported Service in each Ingress-hosting cluster:

    kubectl get service amcs-nginx

    Expected output:

    NAME         TYPE        CLUSTER-IP        EXTERNAL-IP   PORT(S)   AGE
    amcs-nginx   ClusterIP   192.168.118.171   <none>        80/TCP    34m
    kubectl get endpointslice

    Expected output:

    NAME                                                     ADDRESSTYPE   PORTS   ENDPOINTS               AGE
    imported-c0c4a2c9339e44235a5228d0b0ff2259c-nginx-vnlnd   IPv4          80      10.0.2.4,10.0.2.5       34m
    imported-ce962c460ae284243a3161d6f2c7b4cf7-nginx-d8rr5   IPv4          80      10.0.2.221,10.0.2.236   29m
    imported-ca5b667538054499ab399588efe4ab654-nginx-xfe1a   IPv4          80      10.0.2.145,10.0.2.123   20m

    Each row in the ENDPOINTS column corresponds to the pod IPs in one of the associated clusters. If your setup has three clusters, you should see three EndpointSlice entries. Cross-reference each IP range with pod IPs in each cluster to confirm the import is correct.

Step 2: Configure Ingress rules

Run the following steps in each cluster that hosts an Ingress.

  1. Create a file named ingress.yaml with the following content:

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: multi-cluster-nginx-ingress
    spec:
      rules:
      - host: example.com
        http:
          paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: amcs-nginx
                port: 80
  2. Apply the Ingress:

    kubectl apply -f ingress.yaml
  3. Query the Ingress endpoints of the clusters that are configured with Ingresses to get the external IP address assigned to each Ingress:

    # ACK Cluster 1
    kubectl get ingress
    NAME                          CLASS   HOSTS         ADDRESS         PORTS   AGE
    multi-cluster-nginx-ingress   nginx   example.com   XX.XX.XX.XX     80      26m
    
    # ACK Cluster 2
    kubectl get ingress
    NAME                          CLASS   HOSTS         ADDRESS         PORTS   AGE
    multi-cluster-nginx-ingress   nginx   example.com   XX.XX.XX.XX     80      26m

    Note the value in the ADDRESS column for each cluster. You will use these IP addresses in the next step when you create DNS records.

Step 3: Set up DNS resolution

Create an A record for each Ingress-hosting cluster that maps your domain name to the Ingress IP address you retrieved in Step 2. For instructions, see Add DNS records.

For example, if ACK Cluster 1 has Ingress IP 1.2.3.4 and ACK Cluster 2 has Ingress IP 5.6.7.8, create two A records for example.com: one pointing to 1.2.3.4 and one pointing to 5.6.7.8. DNS will distribute traffic between the two clusters.

What's next