Use a multi-cluster Ingress
Multi-cluster Ingresses act as Layer-7 load balancers that route external HTTP traffic to Services deployed across multiple Kubernetes clusters. By combining multi-cluster Services (MCS) with Ingress rules, a single domain name resolves to Ingress endpoints in multiple clusters, and each cluster forwards requests to backends across all associated clusters.
Prerequisites
Before you begin, ensure that you have:
-
Fleet management enabled
-
A Fleet instance associated with multiple clusters running Kubernetes 1.21 or later. For more information, see Associate clusters with a Fleet instance
-
The kubeconfig file of the Fleet instance downloaded from the Distributed Cloud Container Platform for Kubernetes (ACK One) console and kubectl connected to the Fleet instance
-
The AMC command-line tool installed
-
A multi-cluster Service created and mapped to the backend applications and Services. For more information, see MCS overview
-
The NGINX Ingress controller version 1.7.0-aliyun.1 or later installed in every cluster that will host an Ingress. For more information, see Update the NGINX Ingress controller
How it works
After you configure multi-cluster Services and Ingress rules, requests to your domain name are routed to Services deployed across multiple clusters.
The following diagram shows the overall traffic flow:
In the following example, Ingresses are deployed in ACK Cluster 1 and ACK Cluster 2, while the backend application runs in ACK Cluster 1, ACK Cluster 2, and ACK Cluster 3. The diagram below shows how requests are distributed:
Key components in the flow:
-
ServiceExport: Exports a cluster-local Service to the multi-cluster Service.
-
ServiceImport: Imports the multi-cluster Service into a cluster so that the local Ingress can reach backends in all associated clusters. Imported Services are named in the
amcs-{Service name}format. -
NGINX Ingress controller: Resolves requests against Ingress rules and forwards them to the imported Service.
Configure multi-cluster Ingresses
Step 1: Map the multi-cluster Service to clusters with Ingresses
For each cluster that will host an Ingress, follow the steps in Use the CLI to configure MCS:
-
In each cluster, create a ServiceExport to export the local Service to the multi-cluster Service.
-
In each cluster that hosts an Ingress, create a ServiceImport to import the multi-cluster Service. The imported Service is automatically named
amcs-{Service name}(for example,amcs-nginx). -
Verify the imported Service in each Ingress-hosting cluster:
kubectl get service amcs-nginxExpected output:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE amcs-nginx ClusterIP 192.168.118.171 <none> 80/TCP 34mkubectl get endpointsliceExpected output:
NAME ADDRESSTYPE PORTS ENDPOINTS AGE imported-c0c4a2c9339e44235a5228d0b0ff2259c-nginx-vnlnd IPv4 80 10.0.2.4,10.0.2.5 34m imported-ce962c460ae284243a3161d6f2c7b4cf7-nginx-d8rr5 IPv4 80 10.0.2.221,10.0.2.236 29m imported-ca5b667538054499ab399588efe4ab654-nginx-xfe1a IPv4 80 10.0.2.145,10.0.2.123 20mEach row in the
ENDPOINTScolumn corresponds to the pod IPs in one of the associated clusters. If your setup has three clusters, you should see three EndpointSlice entries. Cross-reference each IP range with pod IPs in each cluster to confirm the import is correct.
Step 2: Configure Ingress rules
Run the following steps in each cluster that hosts an Ingress.
-
Create a file named
ingress.yamlwith the following content:apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: multi-cluster-nginx-ingress spec: rules: - host: example.com http: paths: - path: / pathType: Prefix backend: service: name: amcs-nginx port: 80 -
Apply the Ingress:
kubectl apply -f ingress.yaml -
Query the Ingress endpoints of the clusters that are configured with Ingresses to get the external IP address assigned to each Ingress:
# ACK Cluster 1 kubectl get ingress NAME CLASS HOSTS ADDRESS PORTS AGE multi-cluster-nginx-ingress nginx example.com XX.XX.XX.XX 80 26m # ACK Cluster 2 kubectl get ingress NAME CLASS HOSTS ADDRESS PORTS AGE multi-cluster-nginx-ingress nginx example.com XX.XX.XX.XX 80 26mNote the value in the
ADDRESScolumn for each cluster. You will use these IP addresses in the next step when you create DNS records.
Step 3: Set up DNS resolution
Create an A record for each Ingress-hosting cluster that maps your domain name to the Ingress IP address you retrieved in Step 2. For instructions, see Add DNS records.
For example, if ACK Cluster 1 has Ingress IP 1.2.3.4 and ACK Cluster 2 has Ingress IP 5.6.7.8, create two A records for example.com: one pointing to 1.2.3.4 and one pointing to 5.6.7.8. DNS will distribute traffic between the two clusters.
What's next
-
To learn more about multi-cluster Service configuration, see MCS overview.
-
To update the NGINX Ingress controller to meet the minimum version requirement, see Update the NGINX Ingress controller.