Implement elastic scaling for applications with HPA based on QPS data
If your application must dynamically adjust total compute resources based on request volume, use ALB Ingress QPS metrics from Log Service to autoscale application pods with HPA.
Before you read
Before you read this topic, we recommend that you read Expose services with ALB Ingress to learn the basics of ALB Ingress.
Prerequisites
Before you begin, ensure that:
-
alibaba-cloud-metrics-adapter version 2.3.0 or later is installed.
-
The ALB Ingress controller is installed.
-
Apache Benchmark is installed.
-
The cluster is associated with a Log Service project that stores the ALB access logs.
-
Two vSwitches are deployed in different zones of the Virtual Private Cloud (VPC) where your cluster resides.
How it works
-
Create a Deployment and Service for your application.
-
Create an ALB Ingress to route external traffic to the Service.
-
Create a Horizontal Pod Autoscaler (HPA) that watches the
sls_alb_ingress_qpsmetric from Log Service. -
QPS (queries per second) is the number of requests received each second. When QPS exceeds the per-pod threshold, the HPA scales out the Deployment. When QPS drops, the HPA scales it back in.
Step 1: Create an application and a service
-
Create a file named
tea.yaml:apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment-basic labels: app: tea spec: replicas: 2 selector: matchLabels: app: tea template: metadata: labels: app: tea spec: containers: - name: tea image: nginx:1.7.9 ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: tea-svc namespace: default spec: ports: - port: 80 protocol: TCP targetPort: 80 selector: app: tea type: NodePort -
Apply the manifest:
kubectl apply -f tea.yaml
Step 2: Create an ALB Ingress
The procedure creates billable resources. You are charged for the Internet-facing ALB instance and for the public network traffic that passes through it. The ALB access logs are written to the Log Service project and Logstore, which are also billable. To stop incurring charges after the test, delete the AlbConfig object, which releases the ALB instance, and delete the Logstore. For pricing details, see ALB billing rules and Simple Log Service billing overview.
Create an AlbConfig object
-
Find the Log Service project associated with your cluster. In the ACK console, open the cluster details page, choose the Cluster Information tab, locate Log Service Project, and record the project name. Create a file named
alb-test.yaml:Field Description zoneMappingsAt least two vSwitch IDs from different zones in the same VPC. logProjectThe Log Service project name. logStoreThe Logstore name. Must start with alb_. Auto-created if it does not exist.apiVersion: alibabacloud.com/v1 kind: AlbConfig metadata: name: alb-demo spec: config: name: alb-test addressType: Internet # Internet-facing ALB zoneMappings: - vSwitchId: vsw-uf6ccg2a9g71hx8go**** # Replace with your first vSwitch ID - vSwitchId: vsw-uf6nun9tql5t8nh15**** # Replace with your second vSwitch ID (different zone) accessLogConfig: logProject: "****" # Replace with your Log Service project name logStore: "alb_****" # Replace with your Logstore name; must start with alb_ listeners: - port: 80 protocol: HTTP -
Apply the manifest:
kubectl apply -f alb-test.yaml
Create an IngressClass
-
Create a file named
alb.yaml:apiVersion: networking.k8s.io/v1 kind: IngressClass metadata: name: alb spec: controller: ingress.k8s.alibabacloud/alb parameters: apiGroup: alibabacloud.com kind: AlbConfig name: alb-demo # Must match the AlbConfig metadata.name above -
Apply the manifest:
kubectl apply -f alb.yaml
Create the Ingress
-
Create a file named
tea-ingress.yaml:apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: tea-ingress spec: ingressClassName: alb rules: - host: demo.ingress.top # Replace with your domain http: paths: - path: /tea pathType: Prefix backend: service: name: tea-svc port: number: 80 -
Apply the manifest:
kubectl apply -f tea-ingress.yaml -
Get the ALB address assigned to the Ingress:
kubectl get ingressExpected output:
NAME CLASS HOSTS ADDRESS PORTS AGE tea-ingress alb demo.ingress.top alb-110zvs5nhsvfv*****.cn-chengdu.alb.aliyuncs.com 80 7m5sNote the
HOSTSandADDRESSvalues for the stress test in Step 4.
Step 3: Create an HPA
-
Create a file named
hpa.yaml:apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: ingress-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: nginx-deployment-basic # The Deployment to scale minReplicas: 2 # Minimum number of pods maxReplicas: 10 # Maximum number of pods metrics: - type: External external: metric: name: sls_alb_ingress_qps # ALB Ingress QPS metric from Log Service. Do not change. selector: matchLabels: sls.project: "****" # Replace with your Log Service project name sls.logstore: "alb_****" # Replace with your Logstore name sls.ingress.route: "default-tea-svc-80" # Format: <namespace>-<service-name>-<port> # Example: default-nginx-80 target: type: AverageValue # Scale based on average QPS per pod averageValue: 2 # Scale out when average QPS per pod exceeds 2This HPA scales
nginx-deployment-basicbetween 2 and 10 pods. Set minReplicas to an integer that is at least 1, and maxReplicas to a value greater than minReplicas. The HPA scales out when average QPS per pod exceeds 2, and scales in when QPS drops. -
Apply the manifest:
kubectl apply -f hpa.yaml -
Verify the HPA was created:
kubectl get hpaExpected output:
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE ingress-hpa Deployment/nginx-deployment-basic 0/2 (avg) 2 10 2 4h34mQPS is
0because no traffic reaches the application yet. The pod count is at its minimum (2). -
(Optional) Inspect the HPA details:
kubectl describe hpa ingress-hpaExpected output:
Name: ingress-hpa Namespace: default Labels: <none> Annotations: <none> CreationTimestamp: Tue, 31 Jan 2023 11:35:01 +0800 Reference: Deployment/nginx-deployment-basic Metrics: ( current / target ) "sls_alb_ingress_qps" (target average value): 0 / 2 Min replicas: 2 Max replicas: 10 Deployment pods: 2 current / 2 desired
Step 4: Verify auto scaling
Verify scale-out
-
Run a stress test against the ALB address from Step 2. Replace the placeholder with your actual address.
ab -c 5 -n 5000 -H Host:demo.ingress.top http://alb-110zvs5nhsvfv*****.cn-chengdu.alb.aliyuncs.com/tea -
While the test is running, watch the HPA status in real time:
kubectl get hpa ingress-hpa --watchWhen QPS exceeds the threshold, the replica count increases. Press Ctrl+C to stop.
-
After the stress test, confirm the scale-out:
kubectl get hpaExpected output:
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE ingress-hpa Deployment/nginx-deployment-basic 12500m/2 (avg) 2 10 10 15mREPLICASis10, confirming the Deployment scaled to the maximum.
Verify scale-in
After the stress test ends, QPS drops to 0 and the HPA scales the Deployment back in. Scale-in has a ~5-minute stabilization window, so wait before checking.
kubectl get hpa
Expected output:
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
ingress-hpa Deployment/nginx-deployment-basic 0/2 (avg) 2 10 2 60m
REPLICAS is back to 2, confirming the Deployment scaled in.
Next steps
-
Tune scale-out and scale-in by configuring the
behaviorfield in the HPA spec. -
Scale based on additional Alibaba Cloud metrics.
-
To scale an application based on the CPU or memory load of pods, see Horizontal Pod Autoscaler (HPA).
-
To scale an application on a schedule, see Scheduled horizontal scaling for containers (CronHPA).
-
To scale nodes, see Node scaling.