What annotations are supported for configuring an NLB instance in an ACK cluster-Container Compute Service(ACS)-阿里云帮助中心

Add annotations to your Kubernetes Service YAML to configure Network Load Balancer (NLB) instances, listeners, and server groups without accessing the console.

Prerequisites

Before you begin, ensure that you have:

A CCM component at version 2.5.0 or later in your ACK cluster. To upgrade, see Manage components
A Service with spec.loadBalancerClass set to alibabacloud.com/nlb. Without this field, ACK creates a Classic Load Balancer (CLB) by default

Important

Once a Service is created, you cannot change spec.loadBalancerClass. CLB and NLB instances cannot be converted to each other.

Annotation index

All annotations use the prefix service.beta.kubernetes.io/alibaba-cloud-loadbalancer-. The table below lists every annotation by category.

Category	Annotation (suffix only)	Required	Min CCM version
NLB instance	`zone-maps`	Yes	2.5.0
	`address-type`	No	2.5.0
	`name`	No	2.5.0
	`resource-group-id`	No	2.5.0
	`ip-version`	No	2.5.0
	`additional-resource-tags`	No	2.5.0
	`id`	No	—
	`force-override-listeners`	No	2.5.0
	`bandwidth-package-id`	No	2.9.1
	`ipv6-address-type`	No	2.9.1
	`preserve-lb-on-delete`	No	2.10.0
Listener	`security-group-ids`	No	2.6.0
	`protocol-port`	No	2.5.0
	`cert-id`	No	—
	`cacert-id`	No	—
	`cacert`	No	—
	`tls-cipher-policy`	No	—
	`proxy-protocol`	No	2.5.0
	`ppv2-pvl-vpc-id-enabled`	No	2.9.1
	`ppv2-pvl-ep-id-enabled`	No	2.9.1
	`ppv2-pvl-eps-id-enabled`	No	2.9.1
	`cps`	No	2.5.0
	`idle-timeout`	No	2.5.0
	`listener-port-range`	No	2.11.4
Server group	`scheduler`	No	2.5.0
	`connection-drain`	No	2.5.0
	`connection-drain-timeout`	No	—
	`preserve-client-ip`	No	2.5.0
	`health-check-flag`	No	—
	`health-check-type`	No	—
	`health-check-connect-port`	No	—
	`health-check-connect-timeout`	No	—
	`healthy-threshold`	No	—
	`unhealthy-threshold`	No	—
	`health-check-interval`	No	—
	`health-check-uri`	No	—
	`health-check-domain`	No	—
	`health-check-method`	No	—
	`server-group-type`	No	2.8.0
	`vgroup-port`	No	—
	`weight`	No	—
	`ignore-weight-update`	No	2.11.1

NLB instance operations

Create a public-facing NLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps

Description	Default	Min CCM version
The zones for the NLB instance. Specify at least two different zones as a comma-separated list in the format `<zone-ID>:<vSwitch-ID>`, for example, `cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321`. View supported regions and zones in the NLB console.	None	2.5.0

For high availability, select vSwitches in at least two different zones.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a private NLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type

Description	Default	Min CCM version
The network type of the NLB instance. Valid values: `internet` (public-facing), `intranet` (private). Specify at least two different zones in `zone-maps`. View supported regions and zones in the NLB console.	`internet`	2.5.0

For high availability, select vSwitches in at least two different zones.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify a name

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name

Description	Default	Min CCM version
The name of the NLB instance. Must be 2–128 characters, start with a letter or Chinese character, and may contain digits, periods (.), underscores (_), and hyphens (-).	None	2.5.0

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name: "${your-nlb-name}"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify a resource group

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id

Description	Default	Min CCM version
The resource group for the NLB instance. Once set, the resource group cannot be changed. Look up your resource group ID in the Alibaba Cloud Resource Management console.	None	2.5.0

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id: "${your-resource-group-id}"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a dual-stack NLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version

Description	Default	Min CCM version
The IP version of the NLB instance. Cannot be changed after the instance is created. Valid values: `ipv4` (IPv4 only), `DualStack` (IPv4 and IPv6). When using `DualStack`, both vSwitches in `zone-maps` must have IPv6 enabled. The generated IPv6 address is accessible only from IPv6-enabled environments.	`ipv4`	2.5.0

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version: "DualStack"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Add tags

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags

Description	Default	Min CCM version
Custom tags to add to the NLB instance. Separate multiple tags with commas, for example, `k1=v1,k2=v2`.	None	2.5.0

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags: "Key1=Value1,Key2=Value2"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Use an existing NLB instance

Set both annotations together.

Annotation	Required	Description	Default	Min CCM version
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id`	Yes	The ID of the existing NLB instance.	None	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners`	No	Controls whether CCM manages listeners on the existing instance. `true`: CCM creates, updates, and deletes listeners based on the Service. `false`: CCM does not modify the listeners.	`false`	2.5.0

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "${your-nlb-id}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners: "true"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Attach an Internet Shared Bandwidth instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth-package-id

Description	Default	Min CCM version
The ID of the Internet Shared Bandwidth instance to attach to the NLB instance. Look up the bandwidth package ID in the VPC console.	None	2.9.1

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth-package-id: "cbwp-xxxxxxxxxx"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify the IPv6 network type

Use both annotations together to configure a dual-stack NLB instance with a specific IPv6 network type.

Annotation	Required	Description	Default	Min CCM version
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version`	Yes	The IP version. Must be `DualStack` when using `ipv6-address-type`. The cluster's kube-proxy mode must be IPVS. Both vSwitches in `zone-maps` must have IPv6 enabled.	`ipv4`	2.5.0
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ipv6-address-type`	No	The IPv6 network type. `intranet`: private IPv6. `internet`: public IPv6. For a public IPv6 address, an IPv6 gateway must exist in the VPC. See Create and manage an IPv6 gateway.	`intranet`	2.9.1

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version: "DualStack"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ipv6-address-type: internet
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Retain the NLB instance when deleting the service

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-lb-on-delete

Description Min CCM version

Retains the NLB instance created by the Service when you delete the LoadBalancer Service. When enabled, the kubernetes.do.not.delete and ack.aliyun.com tags are removed from the NLB instance and server group, and existing servers are kept. A PreservedOnDelete warning event is generated during service synchronization. Check for this event to confirm the feature is active. Valid values: any non-empty string (enabled), empty or unset (disabled). 2.10.0

Description	Min CCM version
Retains the NLB instance created by the Service when you delete the LoadBalancer Service. When enabled, the `kubernetes.do.not.delete` and `ack.aliyun.com` tags are removed from the NLB instance and server group, and existing servers are kept. A `PreservedOnDelete` warning event is generated during service synchronization. Check for this event to confirm the feature is active. Valid values: any non-empty string (enabled), empty or unset (disabled).	2.10.0

Important

Delete the Service to trigger retention rather than changing the service type. Changing the type may cause the service to be incorrectly re-associated with the retained NLB instance.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-lb-on-delete: "true"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Listener operations

Set security groups

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-security-group-ids

Description	Default	Min CCM version
The security groups to assign to the listener. Separate multiple security group IDs with commas, for example, `sg-aaaaa,sg-bbbbb`.	None	2.6.0

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-security-group-ids: "sg-aaaaa,sg-bbbbb"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure TCP and UDP protocols for a listener

This feature requires Kubernetes 1.24 or later. For upgrade instructions, see Upgrade an ACK cluster.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: udp
    port: 80
    protocol: UDP
    targetPort: 81
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a TCP listener

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a UDP listener

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
  name: nginx
  namespace: default
spec:
  ports:
  - name: udp
    port: 80
    protocol: UDP
    targetPort: 80
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a TCP-over-SSL listener

Set both annotations together.

Annotation	Required	Description	Default	Min CCM version
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port`	Yes	The listener protocol and port. Separate multiple values with commas, for example, `TCP:80,TCPSSL:443`.	None	2.5.0
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id`	Yes	The ID of the server certificate. Create and view certificates in the Certificate Management Service consoleCertificate Management Service consoleCertificate Management Service consoleCertificate Management Service console. The combined certificate ID format is `${your-cert-id}-cn-hangzhou` for regions in the Chinese mainland and `${your-cert-id}-ap-southeast-1` for regions outside the Chinese mainland.	None	—

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    # Chinese mainland regions: ${your-cert-id}-cn-hangzhou
    # Regions outside the Chinese mainland: ${your-cert-id}-ap-southeast-1
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${the-combined-certificate-id}"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Enable mutual authentication

Set all four annotations together.

Annotation	Required	Description	Default	Min CCM version
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port`	Yes	The listener protocol and port. For example, `TCP:80,TCPSSL:443`.	None	2.5.0
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id`	Yes	The ID of the server certificate. View it in the Certificate Management Service consoleCertificate Management Service consoleCertificate Management Service consoleCertificate Management Service console.	None	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert-id`	Yes	The ID of the CA certificate. View it in the Certificate Management Service consoleCertificate Management Service consoleCertificate Management Service consoleCertificate Management Service console.	None	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert`	Yes	Enables or disables mutual authentication. `on`: enabled. `off`: disabled.	None	—

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    # Chinese mainland regions: ${your-cert-id}-cn-hangzhou
    # Regions outside the Chinese mainland: ${your-cert-id}-ap-southeast-1
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${the-combined-certificate-id}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert-id: "${your-cacert-id}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert: "on"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set a TLS security policy

Set all three annotations together.

Annotation	Required	Description	Default	Min CCM version
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port`	Yes	The listener protocol and port. For example, `TCP:80,TCPSSL:443`.	None	2.5.0
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id`	Yes	The ID of the server certificate. View it in the Certificate Management Service consoleCertificate Management Service consoleCertificate Management Service consoleCertificate Management Service console.	None	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy`	No	The TLS security policy. System and custom policies are both supported. Valid values: `tls_cipher_policy_1_0`, `tls_cipher_policy_1_1`, `tls_cipher_policy_1_2`, `tls_cipher_policy_1_2_strict`, `tls_cipher_policy_1_2_strict_with_1_3`.	`tls_cipher_policy_1_0`	—

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    # Chinese mainland regions: ${your-cert-id}-cn-hangzhou
    # Regions outside the Chinese mainland: ${your-cert-id}-ap-southeast-1
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${the-combined-certificate-id}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy: "tls_cipher_policy_1_0"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set Proxy Protocol

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol

Description	Default	Min CCM version
Passes client IP addresses to backend servers over Proxy Protocol. `on`: enabled. `off`: disabled.	`off`	2.5.0

Important

Before enabling Proxy Protocol, ensure that Proxy Protocol v2 is enabled on the backend service. Otherwise, access requests will fail.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol: "on"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Carry additional information over Proxy Protocol

Use proxy-protocol together with one or more of the following annotations to pass PrivateLink metadata to backend servers.

Annotation	Required	Description	Default	Min CCM version
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol`	Yes	Enables Proxy Protocol. Must be set to `on` for the other annotations to take effect.	`off`	2.5.0
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-vpc-id-enabled`	No	Passes the VpcId to backend servers over Proxy Protocol. `on`: enabled. `off`: disabled.	`off`	2.9.1
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-ep-id-enabled`	No	Passes the PrivateLinkEpId to backend servers over Proxy Protocol. `on`: enabled. `off`: disabled.	`off`	2.9.1
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-eps-id-enabled`	No	Passes the PrivateLinkEpsId to backend servers over Proxy Protocol. `on`: enabled. `off`: disabled.	`off`	2.9.1

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-ep-id-enabled: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-eps-id-enabled: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-vpc-id-enabled: "on"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set the maximum new connections per second

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cps

Description	Default	Min CCM version
The maximum number of new connections per second on the NLB instance. Valid values: 0–1,000,000. `0` means unlimited.	None	2.5.0

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cps: "100"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set the idle connection timeout

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout

Description	Default	Min CCM version
The idle connection timeout period in seconds. Valid values: 10–900.	900	2.5.0

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout: "60"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure a listener port range

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-listener-port-range

Description Min CCM version

Configures all-port forwarding on the NLB instance. The listener monitors a port range and forwards traffic to the corresponding backend server ports. Format: <port-range>:<service-port>. Separate multiple entries with commas, for example, 80-100:80,400-500:443. Port ranges for the same protocol cannot overlap. targetPort must be an integer from 1 to 65535 and is used as the health check port on the backend server. 2.11.4

Description	Min CCM version
Configures all-port forwarding on the NLB instance. The listener monitors a port range and forwards traffic to the corresponding backend server ports. Format: `<port-range>:<service-port>`. Separate multiple entries with commas, for example, `80-100:80,400-500:443`. Port ranges for the same protocol cannot overlap. `targetPort` must be an integer from 1 to 65535 and is used as the health check port on the backend server.	2.11.4

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-listener-port-range: "80-100:80,400-500:443"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80 # Health check port on the backend server. Must be an integer.
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Server group operations

Set a scheduling algorithm

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler

Description	Default	Min CCM version
The scheduling algorithm for distributing traffic to backend servers. Valid values: `wrr` (weighted round-robin — servers with higher weights receive more requests), `rr` (round-robin — requests are distributed in sequence), `sch` (source IP hash — requests from the same source IP go to the same server), `tch` (four-tuple hash — consistent hash based on source IP, destination IP, source port, and destination port), `wlc` (weighted least connections — the server with the fewest active connections relative to its weight receives the next request). For details, see the `Scheduler` field in CreateServerGroup.	`wrr`	2.5.0

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "sch"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure connection draining

Set both annotations together.

Annotation	Required	Description	Default	Min CCM version
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain`	Yes	Enables connection draining. `on`: enabled. `off`: disabled.	`off`	2.5.0
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout`	Yes	The connection draining timeout period in seconds. Valid values: 10–900.	None	—

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout: "30"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Enable client IP preservation

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-client-ip

Description	Default	Min CCM version
Preserves the original client IP address when forwarding traffic to backend servers. `on`: enabled. `off`: disabled.	`on`	2.5.0

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-client-ip: "on"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure TCP health checks

TCP health checks are enabled by default. Use the following annotations to customize the behavior.

Annotation	Required	Description	Default	Min CCM version
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag`	No	Enables or disables health checks. `on`: enabled. `off`: disabled.	`on`	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type`	No	The health check protocol. `tcp` or `http`.	`tcp`	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-port`	No	The backend server port used for health checks. Valid values: 0–65535. `0` uses the backend server port.	`0`	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout`	No	The maximum response timeout for a health check in seconds. Valid values: 1–300.	`5`	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold`	No	The number of consecutive successful checks before a backend server is declared healthy. Valid values: 2–10.	`2`	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold`	No	The number of consecutive failed checks before a backend server is declared unhealthy. Valid values: 2–10.	`2`	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval`	No	The interval between consecutive health checks in seconds. Valid values: 1–50.	`10`	—

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "tcp"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "8"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure HTTP health checks

TCP health checks are enabled by default. To switch to HTTP health checks, set health-check-type to http and configure the required annotations below.

Annotation	Required	Description	Default	Min CCM version
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag`	No	Enables or disables health checks. `on`: enabled. `off`: disabled.	`on`	2.5.0
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type`	Yes	Set to `http` to use HTTP health checks.	`tcp`	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri`	Yes	The path for HTTP health checks. Must start with `/`, be 1–80 characters, and may contain letters, digits, and special characters. See CreateServerGroup for valid characters. Takes effect only when `health-check-type` is `http`.	None	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-domain`	Yes	The domain name for HTTP health checks. `$_ip`: uses the backend server's private IP. Or specify a domain name of 1–80 characters containing lowercase letters, digits, hyphens, and periods. Takes effect only when `health-check-type` is `http`.	None	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-port`	No	The backend server port used for health checks. Valid values: 0–65535. `0` uses the backend server port.	`0`	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout`	No	The maximum response timeout for a health check in seconds. Valid values: 1–300.	`5`	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold`	No	The number of consecutive successful checks before a backend server is declared healthy. Valid values: 2–10.	`2`	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold`	No	The number of consecutive failed checks before a backend server is declared unhealthy. Valid values: 2–10.	`2`	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval`	No	The interval between consecutive health checks in seconds. Valid values: 1–50.	`10`	—
`service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method`	No	The HTTP method for health checks. `GET` or `HEAD`. Takes effect only when `health-check-type` is `http`.	None	—

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "http"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri: "/test/index.html"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-domain: "www.test.com"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "10"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method: "head" # Optional.
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set the server group type

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-server-group-type

Description	Default	Min CCM version
The server group type. `Instance`: instance type, supporting ECS and ENI backends. `Ip`: IP address type, supporting backend servers added directly by IP address. For details, see NLB server groups.	`Instance`	2.8.0

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-server-group-type: "Ip"
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Reuse an existing server group

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vgroup-port

Reuses an existing server group. This annotation takes effect only when reusing an existing NLB instance. For a usage example, see Deploy services across clusters by reusing an existing SLB instance.

Set the traffic weight for a service

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight

Sets the traffic weight for the current service when multiple services share the same NLB instance. This annotation takes effect only when reusing an existing server group. For a usage example, see Deploy services across clusters by reusing an existing SLB instance.

Ignore backend server weight updates

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ignore-weight-update

Description	Min CCM version
Skips updating backend server weights in the server group during service synchronization. Use this annotation when managing backend server weights manually. `on`: skip weight updates. `off`: update weights normally. Default: `off`.	2.11.1

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ignore-weight-update: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer