Supported annotations for NLB-Container Service for Kubernetes(ACK)-阿里云帮助中心

You can use annotations in a Service's YAML file to enable additional load balancing features for its Network Load Balancer (NLB), such as specifying the NLB network type, enabling modification protection, and configuring mutual authentication. This topic explains how to use these annotations to configure the NLB. The configurations are organized by three resource types: the NLB instance, listeners, and server groups.

Index

Category	Feature category	Configuration link
Prerequisites
NLB operations	Creation	Create a public NLB Create a private NLB Create a dual-stack NLB Specify the network type of an IPv6 address Use an existing load balancer
NLB operations	Instance configuration	Specify the load balancer name Specify a resource group for the load balancer Add tags to the load balancer Bind a shared bandwidth package Configure cross-zone forwarding Configure an IP whitelist
Listener operations	Creation	Create a TCP listener Create a UDP listener Create a TCP/SSL listener Configure a listener port range
	Listener configuration	Configure a listener for both TCP and UDP Set the per-second new connection rate limit Set Proxy Protocol Send additional information with Proxy Protocol Set the idle connection timeout Configure an ALPN policy Configure additional certificates
	Security configuration	Enable mutual authentication Set a TLS security policy Set a security group for a listener
Server group operations	Server configuration	Configure a scheduling algorithm Configure connection draining Configure client IP preservation Configure TCP health checks Configure HTTP health checks Configure the server group type Add IPv6 backend servers Reuse an existing server group Set the traffic weight for services Ignore backend server weight updates Set the default backend server weight

Notes

Ensure your cluster is version v1.24 or later and the Cloud Controller Manager (CCM) add-on is version v2.5.0 or later. To upgrade the cluster, see Manually upgrade a cluster; to upgrade the add-on, see Manage add-ons.
In the Service definition, set spec.loadBalancerClass to alibabacloud.com/nlb. If this parameter is not specified, a Classic Load Balancer (CLB) is created by default.
Once a Service is created, you cannot change the spec.loadBalancerClass field. You cannot convert the load balancer type from CLB to NLB or vice versa.
You cannot manage NLB instances in the ACK console. You must use kubectl commands to perform all management operations.

NLB operations

Create a public NLB

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps

Description

Supported CCM versions

Specifies the zone, vSwitch, private IP address, and EIP information for the NLB. The format is as follows:

{zone}:{vSwitchID}:{privateIPAddress}:{EIPInstanceID}. Separate multiple entries with a comma (,).

Zone and vSwitch ID: Required. Specify at least two, for example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
For a list of supported regions and zones for NLB, see the NLB console.
Private IP address: Optional.
EIP instance ID: Optional.

Zones and vSwitch IDs: v2.5.0 or later

private IP addresses and EIPs: v2.12.1 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a private NLB

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type

Description

Supported CCM versions

Specifies the network type of the NLB. You can change the value of this annotation to switch the NLB between public and private. Valid values:

internet: a public NLB.
intranet: a private NLB.

Default: internet

You can log on to the NLB console to view the supported regions and zones. At least two zones are required. Multiple zones are separated by commas, for example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.

v2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify a private IP address

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps

Description

Supported CCM versions

Specifies the zone, vSwitch, private IP address, and EIP information for the NLB. The format is as follows:

{zone}:{vSwitchID}:{privateIPAddress}:{EIPInstanceID}. Separate multiple entries with a comma (,).

Zone and vSwitch ID: Required. Specify at least two pairs. For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
For a list of supported regions and zones for NLB, see the NLB console.
Private IP address: Optional.
EIP instance ID: Optional.

Zones and vSwitch IDs: v2.5.0 or later
private IP addresses and EIPs: v2.12.1 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    # For example, cn-hangzhou-k:vsw-i123456:10.1.0.1,cn-hangzhou-j:vsw-j654321:10.2.0.1
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A}:${private-ipv4-A},${zone-B}:${vsw-B}:${private-ipv4-B}"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify an EIP instance ID

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps

Description

Supported CCM versions

Specifies the zone, vSwitch, private IP address, and EIP information for the NLB. The format is as follows:

{zone}:{vSwitchID}:{privateIPAddress}:{EIPInstanceID}. Separate multiple entries with a comma (,).

Zones and vSwitch IDs: Required. Specify at least two pairs, such as cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
For a list of supported regions and zones for NLB, see the NLB console.
Private IP address: Optional.
EIP instance ID: Optional.

Zones and vSwitch IDs: v2.5.0 or later

private IP addresses and EIPs: v2.12.1 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    # If you do not need a private IP address, leave the private IP address field empty.
    # For example, cn-hangzhou-k:vsw-i123456::eip-12345,cn-hangzhou-j:vsw-j654321::eip-54321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A}::${eip-A},${zone-B}:${vsw-B}::${eip-B}"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify the load balancer name

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name

Description	Supported CCM versions
Specifies the name of the load balancer instance. The name must be 2 to 128 characters in length, begin with a letter or a Chinese character and can contain digits, periods (.), underscores (_), and hyphens (-).	v2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name: "${your-nlb-name}" # The name of the NLB instance.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify a resource group

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id

Description

Supported CCM versions

Specifies the resource group for the load balancer. The resource group ID cannot be changed after it is set.

You can find the resource group ID in the Resource Management console.

v2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id:  "${your-resource-group-id}" # The ID of the resource group.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a dual-stack NLB

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version

To attach IPv6 backend servers, see Attach IPv6 backend servers.

Description

Supported CCM versions

The kube-proxy mode of the cluster must be IPVS.

Specifies the IP protocol version of the NLB. The IP protocol version cannot be changed after creation. Valid values:

ipv4: IPv4.
DualStack: Supports both IPv4 and IPv6.
- The two vSwitches specified in service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps must both have IPv6 enabled.
- The generated IPv6 address is accessible only from IPv6-enabled environments.

Default: ipv4

v2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version: "DualStack"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Add additional tags

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags

Description	Supported CCM versions
You can add additional tags. Use a comma (,) to separate multiple tags. For example, `k1=v1,k2=v2`. In versions 2.10.0 and later, you can modify the tags of existing and reused instances. Important After you add this annotation, any manual tag modifications made to the load balancer instance in the console will be overwritten.	v2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags: "Key1=Value1,Key2=Value2"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Use an existing load balancer

Important

Do not add or modify the reuse annotation for an existing LoadBalancer Service. Doing so may cause the reuse to fail or prevent the CCM-created load balancer from being released.

Annotation: Multiple annotations are used, as described in the following table.

Annotation

Description

Supported CCM versions

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id

Specifies the ID of the existing load balancer.

v2.5.0 or later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners

Specifies whether to synchronize NLB listener configurations based on the Service configuration. Valid values:

true: CCM creates, updates, and deletes NLB listeners based on the Service configuration.
false: CCM does not process NLB listeners.

Default: false

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "${your-nlb-id}" # The ID of the NLB instance.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners: "true"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Bind a shared bandwidth package

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth-package-id

Description

Supported CCM versions

Specifies the ID of the shared bandwidth package to bind.

You can find the ID of the shared bandwidth package in the VPC console.

v2.9.1 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth-package-id: "cbwp-xxxxxxxxxx" 
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure cross-zone forwarding

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cross-zone-enabled

Description

Supported CCM versions

Specifies whether to enable cross-zone forwarding. Valid values:

on
off

Default: on

v2.13.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cross-zone-enabled: "off" 
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure a source IP address allowlist

Field: .spec.loadBalancerRanges

Description

Supported CCM versions

This feature applies only to NLB instances that are created and managed by CCM. It is not supported for reused instances.

When you configure this field, CCM automatically creates a security group, sets the required access rules, and associates the security group with the NLB instance. If you do not configure this field or leave it empty, no security group is associated.

Default: Empty

Important

Adding or modifying this field overwrites the existing security group associations on the NLB. If other security groups are manually associated with the NLB, proceed with caution.
Do not modify the security group or its tags created by CCM. Otherwise, the association may be removed or the rules overwritten.

v2.14.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
  name: nginx
  namespace: default
spec:
  # Specify the allowed source IP address CIDR blocks.
  loadBalancerSourceRanges:
    - 10.0.0.0/8
    - 172.16.0.0/16
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify IPv6 network type

Annotation: Multiple annotations are used, as described in the following table.

Annotation

Description

Supported CCM versions

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version

The kube-proxy mode of the cluster must be IPVS. This annotation specifies the IP protocol version for the NLB. The IP protocol version cannot be changed after creation. Valid values:

ipv4: IPv4.
DualStack: Supports both IPv4 and IPv6.
- Both vSwitches specified in service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps must have IPv6 enabled.
- The generated IPv6 address is accessible only from IPv6-enabled environments.

Default: ipv4

v2.5.0 or later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ipv6-address-type

Specifies the network type of the IPv6 address of the NLB. Valid values:

intranet: private IPv6.
internet: public IPv6.

Default: intranet

Note

To use a public IPv6 address, an IPv6 gateway must exist in the VPC where the NLB instance is deployed. For more information, see Create and manage an IPv6 gateway.

v2.9.1 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version: "DualStack"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ipv6-address-type: internet # Specify the IPv6 network type as public.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Listener operations

Set a listener security group

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-security-group-ids

Description	Supported CCM versions
Specifies the IDs of the security groups to associate with the listener. Separate multiple IDs with a comma (,), for example, `sg-aaaaa,sg-bbbbb`.	v2.6.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-security-group-ids: "sg-aaaaa,sg-bbbbb" # Separate multiple security group IDs with a comma (,).
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure a TCP and UDP listener

Description	Supported CCM versions
This feature requires Kubernetes v1.24 or later. For more information about upgrading a cluster, see Upgrade an ACK Kubernetes cluster.	N/A

apiVersion: v1
kind: Service
metadata:
  annotations:
      service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: udp
    port: 80
    protocol: UDP
    targetPort: 81
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a TCP listener

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a UDP listener

Note

If you do not specify a health check when creating a UDP listener, a TCP health check is enabled by default for the server group. For UDP listeners, we recommend using an annotation to explicitly specify a UDP health check or to disable health checks.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: udp
    port: 80
    protocol: UDP
    targetPort: 80
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a TCPSSL listener

Annotation: Refer to the following table for the required annotations.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Specifies the protocol of the listener. To specify multiple protocols, separate them with a comma (,), for example, `TCP:80,TCPSSL:443`.	v2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	To get the ID, log on to the Certificate Management Service console, create a certificate, and find the ID on the SSL Certificate Management page. The following figure shows an example:	v2.5.0 or later

Important

Currently, TCP/SSL listeners do not support server groups with client IP address persistence enabled. This means that you cannot configure the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:${port}" and service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-client-ip: "on" annotations at the same time. If you need to obtain client IP addresses through NLB, see Obtain real client IP addresses on backend servers through NLB.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${CertIdentifier}"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure a listener port range

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-listener-port-range

Description

Supported CCM versions

This feature requires the Terway network plugin.

You can configure a listener to listen on a specified port range. The NLB instance then forwards traffic from this port range to the same port range on the backend servers. For example, if you configure the 80-100 port range, traffic to ports 80 to 100 on the NLB instance is forwarded to ports 80 to 100 on the backend servers.

The format is port-range:service-port. Separate multiple values with a comma (,), for example, 80-100:80,400-500:443. Ports and port ranges for the same protocol cannot overlap.

The targetPort specifies the health check port for the server group and must be an integer from 1 to 65535.

v2.11.4 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    # Configure listeners for the 80-100 and 400-500 port ranges.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-listener-port-range: "80-100:80,400-500:443"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    # This port configuration is mapped to the 80-100 range.
    port: 80
    protocol: TCP
    # The targetPort specifies the health check port for the backend servers and must be an integer.
    targetPort: 80
  - name: https
    # This port configuration is mapped to the 400-500 range.
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Enable mutual authentication

Annotation: Refer to the following table for the required annotations.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Specifies the protocol of the listener. To specify multiple protocols, separate them with a comma (,), for example, `TCP:80,TCPSSL:443`.	v2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	To get the ID, log on to the Certificate Management Service console, create a certificate, and find the ID on the SSL Certificate Management page. The following figure shows an example:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert-id	To get the ID, log on to the Certificate Management Service console and find the ID in the certificate details on the PCA Certificate Management page.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert	Specifies whether to enable mutual authentication. Valid values: `on`: Enables mutual authentication. `off`: Disables mutual authentication. Default value: `off`

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"   
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${CertIdentifier}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert-id: "${your-cacert-id}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set a TLS security policy

Annotation: Refer to the following table for the required annotations.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Specifies the protocol of the listener. To specify multiple protocols, separate them with a comma (,), for example, `TCP:80,TCPSSL:443`.	v2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	To get the ID, log on to the Certificate Management Service console, create a certificate, and find the ID on the SSL Certificate Management page. The following figure shows an example:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy	The ID of the TLS security policy. You can use system-defined or custom security policies. Valid values: `tls_cipher_policy_1_0` `tls_cipher_policy_1_1` `tls_cipher_policy_1_2` `tls_cipher_policy_1_2_strict` `tls_cipher_policy_1_2_strict_with_1_3` Default value: `tls_cipher_policy_1_0`

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${CertIdentifier}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy: "tls_cipher_policy_1_0"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Enable Proxy Protocol

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol

Description

Supported CCM versions

Specifies whether to use the Proxy Protocol to pass client IP addresses to backend servers. Valid values:

on: Enables the feature.
off: Disables the feature.

Default value: off

Important

Before you enable the Proxy Protocol, ensure your backend services support Proxy Protocol v2. Otherwise, connections will fail. Configure this parameter with caution.

v2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Pass additional information via Proxy Protocol

Annotation: Refer to the following table for the required annotations.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol	Specifies whether to use the Proxy Protocol to pass client IP addresses to backend servers. Valid values: `on`: Enables the feature. `off`: Disables the feature. Default value: `off` Important Before you enable the Proxy Protocol, ensure your backend services support Proxy Protocol v2. Otherwise, connections will fail. Configure this parameter with caution.	v2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-vpc-id-enabled	Specifies whether to use the Proxy Protocol to pass the VpcId to backend servers. Valid values: `on`: Enables the feature. `off`: Disables the feature. Default value: `off`	v2.9.1 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-ep-id-enabled	Specifies whether to use the Proxy Protocol to pass the PrivateLinkEpId to backend servers. Valid values: `on`: Enables the feature. `off`: Disables the feature. Default value: `off`	v2.9.1 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-eps-id-enabled	Specifies whether to use the Proxy Protocol to pass the PrivateLinkEpsId to backend servers. Valid values: `on`: Enables the feature. `off`: Disables the feature. Default value: `off`	v2.9.1 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-ep-id-enabled: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-eps-id-enabled: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-vpc-id-enabled: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set a listener connection rate limit

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cps

Description	Supported CCM versions
The maximum number of new connections per second for a Network Load Balancer (NLB) instance. The value must be an integer from 0 to 1,000,000. A value of 0 indicates no limit.	v2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cps: "100"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set a listener idle timeout

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout

Description

Supported CCM versions

The idle connection timeout in seconds. Valid values: 10 to 900.

Default value: 900

v2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout: "60"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set an ALPN policy

Annotation: Refer to the following table for the required annotations.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Specifies the protocol of the listener. To specify multiple protocols, separate them with a comma (,), for example, `TCP:80,TCPSSL:443`.	v2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	To get the ID, log on to the Certificate Management Service console, create a certificate, and find the ID on the SSL Certificate Management page. The following figure shows an example:	v2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-alpn	Specifies whether to enable Application-Layer Protocol Negotiation (ALPN). Valid values: `on`: Enables ALPN. `off`: Disables ALPN. Default value: `off`	v2.10.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-alpn-policy	The ALPN policy. Valid values: `HTTP1Only`: Negotiates only the HTTP/1.x protocol. Priority: HTTP/1.1 > HTTP/1.0. `HTTP2Only`: Negotiates only the HTTP/2.0 protocol. `HTTP2Optional`: Prioritizes the HTTP/1.x protocol but also accepts the HTTP/2.0 protocol. Priority: HTTP/1.1 > HTTP/1.0 > HTTP/2.0. `HTTP2Preferred`: Prioritizes the HTTP/2 protocol but also accepts the HTTP/1.x protocol. Priority: HTTP/2.0 > HTTP/1.1 > HTTP/1.0. For more information, see the `AlpnPolicy` parameter in the CreateListener API reference.	v2.10.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${CertIdentifier}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-alpn: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-alpn-policy: "HTTP1Only" 
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure additional certificates

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Specifies the protocol of the listener. To specify multiple protocols, separate them with a comma (,), for example, `TCP:80,TCPSSL:443`.	v2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	To get the ID, log on to the Certificate Management Service console, create a certificate, and find the ID on the SSL Certificate Management page. The following figure shows an example:	v2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-cert-ids	The IDs of additional certificates. Separate multiple certificate IDs with a comma (,). To get the IDs, log on to the Certificate Management Service console, create the certificates, and find their IDs on the SSL Certificate Management page. The following figure shows an example:	v2.13.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${CertIdentifier}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-cert-ids: "${CertIdentifier-1},${CertIdentifier-2}" # Separate multiple additional certificate IDs with a comma (,).
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Typical operations for server groups

Scheduling algorithm

annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler

Description

Supported CCM versions

The scheduling algorithm. Valid values:

wrr: weighted round robin. Backend servers with a higher weight receive more requests.
rr: round robin. Requests are distributed to backend servers in sequence.
sch: source IP hash. Requests from the same source IP address are always routed to the same backend server.
tch: four-tuple hash. Requests are routed based on a consistent hash of a four-tuple (source IP address, destination IP address, source port, and destination port). This ensures that traffic from the same flow is always directed to the same backend server.
wlc: weighted least connections. Requests are distributed based on both the weight and the current load (number of active connections) of each backend server. If backend servers have the same weight, the one with the fewest active connections is more likely to receive the next request.

Default value: wrr

For more information about the values of this annotation, see the Scheduler parameter in the CreateServerGroup API reference.

v2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "sch"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Connection draining

annotation: Multiple annotations are available, as described in the following table.

Annotation

Description

Supported CCM versions

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain

Specifies whether to enable connection draining. If enabled, the following behavior occurs when a backend server is removed or fails health checks:

on: Keeps existing connections active for the specified timeout, allowing them to close gracefully.
off: The load balancer does not terminate existing connections. They close only when the client disconnects or the session expires.

Default value: off

v2.5.0 or later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout

The timeout period for connection draining.

Unit: seconds.
Valid values: 0 to 900. A value of 0 terminates connections immediately.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout: "30"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Client IP preservation

annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-client-ip

Description

Supported CCM versions

Specifies whether to enable client IP preservation. Valid values:

on: Enables the feature.
off: Disables the feature.

Default value: on

v2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-client-ip: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

TCP health checks

annotation: Multiple annotations are available, as described in the following table. To configure TCP health checks, all of the following annotations are required. By default, health checks are enabled for TCP ports.

Annotation	Description
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag	Specifies whether to enable health checks. Valid values: `on`: Enables health checks. `off`: Disables health checks. Default value: `on`
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type	The health check protocol. Valid values: `tcp` `udp` `http` Default value: `tcp`
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-port	The port on the backend server used for health checks. Valid values: `0` to `65535`. The default value `0` indicates that the backend server's port is used.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout	The maximum time to wait for a health check response. Unit: seconds. Valid values: `1` to `300`. Default value: 5
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold	The number of consecutive successful health checks required for a backend server to be considered healthy. Valid values: `2` to `10`. Default value: 2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold	The number of consecutive failed health checks required for a backend server to be considered unhealthy. Valid values: `2` to `10`. Default value: 2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval	The interval between consecutive health checks. Unit: seconds. Valid values: `1` to `50`. Default value: 5

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "tcp"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "8"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

UDP health checks

annotation: Multiple annotations are available, and all are required. To use UDP health checks, ensure that all ports defined in the Service use the UDP protocol.

Annotation	Description
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag	Specifies whether to enable health checks. Valid values: `on`: Enables health checks. `off`: Disables health checks. Default value: `on`
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type	The health check protocol. Valid values: `tcp` `udp` `http` Default value: `tcp`
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-port	The server port that is used for health checks. Valid values: `[0, 65535]`. The default value is `0`, which indicates that the actual port of the backend server is used.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout	The response timeout for a single health check. If a timeout occurs, the current check is considered a failure. Unit: seconds. Value range: `[1, 300]`. Default value: `5`.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold	A backend server is considered healthy after it passes the specified number of consecutive health checks. Valid values: `[2, 10]`. Default value: `2`.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold	Default value: 2 The backend server is considered unhealthy after this number of consecutive health check failures. Valid values: `[2, 10]`. Default value: `2`.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval	The interval between consecutive health checks. Unit: seconds. Value range: `[1, 300]`. Default value: `5`. This value must be greater than the value of `health-check-connect-timeout`. Otherwise, the health check may not function correctly.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "udp"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "3"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: udp
    port: 80
    protocol: UDP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

HTTP health checks

annotation: Multiple annotations are available, as described in the following table. To configure HTTP health checks, all of the following annotations are required. By default, health checks use the TCP protocol.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag	Specifies whether to enable health checks. Valid values: `on`: Enables health checks. `off`: Disables health checks. Default value: `on`	v2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type	The health check protocol. Valid values: `tcp` `udp` `http` Default value: `tcp`
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri	The path for the health check. It must be 1 to 80 characters long, start with a forward slash (`/`), and can contain letters, digits, and the following special characters: `- . _ % / ? # &`. For more information, see CreateServerGroup. Note This parameter is valid only when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type` is set to `http`.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-domain	The domain name used for health checks. Valid values: $SERVER_IP: The private IP address of the backend server. domain: A specific domain name. The domain name must be 1 to 80 characters in length and can contain only lowercase letters, digits, hyphens (-), and periods (.). Note This parameter is valid only when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type` is set to `http`.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-port	The port on the backend server used for health checks. Valid values: `0` to `65535`. The default value `0` indicates that the backend server's port is used.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout	The maximum time to wait for a health check response. Unit: seconds. Valid values: `1` to `300`. Default value: 5
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold	The number of consecutive successful health checks required for a backend server to be considered healthy. Valid values: `2` to `10`. Default value: 2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold	The number of consecutive failed health checks required for a backend server to be considered unhealthy. Valid values: `2` to `10`. Default value: 2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval	The interval between consecutive health checks. Unit: seconds. Valid values: `1` to `50`. Default value: 5
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method	The health check method. Valid values: `GET` `HEAD` Note This parameter is valid only when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type` is set to `http`.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "http"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri: "/test/index.html"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-domain: "www.test.com"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "10"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
    # Set the health check method. This annotation is optional.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method: "head"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Server group type

annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-server-group-type

Description

Supported CCM versions

The type of the server group. Valid values:

Ip: IP address. You can add backend servers by specifying their IP addresses.
Instance (default): You can add ECS instances or elastic network interfaces (ENIs) as backend servers.

Default value: Instance

For more information about NLB server group types, see Server groups.

v2.8.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-server-group-type: "Ip"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

IPv6 backend servers

annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-backend-ip-version

Description

Supported CCM versions

The IP version of the backend servers in the server group.

ipv4: The server group contains only IPv4 backend servers.
ipv6: The server group contains only IPv6 backend servers.
DualStack: The server group contains both IPv4 and IPv6 backend servers, and IP version affinity is automatically enabled.

Default value: ipv4

Note

When you add both IPv4 and IPv6 backend servers to a server group, pod quota usage is doubled.

ipv6: v2.9.1 or later
DualStack: v2.14.0 or later

Reusing a server group

annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vgroup-port

This annotation allows you to reuse an existing server group. It takes effect only when you reuse an existing NLB instance. For an example, see Deploy services across clusters by reusing a load balancer.

Service traffic weight

annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight

In scenarios where multiple Services reuse the same NLB instance, you can use this annotation to set the traffic weight for the current Service. This annotation takes effect only when you reuse an existing server group. For an example, see Deploy services across clusters by reusing a load balancer.

Ignoring weight updates

annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ignore-weight-update

Description

Supported CCM versions

When set to on, the CCM does not update the weights of backend servers in the server group during Service synchronization. This is useful if you manage backend server weights by using other mechanisms.

on
off

Default value: off

v2.11.1 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ignore-weight-update: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Default backend server weight

annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-default-weight

Description

Supported CCM versions

The default weight of backend servers in the server group. A higher weight means the server receives a larger share of incoming traffic.

The amount increases.

Valid values: 0 to 100. If this annotation is not configured, the system uses a default weight based on the network type.

This configuration does not take effect when the backend instance is an ECS instance and the externalTrafficPolicy of the Service is set to Local.

v2.14.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    # Sets the default weight of servers in the server group to 35.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-default-weight: "35"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer