MSE Ingress高级用法

在Kubernetes集群中,MSE Ingress对集群服务(Service)中的外部可访问的API对象进行管理,提供七层负载均衡能力。本文介绍MSE Ingress的高级用法,方便您对集群入口流量进行治理。

索引

您可以单击功能名称,查看详细说明。

灰度发布

MSE Ingress提供复杂的路由处理能力,支持基于Header、Query Parameter、Cookie以及权重的灰度发布功能。灰度发布功能可以通过设置注解来实现,为了启用灰度发布功能,需要设置注解nginx.ingress.kubernetes.io/canary: "true",通过不同注解可以实现不同的灰度发布功能。

说明

当多种方式同时配置时,灰度方式选择优先级为:基于Header | 基于Query Parameter > 基于Cookie > 基于权重(从高到低)。

基于Header灰度发布

  • 只配置nginx.ingress.kubernetes.io/canary-by-header:基于Request Header的流量切分,当配置的header值为always时,请求流量会被分配到灰度服务入口;其他情况时,请求流量不会分配到灰度服务。

  • 同时配置nginx.ingress.kubernetes.io/canary-by-header-value和nginx.ingress.kubernetes.io/canary-by-header:当请求中的header和header-value与设置的值匹配时,请求流量会被分配到灰度服务;其他情况时,请求流量不会分配到灰度服务。

说明

相比Nginx Ingress和ALB Ingress灰度发布时最多只支持两个版本服务,MSE Ingress灰度发布时支持多个版本服务(无上限)。

例如:

  • 请求Header为mse:always时会访问灰度服务demo-service-canary;其他情况将访问正式服务demo-service。配置如下。

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
      name: demo-canary
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service-canary
                  servicePort: 80
    ---
    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
      name: demo-canary
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service-canary
                    port: 
                      number: 80
                path: /hello
                pathType: Exact
    ---
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /hello
                pathType: Exact          
  • 请求Header为mse:v1时将访问灰度服务demo-service-canary-v1;请求Header为mse:v2时将访问灰度服务demo-service-canary-v2;其他情况将访问正式服务demo-service。配置如下。

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
        nginx.ingress.kubernetes.io/canary-by-header-value: "v1"
      name: demo-canary-v1
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service-canary-v1
                  servicePort: 80
    ---
    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
        nginx.ingress.kubernetes.io/canary-by-header-value: "v2"
      name: demo-canary-v2
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service-canary-v2
                  servicePort: 80
    ---
    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
        nginx.ingress.kubernetes.io/canary-by-header-value: "v1"
      name: demo-canary-v1
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service-canary-v1
                    port: 
                      number: 80
                path: /hello
                pathType: Exact
    ---
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
        nginx.ingress.kubernetes.io/canary-by-header-value: "v2"
      name: demo-canary-v2
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service-canary-v2
                    port: 
                      number: 80
                path: /hello
                pathType: Exact
    ---
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /hello
                pathType: Exact

基于Query Parameter灰度发布

  • 仅配置mse.ingress.kubernetes.io/canary-by-query

    基于URL Query Parameter的流量切分,当请求的URL中Query Parameter的Key为该参数配置且Value为always时,请求流量会被分配到灰度服务入口。其他情况下,请求流量不会分配到灰度服务。

  • 同时配置mse.ingress.kubernetes.io/canary-by-query-value和mse.ingress.kubernetes.io/canary-by-query

    当请求中的query parameter keyquery parameter value与设置的值匹配时,请求流量会被分配到灰度服务。其他情况下,请求流量不会分配到灰度服务。

    说明

    基于Header的灰度发布可以和基于Query Parameter的灰度发布一起使用,同时满足匹配条件,请求流量才会被分配到灰度服务。

示例:

  • 请求URL的Query Parameter为canary:gray时会访问灰度服务demo-service-canary,其他情况将访问正式服务demo-service。相关配置如下。

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    mse.ingress.kubernetes.io/canary-by-query: "canary"
    mse.ingress.kubernetes.io/canary-by-query-value: "gray"
  name: demo-canary
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /hello
            backend:
              serviceName: demo-service-canary
              servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: demo
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /hello
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion:networking.k8s.io/v1 
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    mse.ingress.kubernetes.io/canary-by-query: "canary"
    mse.ingress.kubernetes.io/canary-by-query-value: "gray"
  name: demo-canary
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: demo-service-canary
                port: 
                  number: 80
            path: /hello
            pathType: Exact
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /hello
            pathType: Exact 
  • 请求URL的Query Parameter为canary:gray,同时请求Header包含x-user-id: test时,会访问灰度服务demo-service-canary,其他情况将访问正式服务demo-service。相关配置如下。

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        mse.ingress.kubernetes.io/canary-by-query: "canary"
        mse.ingress.kubernetes.io/canary-by-query-value: "gray"
        nginx.ingress.kubernetes.io/canary-by-header: "x-user-id"
        nginx.ingress.kubernetes.io/canary-by-header-value: "test"
      name: demo-canary
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service-canary
                  servicePort: 80
    ---
    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion:networking.k8s.io/v1 
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        mse.ingress.kubernetes.io/canary-by-query: "canary"
        mse.ingress.kubernetes.io/canary-by-query-value: "gray"
        nginx.ingress.kubernetes.io/canary-by-header: "x-user-id"
        nginx.ingress.kubernetes.io/canary-by-header-value: "test"
      name: demo-canary
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service-canary
                    port: 
                      number: 80
                path: /hello
                pathType: Exact
    ---
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /hello
                pathType: Exact 

基于Cookie灰度发布

nginx.ingress.kubernetes.io/canary-by-cookie:基于Cookie的流量切分,当配置的cookie值为always时,请求流量会被分配到灰度服务;其他情况时,请求流量将不会分配到灰度服务。

说明

基于Cookie的灰度发布不支持设置自定义值,配置的cookie值只能为always

例如,请求的Cookie为demo=always时会访问灰度服务demo-service-canary;其他情况将访问正式服务demo-service。配置如下:

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-cookie: "demo"
  name: demo-canary
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /hello
            backend:
              serviceName: demo-service-canary
              servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: demo
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /hello
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-cookie: "demo"
  name: demo-canary
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: demo-service-canary
                port: 
                  number: 80
            path: /hello
            pathType: Exact
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /hello
            pathType: Exact

基于权重灰度发布

注解

说明

nginx.ingress.kubernetes.io/canary-weight

设置请求到指定服务的百分比(值为0~100的整数)。

nginx.ingress.kubernetes.io/canary-weight-total

设置权重总和,默认为100。

例如,配置灰度服务demo-service-canary-v1的权重为30%,配置灰度服务demo-service-canary-v2的权重为20%,配置正式服务demo-service的权重为50%。配置如下:

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "30"
  name: demo-canary-v1
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /hello
            backend:
              serviceName: demo-service-canary-v1
              servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "20"
  name: demo-canary-v2
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /hello
            backend:
              serviceName: demo-service-canary-v2
              servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: demo
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /hello
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "30"
  name: demo-canary-v1
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: demo-service-canary-v1
                port: 
                  number: 80
            path: /hello
            pathType: Exact
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "20"
  name: demo-canary-v2
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: demo-service-canary-v2
                port: 
                  number: 80
            path: /hello
            pathType: Exact
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /hello
            pathType: Exact

跨域

跨域资源共享CORS(Cross-Origin Resource Sharing)是指允许Web应用服务器进行跨域访问控制,从而实现跨域数据安全传输。关于跨域的更多信息,请参见跨源资源共享(CORS)

注解

说明

nginx.ingress.kubernetes.io/enable-cors

开启或关闭跨域。

nginx.ingress.kubernetes.io/cors-allow-origin

允许的第三方站点,第三方站点之间使用英文逗号分隔,支持通配符*。默认值为*,即允许所有第三方站点。

nginx.ingress.kubernetes.io/cors-allow-methods

允许的请求方法,如GET、POST、PUT等,请求方法之间使用英文逗号分隔,支持通配符*。默认值为GET、PUT、POST、DELETE、PATCH、OPTIONS。

nginx.ingress.kubernetes.io/cors-allow-headers

允许的请求Header,Header之间使用英文逗号分隔,支持通配符*。默认值为DNT、X-CustomHeader、Keep-Alive、User-Agent、X-Requested-With、If-Modified-Since、Cache-Control、Content-Type、Authorization。

nginx.ingress.kubernetes.io/cors-expose-headers

允许暴露给浏览器的响应Header,响应Header之间使用英文逗号分隔。

nginx.ingress.kubernetes.io/cors-allow-credentials

是否允许携带凭证信息。默认允许。

nginx.ingress.kubernetes.io/cors-max-age

预检结果的最大缓存时间,单位为秒。默认值为1728000秒。

例如,跨域请求被限制为只能来自example.com域的请求,并且HTTP的请求方法只能是GET和POST,允许的请求头部为X-Foo-Bar,不允许携带凭证信息。配置如下:

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-origin: "example.com"
    nginx.ingress.kubernetes.io/cors-allow-methods: "GET,POST"
    nginx.ingress.kubernetes.io/cors-allow-headers: "X-Foo-Bar"
    nginx.ingress.kubernetes.io/cors-allow-credentials: "false"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - path: /hello
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-origin: "example.com"
    nginx.ingress.kubernetes.io/cors-allow-methods: "GET,POST"
    nginx.ingress.kubernetes.io/cors-allow-headers: "X-Foo-Bar"
    nginx.ingress.kubernetes.io/cors-allow-credentials: "false"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /hello
            pathType: Exact

Rewrite重写Path和Host

在请求转发给目标后端服务之前,重写可以修改原始请求的路径(Path)和主机域(Host)。

注解

说明

nginx.ingress.kubernetes.io/rewrite-target

重写Path,支持捕获组(Capture Group)。

nginx.ingress.kubernetes.io/upstream-vhost

重写Host。

Rewrite重写Path

例如:

  • 将请求example.com/test在转发至后端服务之前,重写为example.com/dev。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: "/dev"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: "/dev"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact
  • 将请求example.com/v1/app在转发至后端服务之前,去掉Path前缀/v1。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: "/$1"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /v1/(app)
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: "/$1"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /v1/(app)
                pathType: Prefix
  • 请求example.com/v1/app在转发至后端服务之前,把Path前缀/v1更改为/v2。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: "/v2/$1"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /v1/(app)
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/rewrite-target: "/v2/$1"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /v1/(app)
                pathType: Prefix

Rewrite重写Host

例如,把请求example.com/test在转发至后端服务之前,重写为test.com/test。配置如下:

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/upstream-vhost: "test.com"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com 
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/upstream-vhost: "test.com"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

重定向

通过重定向可以把原始客户端请求更改为目标请求。

配置HTTP重定向至HTTPS

注解

说明

nginx.ingress.kubernetes.io/ssl-redirect

HTTP重定向到HTTPS

nginx.ingress.kubernetes.io/force-ssl-redirect

HTTP重定向到HTTPS

说明

MSE Ingress对于以上两个注解不区分对待,都是强制将HTTP重定向到HTTPS。

例如,将请求http://example.com/test重定向为https://example.com/test。配置如下:

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com 
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

永久重定向

注解

说明

nginx.ingress.kubernetes.io/permanent-redirect

永久重定向的目标URL,必须包含Scheme(HTTP或HTTPS)。

nginx.ingress.kubernetes.io/permanent-redirect-code

永久重定向的HTTP状态码,默认值为301。

例如,把请求http://example.com/test永久重定向为http://example.com/app。配置如下:

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/permanent-redirect: "http://example.com/app"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com 
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/permanent-redirect: "http://example.com/app"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

临时重定向

nginx.ingress.kubernetes.io/temporal-redirect:临时重定向的目标URL,必须包含Scheme(HTTP或者HTTPS)。

例如,将请求http://example.com/test临时重定向为http://example.com/app。配置如下:

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/temporal-redirect: "http://example.com/app"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com 
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/temporal-redirect: "http://example.com/app"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

Header控制

通过Header控制,您可以在请求转发到后端服务之前对请求Header进行增删改,在收到响应转发给客户端时对响应Header进行增删改。

请求Header控制

注解

说明

mse.ingress.kubernetes.io/request-header-control-add

请求在转发给后端服务时,添加指定Header。若该Header存在,则其值拼接在原有值后面。语法如下:

  • 单个Header:Key Value。

  • 多个Header:使用YAML特殊符号|,每对Key Value单独处于一行。

mse.ingress.kubernetes.io/request-header-control-update

请求在转发给后端服务时,修改指定Header。若该Header存在,则其值覆盖原有值。语法如下:

  • 单个Header:Key Value。

  • 多个Header:使用YMAL特殊符号|,每对Key Value单独处于一行。

mse.ingress.kubernetes.io/request-header-control-remove

请求在转发给后端服务时,删除指定Header。语法如下:

  • 单个Header:Key。

  • 多个Header:使用英文逗号分隔。

例如:

  • 对于请求example.com/test添加两个Header,分别是foo: bar和test: true。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/request-header-control-add: |
          foo bar
          test true
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com 
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/request-header-control-add: |
          foo bar
          test true
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact
  • Header控制可以结合灰度发布,对灰度流量进行染色。请求Header为mse:v1时将访问灰度服务demo-service-canary-v1,并添加Header(stage: gray);其他情况将访问正式服务demo-service,并添加Header(stage: production)。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
        nginx.ingress.kubernetes.io/canary-by-header-value: "v1"
        mse.ingress.kubernetes.io/request-header-control-add: "stage gray"
      name: demo-canary-v1
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service-canary-v1
                  servicePort: 80
    ---
    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/request-header-control-add: |
          foo bar
          test true
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com 
          http:
            paths:
              - path: /hello
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/canary: "true"
        nginx.ingress.kubernetes.io/canary-by-header: "mse"
        nginx.ingress.kubernetes.io/canary-by-header-value: "v1"
        mse.ingress.kubernetes.io/request-header-control-add: "stage gray"
      name: demo-canary-v1
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service-canary-v1
                    port: 
                      number: 80
                path: /hello
                pathType: Exact
    ---
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/request-header-control-add: "stage production"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /hello
                pathType: Exact

响应Header控制

注解

说明

mse.ingress.kubernetes.io/response-header-control-add

请求在收到后端服务响应之后并且转发响应给客户端之前,添加指定Header。若该Header存在,则其值拼接在原有值后面。语法如下:

  • 单个Header:Key Value。

  • 多个Header:使用YAML特殊符号|,每对Key Value单独处于一行。

mse.ingress.kubernetes.io/response-header-control-update

请求在收到后端服务响应之后并且转发响应给客户端之前,修改指定Header。若该Header存在,则其值覆盖原有值。语法如下:

  • 单个Header:Key Value。

  • 多个Header:使用YMAL特殊符号|,每对Key Value单独处于一行。

mse.ingress.kubernetes.io/response-header-control-remove

请求在收到后端服务响应之后并且转发响应给客户端之前,删除指定Header。语法如下:

  • 单个Header:Key。

  • 多个Header:使用英文逗号分隔。

例如,对于请求example.com/test的响应删除Header:req-cost-time。配置如下:

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/response-header-control-remove: "req-cost-time"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com 
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/response-header-control-remove: "req-cost-time"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

重试

MSE Ingress提供路由级别的重试设置,可以为出错的请求自动进行重试。您可以按需设置重试条件,例如建立连接失败、后端服务不可用或者对指定HTTP状态码的响应等进行请求重试。

注解

说明

nginx.ingress.kubernetes.io/proxy-next-upstream-tries

请求的最大重试次数。默认为3次。

nginx.ingress.kubernetes.io/proxy-next-upstream-timeout

请求重试的超时时间,单位秒。默认未配置超时时间。

nginx.ingress.kubernetes.io/proxy-next-upstream

请求重试条件,使用英文逗号作为分隔。默认值为error,timeout,合法值如下:

  • error:建立连接失败,请求出错5xx。

  • timeout:建立连接超时,请求出错5xx。

  • invalid_header:请求出错5xx。

  • http_xxx:针对具体响应状态码的情况进行重试。例如:http_502、http_403。

  • non_idempotent:对于非幂等请求出错时进行重试。默认情况下,MSE Ingress针对非幂等POST、PATCH请求出错时不会进行重试;如果配置non_idempotent,可以开启重试。

  • off:关闭重试。

例如,设置example/test请求的最大重试次数为2次,重试超时时间为5秒,只有在响应状态码为502才重试,并且开启非幂等重试。配置如下:

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-next-upstream-tries: "2"
    nginx.ingress.kubernetes.io/proxy-next-upstream-timeout: "5"
    nginx.ingress.kubernetes.io/proxy-next-upstream: "http_502,non_idempotent"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com 
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-next-upstream-tries: "2"
    nginx.ingress.kubernetes.io/proxy-next-upstream-timeout: "5"
    nginx.ingress.kubernetes.io/proxy-next-upstream: "http_502,non_idempotent"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

IP黑白名单访问控制

MSE Ingress提供域名级和路由级的IP黑/白名单访问控制,且路由级的优先级高于域名级。

路由级IP访问控制

注解

说明

nginx.ingress.kubernetes.io/whitelist-source-range

指定路由上的IP白名单,支持IP地址或CIDR地址块,以英文逗号分隔。

mse.ingress.kubernetes.io/blacklist-source-range

指定路由上的IP黑名单,支持IP地址或CIDR地址块,以英文逗号分隔。

例如:

  • 仅允许客户端IP为1.1.xx.xx访问example.com/test。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/whitelist-source-range: 1.1.1.1
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com 
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/whitelist-source-range: 1.1.1.1
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact
  • 拒绝客户端IP为2.2.xx.xx访问example.com/test。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/blacklist-source-range: 2.2.2.2
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/blacklist-source-range: 2.2.2.2
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact

域名级IP访问控制

注解

说明

mse.ingress.kubernetes.io/domain- whitelist-source-range

指定域名上的IP白名单,域名优先级低于路由级别,支持IP地址或CIDR地址块,IP之间以英文逗号分隔。

mse.ingress.kubernetes.io/domain- blacklist-source-range

指定域名上的IP黑名单,域名优先级低于路由级别,支持IP地址或CIDR地址块,IP之间以英文逗号分隔。

例如:

  • 仅允许客户端IP为1.1.xx.xx和2.2.xx.xx可以访问example.com域名下所有路由。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/domain-whitelist-source-range: 1.1.1.1,2.2.2.2
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80
              - path: /app
                backend:
                  serviceName: app-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/domain-whitelist-source-range: 1.1.1.1,2.2.2.2
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact
             - backend:
                  service:
                    name: app-service
                    port: 
                      number: 80
                path: /app
                pathType: Exact
  • 域名级和路由级IP访问控制可以结合使用,仅允许客户端IP为1.1.xx.xx和2.2.xx.xx可以访问example.com域名下所有路由,但对于example.com/order这条路由,仅允许客户端IP为3.3.xx.xx可以访问。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/domain-whitelist-source-range: 1.1.1.1,2.2.2.2
      name: demo-domain
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80
              - path: /app
                backend:
                  serviceName: app-service
                  servicePort: 80
    ---
    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/domain-whitelist-source-range: 1.1.1.1,2.2.2.2
      name: demo-route
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /order
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/domain-whitelist-source-range: 1.1.1.1,2.2.2.2
      name: demo-domain
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact
             - backend:
                  service:
                    name: app-service
                    port: 
                      number: 80
                path: /app
                pathType: Exact
    ---
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/whitelist-source-range: 3.3.3.3
      name: demo-route
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /order
                pathType: Exact

单机限流

MSE Ingress支持针对路由级别的限流策略,在设定的时间周期内细粒度的限制匹配在某个路由上的请求数量不大于阈值。

说明

目前MSE Ingress提供限流是单机级别,即对匹配某条路由的请求访问单个网关实例进行流控。

注解

说明

mse.ingress.kubernetes.io/route-limit-rpm

该Ingress定义的路由在每个网关实例上每分钟最大请求次数。瞬时最大请求次数为该值乘以limit-burst-multiplier。

mse.ingress.kubernetes.io/route-limit-rps

该Ingress定义的路由在每个网关实例上每秒最大请求次数。瞬时最大请求次数为该值乘以limit-burst-multiplier。

mse.ingress.kubernetes.io/route-limit-burst-multiplier

瞬时最大请求次数的因子,默认为5。

例如:

  • 限制example.com/test的请求每分钟最大请求数为100,瞬时请求数200。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/route-limit-rpm: "100"
        mse.ingress.kubernetes.io/route-limit-burst-multiplier: "2"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/route-limit-rpm: "100"
        mse.ingress.kubernetes.io/route-limit-burst-multiplier: "2"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact
  • 限制example.com/test的请求每秒最大请求数为10,瞬时请求数50。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/route-limit-rps: "10"
        # 默认为5
        # mse.ingress.kubernetes.io/route-limit-burst-multiplier: "5"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        mse.ingress.kubernetes.io/route-limit-rps: "10"
        # 默认为5
        # mse.ingress.kubernetes.io/route-limit-burst-multiplier: "5"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact

流量镜像

通过配置流量镜像,可以复制流量到指定服务,常用于操作审计和流量测试等场景。

mse.ingress.kubernetes.io/mirror-target-service:复制流量转发到指定镜像服务。服务格式为:namespace/name:port。

  • namespace: K8s Service所在的命名空间,可选,默认为Ingress所在的命名空间。

  • name:K8s Service的名称,必选。

  • port:待转发至K8s Service的端口,可选,默认为第一个端口。

说明

复制的流量在转发给目标服务时,原始请求中的Host会被自动加上-shadow后缀。

例如,将example.com/test的流量复制并转发到目标服务:命名空间为test,服务名为app,端口为8080。

说明

本示例中,复制的流量在转发给目标服务时,Host会被自动改写为example.com-shadow。

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/mirror-target-service: test/app:8080
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/mirror-target-service: test/app:8080
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

配置后端服务协议:HTTPS或gRPC

MSE Ingress默认使用HTTP协议转发请求到后端业务容器。当您的业务容器为HTTPS协议时,可以通过使用注解nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"来转发请求到后端业务容器;当您的业务容器为gRPC服务时,可以通过使用注解nginx.ingress.kubernetes.io/backend-protocol: "GRPC"来转发请求到后端业务容器。

说明

相比Nginx Ingress的优势,如果您的后端服务所属的K8s Service资源中关于Port Name的定义为gRPC或HTTP2,您无需配置注解nginx.ingress.kubernetes.io/backend-protocol: "GRPC",MSE Ingress会自动使用gRPC或者HTTP2。

例如:

  • 请求example/test转发至后端服务使用HTTPS协议。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /
                pathType: Exact
  • 请求example/test转发至后端服务使用gRPC协议。此处列举两种做法,如下:

    • 方法1:通过注解,配置如下:

      1.19版本之前集群

      apiVersion: networking.k8s.io/v1beta1
      kind: Ingress
      metadata:
        annotations:
          nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
        name: demo
      spec:
        ingressClassName: mse
        rules:
          - host: example.com
            http:
              paths:
                - path: /test
                  backend:
                    serviceName: demo-service
                    servicePort: 80

      1.19及之后版本集群

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        annotations:
          nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
        name: demo
      spec:
        ingressClassName: mse
        rules:
          - host: example.com
            http:
              paths:
                - backend:
                    service:
                      name: demo-service
                      port: 
                        number: 80
                  path: /test
                  pathType: Exact
    • 方法2:通过Service Port Name,配置如下:

      1.19版本之前集群

      apiVersion: networking.k8s.io/v1beta1
      kind: Ingress
      metadata:
        name: demo
      spec:
        ingressClassName: mse
        rules:
          - host: example.com
            http:
              paths:
                - path: /test
                  backend:
                    serviceName: demo-service
                    servicePort: 80
      ---
      apiVersion: v1
      kind: Service
      metadata:
        name: demo-service
      spec:
        ports:
          - name: grpc
            port: 80
            protocol: TCP
        selector:
          app: demo-service

      1.19及之后版本集群

      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        name: demo
      spec:
        ingressClassName: mse
        rules:
          - host: example.com
            http:
              paths:
                - backend:
                    service:
                      name: demo-service
                      port: 
                        number: 80
                  path: /order
                  pathType: Exact
      ---
      apiVersion: v1
      kind: Service
      metadata:
        name: demo-service
      spec:
        ports:
          - name: grpc
            port: 80
            protocol: TCP
        selector:
          app: demo-service

配置后端服务的负载均衡算法

负载均衡决定着网关在转发请求至后端服务时如何选择节点。

普通负载均衡算法

nginx.ingress.kubernetes.io/load-balance:后端服务的普通负载均衡算法。默认为round_robin。合法值如下:

  • round_robin:基于轮询的负载均衡。

  • least_conn:基于最小请求数的负载均衡。

  • random:基于随机的负载均衡。

重要

云原生网关不支持EWMA算法,若配置为EWMA算法,会回退到Round Robin算法。

例如,设置后端服务demo-service的负载均衡算法为least_conn。设置如下:

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/load-balance: "least_conn"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/load-balance: "least_conn"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /order
            pathType: Exact

基于一致性Hash的负载均衡算法

基于一致性Hash的负载均衡算法具备请求亲和性,具有相同特征的请求会始终负载到相同节点上。MSE Ingress支持基于部分Nginx变量的请求Header和请求路径参数作为Hash Key。

nginx.ingress.kubernetes.io/upstream-hash-by:基于一致性Hash的负载均衡算法,云原生网关支持以下几种形式:

  • 云原生网关支持配置部分nginx变量:

    • $request_uri:请求的Path(包括路径参数)作为Hash Key。

    • $host:请求的Host作为Hash Key。

    • $remote_addr:请求的客户端IP作为Hash Key。

  • 基于请求Header的一致性Hash。您只需配置为$http_headerName。

  • 基于请求路径参数的一致性Hash。您只需配置为$arg_varName。

例如:

  • 基于请求的客户端IP作为Hash Key,同一个客户端IP的请求始终负载到同一个节点。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/upstream-hash-by: "$remote_addr"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/upstream-hash-by: "$remote_addr"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact
  • 基于请求Header x-stage作为Hash key,带有x-stage头部的请求且值相同的请求始终负载到同一个节点。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/upstream-hash-by: "$http_x-stage"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/upstream-hash-by: "$http_x-stage"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact
  • 基于请求路径参数 x-stage作为Hash key,带有路径参数x-stage的请求且值相同的请求始终负载到同一个节点。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/upstream-hash-by: "$arg_x-stage"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/upstream-hash-by: "$arg_x-stage"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact

服务预热(无损上线)

服务预热可以保证新节点上线时,流量在指定预热窗口内是逐步调大,充分保证新节点完成预热。

mse.ingress.kubernetes.io/warmup:服务预热时间,单位为秒。默认不开启。

说明

服务预热依赖于所选的负载均衡算法,目前仅支持Round Robin和least_conn。

例如,对于后端服务demo-service开启预热,预热窗口为30s。配置如下:

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/warmup: "30"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/warmup: "30"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

Cookie亲和性(会话保持)

具备相同Cookie的请求会被网关始终负载到同一个节点,并且如果第一次访问携带Cookie,MSE Ingress会在第一次响应时为客户端生成一个Cookie,用来保证后续的请求被网关始终负载到相同节点。

注解

说明

nginx.ingress.kubernetes.io/affinity

亲和性种类,目前只支持Cookie,默认为Cookie。

nginx.ingress.kubernetes.io/affinity-mode

亲和性模式,云原生网关目前只支持Balanced模式,默认为Balanced模式。

nginx.ingress.kubernetes.io/session-cookie-name

配置指定Cookie的值作为Hash Key,默认为INGRESSCOOKIE。

nginx.ingress.kubernetes.io/session-cookie-path

当指定Cookie不存在,生成的Cookie的Path值,默认为/。

nginx.ingress.kubernetes.io/session-cookie-max-age

当指定Cookie不存在,生成的Cookie的过期时间,单位为秒,默认为Session会话级别。

nginx.ingress.kubernetes.io/session-cookie-expires

当指定Cookie不存在,生成的Cookie的过期时间,单位为秒,默认为Session会话级别。

例如:

  • 开启Cookie亲和性,利用MSE Ingress的默认配置,即Cookie的名字为INGRESSCOOKIE,Path为/,Cookie的生命周期为Session会话级别。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/affinity: "cookie"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/affinity: "cookie"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact
  • 开启Cookie亲和性,Cookie的名字为test,Path为/,Cookie的过期时间为10s。配置如下:

    1.19版本之前集群

    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/affinity: "cookie"
        nginx.ingress.kubernetes.io/session-cookie-name: "test"
        nginx.ingress.kubernetes.io/session-cookie-max-age: "10"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - path: /test
                backend:
                  serviceName: demo-service
                  servicePort: 80

    1.19及之后版本集群

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        nginx.ingress.kubernetes.io/affinity: "cookie"
        nginx.ingress.kubernetes.io/session-cookie-name: "test"
        nginx.ingress.kubernetes.io/session-cookie-max-age: "10"
      name: demo
    spec:
      ingressClassName: mse
      rules:
        - host: example.com
          http:
            paths:
              - backend:
                  service:
                    name: demo-service
                    port: 
                      number: 80
                path: /test
                pathType: Exact

网关与后端服务之间的连接池配置

通过在网关侧对指定服务进行连接池配置,可以控制网关与后端服务之间的连接数量,有效防止后端服务过载,提高后端服务的稳定性和高可用。

  • mse.ingress.kubernetes.io/connection-policy-tcp-max-connection:网关与后端服务之间可以建立连接的最大数量。

  • mse.ingress.kubernetes.io/connection-policy-tcp-max-connection-per-endpoint:网关与后端服务的单个节点之间可以建立连接的最大数量。

  • mse.ingress.kubernetes.io/connection-policy-http-max-request-per-connection:网关与后端服务之间单个连接上的最大请求数。

例如,对后端服务demo-service配置,网关与后端服务之间可以建立连接的最大数量为10,网关与后端服务的单个节点之间可以建立连接的最大数量为2。

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/connection-policy-tcp-max-connection:10
  	mse.ingress.kubernetes.io/connection-policy-tcp-max-connection-per-endpoint:2
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/connection-policy-tcp-max-connection:10
  	mse.ingress.kubernetes.io/connection-policy-tcp-max-connection-per-endpoint:2
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

配置客户端到网关之间的TLS版本以及加密套件

目前,MSE Ingress默认最小TLS版本为TLSv1.0,默认最大TLS版本为TLSv1.3,默认加密套件为:

  • ECDHE-ECDSA-AES128-GCM-SHA256

  • ECDHE-RSA-AES128-GCM-SHA256

  • ECDHE-ECDSA-AES128-SHA

  • ECDHE-RSA-AES128-SHA

  • AES128-GCM-SHA256

  • AES128-SHA

  • ECDHE-ECDSA-AES256-GCM-SHA384

  • ECDHE-RSA-AES256-GCM-SHA384

  • ECDHE-ECDSA-AES256-SHA

  • ECDHE-RSA-AES256-SHA

  • AES256-GCM-SHA384

  • AES256-SHA

您可以通过以下注解为特定的域名设置最小或者最大TLS版本以及加密套件。

注解

说明

mse.ingress.kubernetes.io/tls-min-protocol-version

指定TLS的最小版本,默认值为TLSv1.0。合法值如下:

  • TLSv1.0

  • TLSv1.1

  • TLSv1.2

  • TLSv1.3

mse.ingress.kubernetes.io/tls-max-protocol-version

指定TLS的最大版本,默认值为TLSv1.3。

nginx.ingress.kubernetes.io/ssl-cipher

指定TLS的加密套件,可以指定多个,英文逗号分隔,仅当TLS握手时采用TLSv1.0~1.2生效。

例如,对于域名example.com,设置TLS最小版本为TLSv1.2,最大版本为TLSv1.2。配置如下:

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/tls-min-protocol-version: "TLSv1.2"
    mse.ingress.kubernetes.io/tls-max-protocol-version: "TLSv1.2"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    mse.ingress.kubernetes.io/tls-min-protocol-version: "TLSv1.2"
    mse.ingress.kubernetes.io/tls-max-protocol-version: "TLSv1.2"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

网关与后端服务双向认证(MTLS)

MSE Ingress默认使用HTTP协议转发请求到后端业务容器。你可以通过使用注解nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"配置MSE Ingress访问后端服务使用HTTPS协议,但这是单向TLS,也就是说只有MSE Ingress会验证后端服务提供的证书,且一般后端服务使用的证书需要是权威CA(Catificate Authority)签发的。另一种更安全的模式是零信任,网关会验证后端服务的证书是否合法,同样后端服务也会验证网关提供的证书是否合法,这就是MTLS,网关与后端服务进行双向认证。

注解

说明

nginx.ingress.kubernetes.io/proxy-ssl-secret

网关使用的客户端证书,用于后端服务对网关进行身份认证,格式为secretNamespace/secretName。

nginx.ingress.kubernetes.io/proxy-ssl-name

TLS握手期间使用的SNI。

nginx.ingress.kubernetes.io/proxy-ssl-server-name

开启或关闭TLS握手期间使用的SNI。

例如,网关与后端服务进行双向认证,网关使用的secret name为gateway-cert,命名空为default。配置如下:

1.19版本之前集群

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-ssl-secret: "default/ateway-cert"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - path: /test
            backend:
              serviceName: demo-service
              servicePort: 80

1.19及之后版本集群

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-ssl-secret: "default/ateway-cert"
  name: demo
spec:
  ingressClassName: mse
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: demo-service
                port: 
                  number: 80
            path: /test
            pathType: Exact

阿里云首页 容器服务Kubernetes版 相关技术圈