ASM gateway CRD reference

更新时间:
复制 MD 格式

Alibaba Cloud Service Mesh (ASM) provides a custom resource definition (CRD) with kind set to IstioGateway and apiVersion set to istio.alibabacloud.com/v1beta1. ASM also provides a corresponding controller. By listening for changes to this CRD resource, the controller synchronizes the associated Kubernetes Service, Deployment, ServiceAccount, and other related resources in the cluster. This topic describes configuration examples and field definitions for the ASM gateway CRD.

Configuration examples

Prerequisites

Add a cluster to your ASM instance.

Important

Create all new ASM gateways in the istio-system namespace to ensure they can access required configuration information. In Istio 1.6 and later versions, deploying a gateway in any other namespace prevents it from accessing necessary configurations, causing the ASM gateway to fail to start.

Example 1: Manage an ingress gateway using the Kubernetes API

  1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

  2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose ASM Gateways > Ingress Gateway.

  3. On the Ingress Gateway page, click Create from YAML.

  4. On the Create page, enter a Name for the gateway, select Namespaces as istio-system, configure the following YAML, and then click Create.

    For details about each field, see CRD field reference.

    Expand to view the YAML file

    apiVersion: istio.alibabacloud.com/v1beta1
    kind: IstioGateway
    metadata:   
      name: "myexample-customingressgateway"  
      namespace: "istio-system"
      labels:
        asm-gateway-type: ingress
    spec:
      runAsRoot: true
      dnsPolicy: ClusterFirst
      hostNetwork: false
      clusterIds:
        - "${actual cluster ID}"
        - "${actual cluster ID}"   # The ASM gateway supports deployment across multiple clusters.
      cpu: 
        targetAverageUtilization: 80
      env:
        - name: "envname1"
          value: "envvalue1"
      externalTrafficPolicy: Local
      gatewayType: ingress
      podLabels:
        key1: value1
      ports:  
      - name: status-port    
        port: 15020    
        targetPort: 15020  
      - name: http2    
        port: 80    
        targetPort: 80  
      - name: https    
        port: 443    
        targetPort: 443
      - name: tls    
        port: 15443    
        targetPort: 15443  
      replicaCount: 1  
      resources:
        limits:
          cpu: '2'
          memory: 2G
        requests:
          cpu: 200m
          memory: 256Mi
    # - name: config-volume-lua
    #  configMapName: lua-libs
    #  mountPath: /var/lib/lua
    # secretVolumes:
    # - name: myexample-customingressgateway-certs
    #   secretName: istio-myexample-customingressgateway-certs
    #   mountPath: /etc/istio/myexample-customingressgateway-certs
      serviceType: LoadBalancer  
      serviceAnnotations:    
        service.beta.kubernetes.io/alicloud-loadbalancer-address-type: internet  
      serviceLabels:
        serviceLabelKey1: "serviceLabelValue1"
      podAnnotations:
        podAnnotationsKey1: "podAnnotationsValue1"
      rollingMaxSurge: "100%"
      rollingMaxUnavailable: "25%"
      overrides:
        cluster1Id:
          replicaCount: 1 
          resources:
            limits:
              cpu: '2'
              memory: 2G
            requests:
              cpu: 200m
              memory: 256Mi
          serviceAnnotations:    
            service.beta.kubernetes.io/alicloud-loadbalancer-address-type: internet
            service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec: "slb.s1.small"
        cluster2Id:
          replicaCount: 2
          resources:
            limits:
              cpu: '4'
              memory: 4G
            requests:
              cpu: 400m
              memory: 512Mi
          serviceAnnotations:    
            service.beta.kubernetes.io/alicloud-loadbalancer-address-type: internet
            service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec: "slb.s2.small"

    Note

    If you want to manage the ingress gateway using the Kubernetes client, see Gateway Go struct implementation.

  5. View the service information of the ingress gateway.

    1. Log on to the ACK console. In the left navigation pane, click Clusters.

    2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Network > Services.

    3. At the top of the Services page, select istio-system from the Namespace drop-down list.

    4. In the services list, click the name of your target service to view its ingress gateway information.

  6. View the pod information of the ingress gateway.

    1. Log on to the ACK console. In the left navigation pane, click Clusters.

    2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Workloads > Pods.

    3. At the top of the Pods page, select istio-system from the Namespace drop-down list.

    4. In the pods list, click the name of your target pod to view its ingress gateway information.

Example 2: Manage an egress gateway using the Kubernetes API

ASM provides an egress gateway service that offers a unified traffic exit point for applications within the mesh.

  1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

  2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose ASM Gateways > Egress Gateway.

  3. On the Egress Gateway page, click Create from YAML.

  4. On the Create page, enter a Name for the egress gateway, select Namespaces as istio-system, configure the following YAML, and then click Create.

    For details about each field, see CRD field reference.

    Expand to view the YAML file

    apiVersion: istio.alibabacloud.com/v1beta1
    kind: IstioGateway
    metadata:
      name: egressgateway
      namespace: istio-system
      labels:
        asm-gateway-type: egress
    spec:
      autoCreateGatewayYaml: false
      clusterIds:
        - ${actual cluster ID}
      disableContainerPortExposed: true
      dnsPolicy: ClusterFirst
      gatewayType: egress
      hostNetwork: false
      podLabels:
        security.istio.io/tlsMode: istio
      ports:
        - name: http-0
          port: 80
          protocol: HTTP
          targetPort: 80
        - name: https-1
          port: 443
          protocol: HTTPS
          targetPort: 443
      readinessProbe: {}
      replicaCount: 1
      resources:
        limits:
          cpu: '2'
          memory: 4G
        requests:
          cpu: 200m
          memory: 256Mi
      rollingMaxSurge: 25%
      rollingMaxUnavailable: 25%
      runAsRoot: true
      serviceType: ClusterIP
  5. View the service information of the egress gateway.

    1. Log on to the ACK console. In the left navigation pane, click Clusters.

    2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Network > Services.

    3. On the Services page, select istio-system from the Namespace drop-down list.

    4. In the services list, click the name of your target service to view its egress gateway information.

  6. View the pod information of the egress gateway.

    1. Log on to the ACK console. In the left navigation pane, click Clusters.

    2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Workloads > Pods.

    3. On the Pods page, select istio-system from the Namespace drop-down list.

    4. In the pods list, click the name of your target pod to view its egress gateway information.

CRD field reference

Field

Description

Default

metadata.name

Name. The generated Kubernetes Service and Deployment names are istio-{this value}.

None

metadata.namespace

Namespace where the generated Kubernetes Service and Deployment reside.

Important

To maintain compatibility with Istio 1.6 and later versions, this namespace must be istio-system.

istio-system

clusterIds

Array type. IDs of clusters where the gateway will be deployed. These clusters belong to the current mesh instance.

None

env

Array type. Environment variables for the gateway pod.

None

externalTrafficPolicy

Specifies whether external traffic should route to node-local or cluster-wide endpoints. Valid values are Cluster or Local.

Note

This field applies only to ingress gateways.

Cluster

ports

Array type. List of ports and protocols defined for the gateway pod. Examples:

  • name: http2 port: 80 targetPort: 80 protocol: HTTP2

  • name: https port: 443 targetPort: 443 protocol: HTTPS

Note

In versions earlier than 1.9.7.107, the protocol field is not explicitly declared. You must declare it uniformly as TCP.

None

gatewayType

Gateway type. Valid values are ingress or egress.

  • ingress: An ingress gateway provides a unified traffic entry point for applications within the mesh.

  • egress: An egress gateway provides a unified traffic exit point for applications within the mesh.

ingress

replicaCount

Number of replicas.

1

configVolumes

ConfigMap volumes mounted to the gateway pod. Example:

- name: config-volume-lua
  configMapName: lua-libs
  mountPath: /var/lib/lua

None

resources

Resource configuration for the gateway pod.

  • limits:

    • cpu: '2'

    • memory: 2G

  • requests:

    • cpu: 200m

    • memory: 256Mi

secretVolumes

Secret volumes mounted to the gateway pod. Example:

- name: myexample-customingressgateway-certs
  secretName: istio-myexample-customingressgateway-certs
  mountPath: /etc/istio/myexample-customingressgateway-certs

None

serviceType

Service type for the gateway. Valid values are LoadBalancer, NodePort, or ClusterIP. Egress gateways are typically set to ClusterIP.

Important

If your data plane cluster is a registered cluster and you set this field to LoadBalancer, ensure the cluster supports LoadBalancer-type Services to avoid ASM gateway creation failures.

LoadBalancer

loadBalancerClass

Specifies the loadBalancerClass for the gateway service. This field takes effect only when serviceType is LoadBalancer. This field is supported starting from ASM version 1.18. For more information, see Use Network Load Balancer (NLB) with ASM ingress gateways.

None

serviceAnnotations

Annotations for the gateway service. Example: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain: 'on' service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout: '20'. For common annotation examples, see Configure load balancing using annotations.

Note

This field applies only to gateways with serviceType set to LoadBalancer. Egress gateways typically do not use this field.

None

serviceLabels

Labels for the gateway service.

None

podLabels

Labels for the gateway pod.

None

podAnnotations

Annotations for the gateway pod.

None

rollingMaxSurge

Maximum number of pods that can exist above the desired number during a rolling update. This value can be an absolute number or a percentage.

"100%"

rollingMaxUnavailable

Maximum number of unavailable pods during a rolling update. This value can be an absolute number or a percentage.

"25%"

overrides

When clusterIds specifies two or more clusters, you can define cluster-specific configurations that differ from the general parameters above. The value is a map type.

Note
  • key: One of the cluster IDs listed in clusterIds.

  • Fields supported by overrides include the following:

    • serviceAnnotations

    • podAnnotations

    • resources

    • replicaCount

    • HPA-related configurations (supported starting from ASM 1.16)

    • serviceType (supported starting from ASM 1.16)

None

kernel.enabled

Are custom kernel parameters enabled?

false

kernel.parameters

Kernel parameter settings. The following kernel parameters are currently supported:

  • net.core.somaxconn

  • net.core.netdev_max_backlog

  • net.ipv4.tcp_rmem

  • net.ipv4.tcp_wmem

  • net.ipv4.ip_local_port_range

  • net.ipv4.tcp_fin_timeout

  • net.ipv4.tcp_tw_timeout

  • net.ipv4.tcp_tw_reuse

  • net.ipv4.tcp_tw_recycle

  • net.ipv4.tcp_timestamps

  • net.ipv4.tcp_retries2

  • net.ipv4.tcp_slow_start_after_idle

  • net.ipv4.tcp_max_orphans

  • net.ipv4.tcp_max_syn_backlog

  • net.ipv4.tcp_no_metrics_save

  • net.ipv4.tcp_autocorking

  • kernel.printk

  • vm.swappiness

Note

Depending on the host kernel version, some kernel parameters may not be supported. If this occurs, the mesh pod may report errors.

Use the kubectl describe pod command to check gateway error messages. After removing unsupported parameters, the container starts normally.

All kernel parameter values must be strings. Because YAML syntax parses plain numbers as numeric types, wrap your values in double quotes (""), for example: net.core.somaxconn: "65535".

None

compression.enabled

Enables compression capability for the ingress gateway.

Note

Fields related to compression apply only to ingress gateways.

false

compression.content_type

List of ContentTypes to compress. Examples:

  • text/html

  • application/json

None

compression.disable_on_etag_header

  • Set to true: Disable compression when the response contains an etag_header.

  • Set to false: Compress the response even if it contains an etag_header.

false

compression.min_content_length

Triggers compression when ContentLength is greater than or equal to this value.

30

compression.remove_accept_encoding_header

  • Set to true: Remove the Accept-Encoding header from the client request before forwarding it upstream.

  • Set to false: Keep the Accept-Encoding header in the client request before forwarding it upstream.

false

compression.gzip

Only gzip compression is currently supported. To enable compression, you must include this field. Even if all parameters use their defaults, include an empty structure, for example: gzip: {}.

None

compression.gzip.memory_level

Memory usage level for zlib internals. Valid values are 1–9. Higher values use more memory but provide faster compression speeds and better compression ratios.

5

compression.gzip.compression_level

Compression level for zlib. Valid values:

  • COMPRESSION_LEVEL_1

  • COMPRESSION_LEVEL_2

  • COMPRESSION_LEVEL_3

  • COMPRESSION_LEVEL_4

  • COMPRESSION_LEVEL_5

  • COMPRESSION_LEVEL_6

  • COMPRESSION_LEVEL_7

  • COMPRESSION_LEVEL_8

  • COMPRESSION_LEVEL_9

  • DEFAULT_COMPRESSION

  • BEST_COMPRESSION

  • BEST_SPEED

Note

BEST_COMPRESSION provides the highest compression quality. BEST_SPEED provides the fastest compression speed. Specifically:

  • COMPRESSION_LEVEL_1 is equivalent to BEST_SPEED. It compresses fastest but has a lower compression ratio.

  • COMPRESSION_LEVEL_9 is equivalent to BEST_COMPRESSION. Levels 7–9 provide maximum compression ratios but slower speeds.

  • COMPRESSION_LEVEL_6 is equivalent to DEFAULT_COMPRESSION. Levels 2–6 offer medium compression with balanced speed and ratio.

DEFAULT_COMPRESSION

compression.gzip.compression_strategy

Compression strategy for zlib. Valid values:

  • DEFAULT_STRATEGY

  • FILTERED

  • FIXED

  • HUFFMAN_ONLY

  • RLE

Note

DEFAULT_STRATEGY 

The default compression strategy uses underlying default behavior and produces satisfactory results in most cases.

FILTERED 

Prioritizes filter-layer compression algorithms for scenarios with mostly repetitive data. Offers high compression ratios for text and artificially generated data.

HUFFMAN_ONLY 

Uses only Huffman coding without other compression algorithms. Typically used for static data that requires fast decompression.

RLE 

Uses run-length encoding, which efficiently compresses sequences of identical data, such as bitmaps.

FIXED 

Uses predefined encoding trees for Huffman coding instead of dynamically generating them based on input data. Enables fast compression and decompression but generally yields lower compression ratios.

DEFAULT_STRATEGY

compression.gzip.window_bits

Window size for zlib. Valid values are 9–15.

12

compression.gzip.chunk_size

Output buffer size for zlib.

4096

hostNetwork

Host networking.

  • Set to true: The gateway pod uses the host's network.

  • Set to false: The gateway pod uses its own network.

false

dnsPolicy

DNS policy. For more information about dnsPolicy, see DNS for Services and Pods.

ClusterFirst

cpu.targetAverageUtilization

CPU target utilization when HPA is enabled. Valid values are 1–100. If utilization exceeds this value, replicas increase. If it falls below, replicas decrease.

None

memory.targetAverageUtilization

Memory target utilization when HPA is enabled. Valid values are 1–100. If utilization exceeds this value, replicas increase. If it falls below, replicas decrease.

None

maxReplicas

Maximum number of replicas adjusted by HPA. Must be a positive integer.

None

minReplicas

Minimum number of replicas adjusted by HPA. Must be a positive integer with a minimum value of 1.

None

nodeSelector

Specifies the node where the gateway pod runs. Define custom key-value pairs under this field. Example:

nodeSelector:
  key1: value1

None

affinity

Specifies the node for the gateway pod using node affinity. For more information, see Affinity and anti-affinity.

None

tolerations

Used together with node taints. For more information, see Taints and tolerations.

None

Fields that trigger a rolling restart

Changing the following fields triggers a rolling restart of the gateway. Proceed with caution.

  • env

  • ports (triggers a restart in ASM versions earlier than 1.16; no restart in 1.16 and later)

  • configVolumes

  • resources

  • secretVolumes

  • podLabels

  • podAnnotations

  • kernel.enabled

  • kernel.parameters

  • hostNetwork

  • dnsPolicy

  • nodeSelector (depends on scheduling)

  • affinity (depends on scheduling)

  • tolerations (depends on scheduling)

Gateway Go struct implementation

Expand to view the file example

package v1beta1

import (
    corev1 "k8s.io/api/core/v1"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

type GatewayType string
type ConfigState string

const (
    GatewayTypeIngress GatewayType = "ingress"
    GatewayTypeEgress  GatewayType = "egress"
)

// IstioGatewaySpec defines the desired state of Istio
// +k8s:openapi-gen=true
type IstioGatewaySpec struct {
    // +kubebuilder:validation:Enum=ingress;egress
    GatewayType                    GatewayType `json:"gatewayType,omitempty"`
    IstioGatewayBasicConfiguration `json:",inline"`
    ClusterIds                     []string                                  `json:"clusterIds,omitempty"`
    clusterId                      string                                    `json:"clusterId,omitempty"`
    Overrides                      map[string]IstioGatewayBasicConfiguration `json:"overrides,omitempty"`
}

type KernelParameters struct {
    NetCoreSoMaxConn             *string `json:"net.core.somaxconn,omitempty"`
    NetCoreNetdevMaxBacklog      *string `json:"net.core.netdev_max_backlog,omitempty"`
    NetIpv4TcpRMem               *string `json:"net.ipv4.tcp_rmem,omitempty"`
    NetIpv4TcpWMem               *string `json:"net.ipv4.tcp_wmem,omitempty"`
    NetIpv4IpLocalPortRange      *string `json:"net.ipv4.ip_local_port_range,omitempty"`
    NetIpv4TcpFinTimeout         *string `json:"net.ipv4.tcp_fin_timeout,omitempty"`
    NetIpv4TcpTwTimeout          *string `json:"net.ipv4.tcp_tw_timeout,omitempty"`
    NetIpv4TcpTwReuse            *string `json:"net.ipv4.tcp_tw_reuse,omitempty"`
    NetIpv4TcpTwRecycle          *string `json:"net.ipv4.tcp_tw_recycle,omitempty"`
    NetIpv4TcpTimestamps         *string `json:"net.ipv4.tcp_timestamps,omitempty"`
    NetIpv4TcpRetries2           *string `json:"net.ipv4.tcp_retries2,omitempty"`
    NetIpv4TcpSlowStartAfterIdle *string `json:"net.ipv4.tcp_slow_start_after_idle,omitempty"`
    NetIpv4TcpMaxOrphans         *string `json:"net.ipv4.tcp_max_orphans,omitempty"`
    NetIpv4TcpMaxSynBacklog      *string `json:"net.ipv4.tcp_max_syn_backlog,omitempty"`
    NetIpv4TcpNoMetricsSave      *string `json:"net.ipv4.tcp_no_metrics_save,omitempty"`
    NetIpv4TcpAutocorking        *string `json:"net.ipv4.tcp_autocorking,omitempty"`
    KernelPrintk                 *string `json:"kernel.printk,omitempty"`
    VmSwappiness                 *string `json:"vm.swappiness,omitempty"`
}

type KernelConfiguration struct {
    Enabled    *bool            `json:"enabled,omitempty"`
    Parameters KernelParameters `json:"parameters,omitempty"`
}

type GzipCompressorConfiguration struct {
    MemoryLevel         *uint32 `json:"memory_level,omitempty"`
    CompressionLevel    *string `json:"compression_level,omitempty"`
    CompressionStrategy *string `json:"compression_strategy,omitempty"`
    WindowBits          *uint32 `json:"window_bits,omitempty"`
    ChunkSize           *uint32 `json:"chunk_size,omitempty"`
}

type CompressorConfiguration struct {
    Enabled                    *bool                        `json:"enabled,omitempty"`
    MinContentLength           *uint32                      `json:"min_content_length,omitempty"`
    ContentType                *[]string                    `json:"content_type,omitempty"`
    DisableOnETagHeader        *bool                        `json:"disable_on_etag_header,omitempty"`
    RemoveAcceptEncodingHeader *bool                        `json:"remove_accept_encoding_header,omitempty"`
    Gzip                       *GzipCompressorConfiguration `json:"gzip,omitempty"`
}

type ReadinessProbeConfiguration struct {
    FailureThreshold int32 `json:"failureThreshold,omitempty"`
    PeriodSeconds    int32 `json:"periodSeconds,omitempty"`
    SuccessThreshold int32 `json:"successThreshold,omitempty"`
}

// +k8s:openapi-gen=true
type IstioGatewayBasicConfiguration struct {
    ReplicaCount     int32 `json:"replicaCount,omitempty"`
    AutoscaleEnabled bool  `json:"autoscaleEnabled,omitempty"`
    MinReplicas      int32 `json:"minReplicas,omitempty"`
    MaxReplicas      int32 `json:"maxReplicas,omitempty"`
    // +kubebuilder:validation:Enum=ClusterIP;NodePort;LoadBalancer
    ServiceType           corev1.ServiceType           `json:"serviceType,omitempty"`
    LoadBalancerIP        string                       `json:"loadBalancerIP,omitempty"`
    ExternalTrafficPolicy string                       `json:"externalTrafficPolicy,omitempty"`
    ServiceAnnotations    map[string]string            `json:"serviceAnnotations,omitempty"`
    PodAnnotations        map[string]string            `json:"podAnnotations,omitempty"`
    ServiceLabels         map[string]string            `json:"serviceLabels,omitempty"`
    PodLabels             map[string]string            `json:"podLabels,omitempty"`
    Resources             *corev1.ResourceRequirements `json:"resources,omitempty"`
    EnvVars               []corev1.EnvVar              `json:"env,omitempty"`
    Ports                 []corev1.ServicePort         `json:"ports,omitempty"`
    MeshExpansionPorts    []corev1.ServicePort         `json:"meshExpansionPorts,omitempty"`
    NodeSelector          map[string]string            `json:"nodeSelector,omitempty"`
    ApplicationPorts      string                       `json:"applicationPorts,omitempty"`
    RequestedNetworkView  string                       `json:"requestedNetworkView,omitempty"`
    Affinity              *corev1.Affinity             `json:"affinity,omitempty"`
    Tolerations           []corev1.Toleration          `json:"tolerations,omitempty"`
    SecretVolumes         []SecretVolume               `json:"secretVolumes,omitempty"`
    ConfigVolumes         []ConfigVolume               `json:"configVolumes,omitempty"`
    RollingMaxSurge       *string                      `json:"rollingMaxSurge,omitempty"`
    RollingMaxUnavailable *string                      `json:"rollingMaxUnavailable,omitempty"`
    CPU                   HpaCPU                       `json:"cpu,omitempty"`
    Memory                HpaMemory                    `json:"memory,omitempty"`
    Kernel                KernelConfiguration          `json:"kernel,omitempty"`
    Compressor            CompressorConfiguration      `json:"compression,omitempty"`
    RunAsRoot             *bool                        `json:"runAsRoot,omitempty"`
    Lifecycle             *corev1.Lifecycle            `json:"lifecycle,omitempty"`
    ReadinessProbe        ReadinessProbeConfiguration  `json:"readinessProbe,omitempty"`
    HostNetwork           *bool                        `json:"hostNetwork,omitempty"`
    DnsPolicy             corev1.DNSPolicy             `json:"dnsPolicy,omitempty"`
    AutoCreateGatewayYaml *bool                        `json:"autoCreateGatewayYaml,omitempty"`
}

type SecretVolume struct {
    Name       string `json:"name,omitempty"`
    SecretName string `json:"secretName,omitempty"`
    MountPath  string `json:"mountPath,omitempty"`
}

type ConfigVolume struct {
    Name          string `json:"name,omitempty"`
    ConfigMapName string `json:"configMapName,omitempty"`
    MountPath     string `json:"mountPath,omitempty"`
}

// +k8s:openapi-gen=true
type HpaCPU struct {
    TargetAverageUtilization *int32 `json:"targetAverageUtilization,omitempty"`
}

// +k8s:openapi-gen=true
type HpaMemory struct {
    TargetAverageUtilization *int32 `json:"targetAverageUtilization,omitempty"`
}

// +k8s:openapi-gen=true
type IstioGatewayStatus struct {
    // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
    // Important: Run "operator-sdk generate k8s" to regenerate code after modifying this file
    // Add custom validation using kubebuilder tags: https://book.kubebuilder.io/beyond_basics/generating_crd.html
    Status                     ConfigState      `json:"Status,omitempty"`
    GatewayAddress             []string         `json:"GatewayAddress,omitempty"`
    ClusterIdGatewayAddressMap []GatewayAddress `json:"ClusterIdGatewayAddressMap,omitempty"`

    ErrorMessage string `json:"ErrorMessage,omitempty"`
}

// +k8s:openapi-gen=true
type GatewayAddress struct {
    IPAddress string `json:"ipAddress,omitempty"`
    ClusterId string `json:"clusterId,omitempty"`
}

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// IstioGateway is the Schema for the istiogateways API
// +k8s:openapi-gen=true
// +kubebuilder:subresource:status
type IstioGateway struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`

    Spec   IstioGatewaySpec   `json:"spec,omitempty"`
    Status IstioGatewayStatus `json:"status,omitempty"`
}

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// IstioGatewayList contains a list of IstioGateway
type IstioGatewayList struct {
    metav1.TypeMeta `json:",inline"`
    metav1.ListMeta `json:"metadata,omitempty"`
    Items           []IstioGateway `json:"items"`
}