Terway release notes

Terway version

Kubernetes version

>=v1.10

>= 1.31

v1.9

>= 1.30

Version

Image address

Date

Changes

Impact

v1.8.4

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.8.4

April 22, 2024

• Fixed an issue where the IP resource pool warm-up failed.

• Improved the idempotence of network interface creation and IP address allocation to prevent resource leaks.

• Adjusted the CNI configuration sequence to ensure containers are scheduled to a node only after the CNI is fully operational.

This upgrade does not affect your workloads.

v1.8.3

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.8.3

April 10, 2024

• Added support for Red Hat 9.3.

• Fixed an issue where the VSwitch selection policy failed to apply.

• Fixed an eRDMA quota calculation issue.

• Reduced excessive alerts for invalid resource events.

• Fixed a Hubble data collection issue.

• Fixed a forwarding issue in DataPath V2 mode.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.8.0

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.8.0

March 18, 2024

• Added support for DataPath V2.

• Added support for alibaba-addon-secret in registered clusters.

When creating a cluster, selecting either DataPathV2 or IPvlan enables DataPathV2.

v1.7.4

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.7.4

March 18, 2024

Fixed an issue where tokens might fail to update.

This upgrade does not affect your workloads.

v1.7.3

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.7.3

March 11, 2024

• Added support for Lingjun clusters.

• Fixed an issue with eRDMA quota calculation.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.7.2

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.7.2

February 28, 2024

• Fixed slow IP address assignment.

• Fixed a bug where resource warmup did not take effect.

This upgrade does not affect your workloads.

v1.7.0

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.7.0

February 18, 2024

• Added support for ERDMA.

• Improved elastic network interface allocation efficiency.

• Reduced kube-apiserver load in IPvlan mode.

• Terway no longer supports EIPs.

• Requires Kubernetes 1.24 or later.

To continue using EIPs, see Migrate EIPs from Terway to ack-extend-network-controller.

In large clusters, the kube-apiserver load may increase temporarily during the upgrade. Monitor your kube-apiserver load.

Dedicated clusters use a node RAM role for OpenAPI authentication. Clusters created before March 1, 2020 may lack the vpc:DescribeVSwitches permission. Without this permission, some Terway features may not function correctly. To add this permission to the node RAM role, see Use RBAC to authorize operations on cluster resources.

Version

Image address

Date

Changes

Impact

v1.6.3

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.6.3

January 24, 2024

• Fixed a potential overwrite of the node condition.

• Fixed the failure to clean up node rules in policy routing mode.

• Fixed delayed IP address reclamation after a stateful application is deleted.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.6.1

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.6.1

November 21, 2023

• Fixed an issue that prevented Service backends from updating in IPvlan mode.

• Fixed a duplicate IP allocation issue.

This upgrade does not affect your workloads.

v1.4.9

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.9

November 21, 2023

Fixed a duplicate IP allocation issue.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.6.0

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.6.0

October 10, 2023

• Added support for migrating EIPs from Terway to ack-extend-network-controller.

• Added support for configuring terway-controlplane as a StatefulSet.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.5.7

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.5.7

August 24, 2023

• Reduced Terway startup time.

• Fixed occasional health check timeouts.

No service impact.

Version

Image address

Date

Changes

Impact

v1.5.6

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.5.6

July 30, 2023

Added support for configuring a custom CNI chain.

No workload impact.

v1.5.5

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.5.5

July 6, 2023

Added support for configuring a whitelist for an elastic network interface (ENI).

No workload impact.

Version

Image address

Date

Changes

Impact

v1.5.4

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.5.4

June 1, 2023

Fixed an issue that could cause network jitter during an upgrade in IPvlan mode.

This upgrade does not affect your workloads.

v1.4.8

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.8

June 1, 2023

Fixed an issue that could cause network jitter during an upgrade in IPvlan mode.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.5.3

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.5.3

May 16, 2023

Fixed an issue in v1.4.4 where an ExternalIP might be inaccessible.

This upgrade does not affect your workloads.

v1.5.2

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.5.2

May 10, 2023

• Fixed an issue where a failed upgrade could cause nodes to become NotReady.

• Fixed an issue where the webhook injected redundant availability zone constraints in trunking mode.

This upgrade does not affect your workloads.

v1.4.6

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.6

May 10, 2023

Fixed an issue where a failed upgrade could cause nodes to become NotReady.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.5.1

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.5.1

April 28, 2023

• Added support for Kubernetes 1.26. Terway now requires Kubernetes 1.22 or later.

• Upgraded Felix to v3.24.5.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.4.5

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.5

March 31, 2023

Reduced RBAC (Role-Based Access Control) permissions.

This upgrade does not affect your workloads.

v1.4.4

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.4

March 14, 2023

• Added support for modifying VSwitch configurations in component management.

• Reduced the component's high memory usage in IPvlan mode.

• Upgraded Cilium to v1.12.7.

• Upgraded iptables to v1.8.8.

If your cluster uses IPvlan mode, you cannot access a LoadBalancer service from within the cluster by using its ExternalIP when the service uses externalTrafficPolicy=Local and the backend is on a different node. To work around this, access the service by using its ClusterIP or change the setting to externalTrafficPolicy=Cluster.

Version

Image address

Date

Changes

Impact

v1.4.3

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.3

December 23, 2022

• Upgraded Cilium to v1.12.4.

• Fixed a duplicate Pod IP allocation issue.

• Changed the component upgrade strategy to RollingUpdate.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.4.2

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.2

November 21, 2022

• Fixed a compatibility issue between trunking mode and LocalDNS.

• Fixed a compatibility issue between NetworkPolicy (in non-IPvlan mode) and the pod PostStartHook.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.4.1

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.1

September 28, 2022

• Added configurable parameters for the terway-eniip component on the Component Management page.

• Fixed an issue that prevented hairpin access.

• Fixed an IP address allocation issue caused by out-of-order CNI execution.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.4.0

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.4.0

August 29, 2022

• Upgraded Cilium to v1.12.1.

• Requires Kubernetes 1.20 or later.

For clusters with IPvlan mode enabled, this upgrade updates Cilium-related CRDs, which may temporarily increase kube-apiserver load on large clusters. Monitor kube-apiserver load.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.3.0

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.3.0

July 22, 2022

Added pod QoS management. See Configure QoS for Pods.

No impact on your workloads.

Version

Image address

Date

Changes

Impact

v1.2.4

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.2.4

June 28, 2022

• Fixed an issue where the portmap plugin was ineffective.

• Fixed an issue that prevented EIPs from being deleted.

• Fixed uneven backend traffic distribution for Services in IPvlan mode.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.2.3

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.2.3

May 26, 2022

• Adds support for Premium EIPs and EIP bandwidth packages.

• Fixes an issue that caused Kubelet container health checks to fail when using Cilium NetworkPolicy.

No impact on existing workloads.

Version

Image address

Date

Changes

Impact

v1.2.2

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.2.2

March 31, 2022

Fixed an ARP detection issue in VLAN mode.

This upgrade does not affect your workloads.

v1.2.1

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.2.1

March 15, 2022

• Fixed an issue where calico-felix reverted to legacy iptables mode on Alinux 3.

• Fixed IP reclamation failures caused by metadata exceptions.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.2.0

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.2.0

January 11, 2022

• Improved lock efficiency during CNI execution.

• Pods can now access their own services in IPvlan mode.

• In IPvlan mode, LoadBalancer traffic to an external IP from within a cluster is now load balanced across service backends. Enabled by default in new clusters. For existing clusters, see How do I enable in-cluster load balancing for a Terway IPvlan cluster?.

• Fixed false alerts in Terway VPC mode.

• Improved calico-felix performance in large-scale clusters.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.1.1

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.1.1

December 20, 2021

• Improved resource reclamation for elastic network interfaces to mitigate IPvlan creation failures caused by netns leaks in containerd environments.

• Added support for configuring multiple security groups for an ENI. See Configure multiple security groups for an ENI.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.1.0

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.1.0

November 22, 2021

• Added IPv6 dual stack support.

• Upgraded Felix to v3.20.2.

• Upgraded CNI from 0.3.0 to 0.3.1.

• Updated the deployment template for Kubernetes 1.22. The minimum supported Kubernetes version is now 1.18.

• Fixed an intermittent network interface card configuration issue in IPvlan mode. See #261.

• Added a 100Mi memory request to the Policy container.

• If your cluster uses Typha to improve NetworkPolicy scalability, update the component version. See Improve NetworkPolicy scalability for large Terway clusters.

• If your cluster does not use Typha, this upgrade will not affect your workloads.

• If your cluster does not use IPvlan mode, the upgrade updates the calico-felix CRDs. In large clusters, the kube-apiserver load may briefly increase. Monitor your kube-apiserver load.

• Ensure that your nodes have sufficient memory to prevent Pod eviction by Kubernetes during the upgrade.

Version

Image address

Date

Changes

Impact

v1.0.10.443-gaa1bfcc-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.443-gaa1bfcc-aliyun

September 14, 2021

Reduced network latency.

This update applies only to users running Terway in exclusive ENI mode. No update is required for other modes.

This update has no impact on your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.398-g63d2e57-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.398-g63d2e57-aliyun

August 4, 2021

• Fixed intermittent NetworkPolicy failures in multi-IP mode.

• Upgraded the Cilium add-on to v1.10.

For clusters that use IPvlan mode, this upgrade updates Cilium-related CRDs. In large-scale clusters, this may temporarily increase kube-apiserver load. Monitor kube-apiserver load.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.390-g5f3c461-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.390-g5f3c461-aliyun

July 2, 2021

• Fixed a permission issue that prevented modifying pod annotations when using EIPs in the Terway network mode.

• Fixed an issue where Terway failed to start when using exclusive ENIs.

• Added support for the containerd network namespace path.

• Added support for Alibaba Cloud Linux 3.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.368-g2890967-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.368-g2890967-aliyun

May 24, 2021

• Fixed an issue with the ResourceInvalid alert in EIP scenarios.

• Added support for Pods in IPvlan mode to communicate directly with their host node, bypassing the VPC by forwarding data locally.

• Added proactive checking and correction of the ip forwarding configuration.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.333-gfd2b7b8-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.333-gfd2b7b8-aliyun

April 26, 2021

• Fixed an address conflict for stateful applications using an Elastic IP Address.

• Added support for enabling Hubble in IPvlan mode. See What is Hubble.

No impact on your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.323-g778c128-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.323-g778c128-aliyun

March 22, 2021

• Reduced the number of OpenAPI requests.

• Improved event alerts for security group configuration checks.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.317-g0652857-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.317-g0652857-aliyun

February 22, 2021

• Fixed a Cilium Identity resource leak.

• Fixed an intermittent IP management issue for stateful application pods.

This upgrade does not affect your workloads.

v1.0.10.309-g5314eee-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.309-g5314eee-aliyun

February 5, 2021

• Removed proactive auto-fix logic for security group rules. Terway now reports required fixes only through events.

• Fixed an intermittent issue where pod IPs were reclaimed and then reassigned on nodes under high load.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.301-g0115576-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.301-g0115576-aliyun

January 21, 2021

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.280-gdc2cb6c-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.280-gdc2cb6c-aliyun

December 25, 2020

• Added proactive checks, including:

  • ENIs and their secondary IP addresses.

  • Security group consistency.

  • Pod network namespace configurations.

  • Host network configurations.

• Now supports host network stack routing in IPVLAN mode.

This upgrade does not affect your workloads.

v1.0.10.263-gdbe50a9-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.263-gdbe50a9-aliyun

December 3, 2020

Fixed an error in exclusive ENI mode.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.261-g8342155-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.261-g8342155-aliyun

November 27, 2020

• Added support for synchronous ECS API calls.

• Fixed an issue where the CNI plugin returned an error.

This upgrade does not affect your workloads.

v1.0.10.250-gb7bb10a-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.250-gb7bb10a-aliyun

November 23, 2020

• Fixed a Pod networking issue caused by ENI driver load failures.

• Fixed an issue where the status of released IPs was incorrect during ENI OpenAPI call throttling.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.247-g4cb77d0-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.247-g4cb77d0-aliyun

October 26, 2020

Adds support for ECS Dedicated Host instance types.

No impact on your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.237-g6a0f948-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.237-g6a0f948-aliyun

September 16, 2020

Accelerated ENI attachment.

This is a non-disruptive upgrade.

Version

Image address

Date

Changes

Impact

v1.0.10.221-g8d6386a-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.221-g8d6386a-aliyun

August 11, 2020

Adds support for IPVLAN container network with eBPF.

This upgrade does not affect your workloads.

v1.0.10.213-g27145cc-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.213-g27145cc-aliyun

August 4, 2020

Fixes an intermittent issue where an elastic network interface (ENI) would become unusable, causing pod network failures.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.208-gf3144bf-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.208-gf3144bf-aliyun

July 20, 2020

• Fixed policy-based routing leaks on high-density nodes.

• Added support for OpenAPI over the internal network.

• Fixed an issue that prevented Pod IP addresses from being released when the vSwitch was full.

• Improved the error messages for CNI failures.

This upgrade does not affect your workloads.

v1.0.10.211-gef088a4-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.211-gef088a4-aliyun

July 24, 2020

Added a cluster ID tag to ENIs.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.156-g8660a0f-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.156-g8660a0f-aliyun

April 22, 2020

• Optimized network resource caching for elastic network interfaces (ENIs).

• Upgraded the built-in Felix to v3.5.8.

• Enabled releasing network resources from Pods in the Completed or Failed state.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.139-g14a4f84-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.139-g14a4f84-aliyun

February 12, 2020

Resolved intermittent Pod creation timeouts.

No impact on your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.133-g001396b-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.133-g001396b-aliyun

January 10, 2020

• Added support for disabling NetworkPolicy.

• Added support for using IPVlan for Pod networking in multi-IP ENI clusters.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.0.10.122-gd0be015-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.122-gd0be015-aliyun

December 24, 2019

Optimized the allocation of multi-IP elastic network interfaces.

This is a non-disruptive upgrade.

Version

Image address

Date

Changes

Impact

v1.0.10.100-g92a3fa5-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.10.100-g92a3fa5-aliyun

October 11, 2019

Fixed an issue that caused a node to become NotReady when many Jobs ran concurrently.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.0.9.20-g35ae000-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.9.20-g35ae000-aliyun

August 23, 2019

Compatible with Kubernetes 1.14.6.

Your services remain unaffected.

Version

Image address

Date

Changes

Impact

v1.0.9.15-g3957085-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.9.15-g3957085-aliyun

April 11, 2019

Fixed an intermittent failure during Terway upgrades.

This upgrade does not affect your workloads.

Version

Image address

Date

Changes

Impact

v1.0.9.14-ga0346bb-aliyun

registry.cn-hangzhou.aliyuncs.com/acs/terway:v1.0.9.14-ga0346bb-aliyun

March 28, 2019

• Fixed a bug that prevented Terway from retrieving elastic network interface information when the meta server was being throttled.

• Fixed a bug where container creation failed with the error: failed to move veth to host network namespace: file exists.

• Added periodic scanning and reclamation for abnormally released elastic network interfaces.

• Optimized health checks by switching from an HTTP path check to a TCP port check.

This upgrade does not affect your workloads.