(Discontinued) ACK release notes for Kubernetes 1.28

Kubernetes 1.28

Non-graceful node shutdown (GA)

When a node shuts down unexpectedly—for example, due to a power failure—Kubernetes can automatically reschedule StatefulSet pods with the same name on another node, preventing service interruptions. Now GA.

Retroactive default StorageClass assignment (GA)

Previously, a PVC without storageClassName remained Pending indefinitely if no default StorageClass existed. Kubernetes now assigns a later-created default StorageClass to unbound PVCs without storageClassName. Now GA.

Scheduler performance

The scheduler's retry logic reduces invalid scheduling attempts, improving performance. If your cluster uses a custom scheduler plugin, update it for the new scheduling framework. See Scheduling framework changes.

Job failure handling

Two new Alpha feature gates improve how failed Jobs are handled:

• `JobPodReplacementPolicy`: A replacement pod is created only when the original reaches the Failed phase (status.phase: Failed), not when it has a deletionTimestamp. This prevents two pods from simultaneously occupying the same index and consuming node resources.

• `JobBackoffLimitPerIndex`: Set .spec.backoffLimitPerIndex on an Indexed Job to cap the number of retries per index, rather than counting all failures toward a single shared .spec.backoffLimit. A persistently failing index no longer causes the entire Job to fail.

Large Indexed Job warning

If a Job has completion above 100,000 and parallelism above 10,000 with many pod failures, Kubernetes displays a warning—termination status may not track reliably at that scale.

CRD validation improvements

reason and fieldPath fields are added to CRD validation rules. When a CEL expression fails, the API server returns the reason code and failing field path. See CRD Validation Expression Language.

Webhook matching with CEL expressions

Webhook matchConditions now support CEL expressions (up to 64 per webhook), giving fine-grained control over which requests trigger a webhook without a full admission controller. See Matching requests: matchConditions.

ValidatingAdmissionPolicy (Beta)

ValidatingAdmissionPolicy provides a declarative, in-cluster alternative to validating admission webhooks. Write CEL expressions directly in the policy object; the API server evaluates them on every matching request—no webhook infrastructure needed. See ValidatingAdmissionPolicy.

PVC resize status field replaced

The .status.resizeStatus field on PVCs is replaced by .status.allocatedResourceStatus, a map that tracks the resize state of each resource independently. Update any tooling that reads .status.resizeStatus. See PersistentVolumeClaimStatus.

Pod index labels for Indexed Jobs and StatefulSets

Pods from Indexed Jobs and StatefulSets now include their ordinal index as a label, simplifying pod selection by position.

Kube Controller Manager concurrency flags

Two new flags control controller concurrency:

• --concurrent-cron-job-syncs: Sets the number of concurrent CronJob reconciliation workers.

• --concurrent-job-syncs: Sets the number of concurrent Job reconciliation workers.

See --concurrent-cron-job-syncs and --concurrent-job-syncs.

API server optimizations

• Reduced GetList memory: Cache-served list requests use less memory. See GetList test data.

• Endpoint cleanup fix: Fixed a bug where a Kubernetes Service endpoint was not removed when only one API server replica remained during graceful shutdown.

• OpenAPI v2 lazy aggregation: The OpenAPI v2 controller now aggregates CRD information only on request, reducing CPU and memory overhead when no client polls the endpoint. This improves install time for large numbers of CRDs but may slow the first request. Migrate clients to OpenAPI v3.

• ConsistentListFromCache (Alpha): The ConsistentListFromCache feature gate lets the API server serve LIST requests from the watch cache with consistency guarantees, reducing etcd load.

• Additional monitoring metrics: More metrics are available through the metrics endpoint.

Kubernetes 1.27

Pod termination status corrected

• Pods deleted while Pending are now set to Failed.

• Pods deleted while Running are set to Succeeded or Failed based on container exit code.

This fixes a bug where pods with a failure policy could stay stuck in Pending. If a pod has RestartPolicy=Always, it may show Succeeded after deletion—review your controllers if this matters. See Set the termination status for pods that do not require a restart.

ReadWriteOncePod for PersistentVolumes (Beta)

ReadWriteOncePod limits a volume to a single pod—more restrictive than ReadWriteOnce, which allows multiple pods on the same node. See Single Pod Access Mode for PersistentVolumes graduates to Beta.

Pod topology spread constraints (Beta)

Several topology spread constraint options are now Beta and enabled by default: minDomains (minimum eligible domains), nodeTaintsPolicy (count tainted nodes), nodeAffinityPolicy (count affinity-excluded nodes), and whenUnsatisfiable (behavior when constraints cannot be met). See More fine-grained pod topology spread policies.

Server-side field validation (GA)

The API server now validates resource fields server-side. kubectl automatically uses Strict mode and skips client-side validation, returning an error if the server rejects a field. See Server-side field validation and OpenAPI v3 move to GA.

OpenAPI v3 (GA)

OpenAPI v3, introduced in Kubernetes 1.23, is now GA. See Server Side Field Validation and OpenAPI V3 move to GA.

HPA ContainerResource metric type (Beta)

HPA now supports the ContainerResource metric type, scaling based on individual container resource usage rather than pod average. This fixes a common issue where a low-usage sidecar pulls the average below the threshold, preventing scale-out even when the application container is resource-constrained. See Container resource metrics.

StatefulSet enhancements (Beta)

Two StatefulSet features reach Beta:

• `StatefulSetStartOrdinal`: Start pod ordinals at a number other than zero using the ordinals.start field.

• `StatefulSetAutoDeletePVC`: Control whether StatefulSets delete PVCs created from volumeClaimTemplate when pods are deleted or the StatefulSet is scaled in.

In-place pod vertical scaling (Alpha)

InPlacePodVerticalScaling (Alpha, disabled by default) lets you change CPU and memory requests and limits for a running container without restarting the pod. See Resize CPU and memory resources assigned to containers.

Parallel image pulls

Set serializeImagePulls: false in the kubelet configuration to pull images in parallel. Use maxParallelImagePulls (added in 1.27) to cap concurrent pulls and avoid saturating network bandwidth or disk I/O. See Serial and parallel image pulls.

Volume group snapshots (Alpha)

A volume group snapshot API lets you create crash-consistent snapshots of multiple PersistentVolumes at once. See Introducing an API for volume group snapshots.

Kubernetes 1.28

Kubernetes 1.27

Deprecated APIs

The storage.k8s.io/v1beta1 CSIStorageCapacity API was deprecated in 1.24 and removed in 1.27. Use storage.k8s.io/v1 (available since 1.24). See Storage Capacity Constraints for Pod Scheduling KEP.

Promoted to GA

These feature gates reached GA and are permanently enabled. The gates are removed.

In Kubernetes 1.28:

In Kubernetes 1.27:

Promoted to Beta (enabled by default)

In Kubernetes 1.28:

In Kubernetes 1.27:

New in Alpha (disabled by default)

In Kubernetes 1.28:

In Kubernetes 1.27:

Removed (no longer needed to configure)

In Kubernetes 1.27:

The IPv6DualStack feature gate was removed after reaching GA in v1.23. If you manually set this in your cluster configuration, remove it before upgrading.