Kubernetes v1.37 ‘Garhwal’: Key Stable Features, Deprecations, and Beta Updates

by

Kubernetes v1.37, named after the Garhwal region of India, arrives with 67 enhancements: 16 graduating to Stable, 23 to Beta, and 27 entering Alpha. This release focuses on API server resilience, storage management, and the eventual sunset of legacy features like cgroup v1.

Stable Graduations

The release hardens the control plane and improves observability with several key Stable features:

  • Resilient Watch Cache Initialization: The API server now handles watch cache warming more gracefully. Instead of overloading etcd with expensive list and watch requests, the server safely delegates bounded requests and rejects others with HTTP 429 responses. Clients must implement exponential backoff to handle these rate limits.
  • KYAML: A safer, less ambiguous subset of YAML designed for Kubernetes. While not a replacement for standard YAML, it allows for better parsing and is now Stable in kubectl.
  • metrics.k8s.io API: After nearly nine years in Beta, this API for retrieving CPU and memory usage for pods and nodes has reached Stability. This standardizes metrics retrieval for tools like kubectl top and HorizontalPodAutoscalers.
  • SELinuxMount and SELinuxChangePolicy: These flags graduate to Stable and are enabled by default. Volumes are now mounted with a specific context label rather than being recursively relabeled. This change can cause Pods with different SELinux labels sharing a volume to fail to start. To retain the old behavior, administrators should set .spec.seLinuxChangePolicy to Recursive on affected Pods.
  • Storage Version Migrator: The storagemigration.k8s.io/v1 API is now Stable and enabled by default. This allows administrators to automatically migrate existing resources to the latest storage version after an API upgrade or encryption change, reducing the need for manual scripts.
  • Pod Certificates and ClusterTrustBundles: First-class support for distributing private keys and X.509 certificates to Pods is now Stable, allowing workloads to opt into identity management.

Beta Highlights

Beta features in v1.37 aim to improve scheduling efficiency, cost management, and observability:

  • HorizontalPodAutoscaler Scale to Zero: HPA can now scale down to zero replicas for workloads using object or external metrics. This is useful for queue consumers and batch jobs that can be restored when demand returns. Note that CPU and memory metrics do not support scale-to-zero, as they require active Pods.
  • Gang Scheduling: This feature graduates to Beta, ensuring that a group of Pods is scheduled only if the cluster has enough resources for the entire group. This addresses deadlocks and inefficiencies common in AI/ML workloads.
  • etcd RangeStream: A new server-streaming RPC allows the API server to stream large list responses in chunks rather than buffering them entirely in memory. This significantly reduces memory spikes during watch cache initialization on large clusters.
  • Memory QoS with cgroups v2: The feature uses cgroup v2 controls (memory.min, memory.low, memory.high) to protect requested memory and throttle usage before hard limits are reached. This is enabled by default.
  • Manifest-based Admission Control: Admission webhooks and CEL policies can now be loaded from manifest files on disk. This configuration is enforced at startup and can protect itself from modification, providing a more robust mechanism during cluster bootstrap.

Deprecations and Removals

Several legacy features are being phased out, requiring action from cluster administrators:

  • Retirement of kube-dns: The kube-dns subproject is being retired. CoreDNS is the recommended replacement. No new packages will be built for kube-dns after v1.40.
  • Deprecation of kube-proxy ipvs mode: While still functional, ipvs mode is deprecated. Support is expected to be disabled by default in v1.40 and removed entirely in v1.43. The rationale is that ipvs alone cannot fully implement Kubernetes Services without relying on iptables underneath.
  • Removal of Static Pod API References: Static Pods can no longer reference Secrets or ConfigMaps. The feature gate that previously allowed this has been removed.
  • End of cgroup v1 Support: Support for the legacy cgroup v1 is being phased out. Since v1.35, the failCgroupV1 setting has defaulted to true. While an override exists, users are strongly encouraged to migrate to cgroup v2, as advanced resource management features like Memory QoS require it.

Alpha Features and Future Directions

Alpha features in v1.37 offer experimental capabilities:

  • Pod-level Checkpoint and Restore: Extending the CRI with new RPCs, this feature allows the kubelet to checkpoint and restore Pods. It requires a compatible container runtime.
  • Dynamic Resize of Memory-Backed Volumes: This Alpha feature allows the size limit of emptyDir volumes to be adjusted dynamically without restarting containers.

The release also introduces new APIs for complex scheduling scenarios, such as the CompositePodGroup API for hierarchical gang scheduling and improved integration between workload controllers and the scheduler.