- [Ephemeral Containers Graduate to Stable](#ephemeral-containers-graduate-to-stable)
    - [Support for cgroups v2 Graduates to Stable](#support-for-cgroups-v2-graduates-to-stable)
    - [Windows support improved](#windows-support-improved)
    - [Moved container registry service from k8s.gcr.io to registry.k8s.io](#moved-container-registry-service-from-k8sgcrio-to-registryk8sio)

# Kubernetes v1.14 Release Notes

## 1.14 What’s New

Support for Windows Nodes is Graduating to Stable ([#116](https://github.com/kubernetes/enhancements/issues/116) )

- Support for Windows Server 2019 for worker nodes and containers
- Support for out of tree networking with Azure-CNI, OVN-Kubernetes and Flannel

  
  ### Magnum
  - Add Magnum support in the Cluster Autoscaler helm chart
  
  ### Packet
  - Allow empty nodepools
  - Add support for multiple nodepools
  - Add pricing support
  
  ## Image
  Image: `k8s.gcr.io/autoscaling/cluster-autoscaler:v1.20.0` ([#97012](https://github.com/kubernetes/kubernetes/pull/97012), [@towca](https://github.com/towca)) [SIG Cloud Provider]

- Add support for PodAndContainerStatsFromCRI featuregate, which allows a user to specify their pod stats must also come from the CRI, not cAdvisor. (#103095, @haircommander) [SIG Node]
- Add support for Portworx plugin to csi-translation-lib. Alpha release
  
  Portworx CSI driver is required to enable migration.
  This PR adds support of the `CSIMigrationPortworx` feature gate, which can be enabled by:

  // This feature depends on whether the underlying cloud-provider supports specifying
  // the loadBalancerIP when a load balancer is created.
  // This field will be ignored if the cloud-provider does not support the feature.
  // Deprecated: This field was under-specified and its meaning varies across implementations,
  // and it cannot support dual-stack.

    - [SIG CLI introduces kubectl debug](#sig-cli-introduces-kubectl-debug)
    - [Introducing Windows CSI support alpha for Kubernetes](#introducing-windows-csi-support-alpha-for-kubernetes)
    - [Other notable announcements](#other-notable-announcements)
  - [Known Issues](#known-issues)
  - [Urgent Upgrade Notes](#urgent-upgrade-notes-1)

- Some of the in-tree storage drivers indicate support for the `MetricsProvider` interface, but fail to configure this for BlockMode volumes. With a recent change, `Kubelet` will call `GetMetrics()` for BlockMode volumes, and the in-tree drivers that miss the support cause a Go panic. Now the in-tree storage drivers that support BlockMode volumes, will return the Capacity of the volume in the `GetMetrics()` call. ([#101587](https://git...

  
  To activate this feature, users can set the `ENABLE_CLIENT_GO_WATCH_LIST_ALPHA` environmental variable.
  It is important to note that the server must support streaming for this feature to function properly.
  If streaming is not supported by the server, the reflector will revert to the previous method

- '`CSIMigrationvSphere` was upgraded to GA and locked to true. Do not upgrade to K8s
  1.26 if you need Windows, or XFS, or raw block support until vSphere CSI Driver
  adds support for them in a version post v2.7.x.'
   ([#113336](https://github.com/kubernetes/kubernetes/pull/113336), [@divyenpatel](https://github.com/divyenpatel))
- '`DelegateFSGroupToCSIDriver` feature is GA.'

  General Support for vSphere 6.7 will end on October 15, 2022. vSphere 6.7 Update 3 is deprecated in Kubernetes v1.24.  Customers are recommended to upgrade vSphere (both ESXi and vCenter) to 7.0u2 or above.  vSphere CSI Driver 2.2.3 and higher supports CSI Migration.