### Internal Storage Layer

    - [SIG Node](#sig-node)
    - [SIG Network](#sig-network)
    - [SIG Scalability](#sig-scalability)
    - [SIG Scheduling](#sig-scheduling)
    - [SIG Storage](#sig-storage)
  - [Before Upgrading](#before-upgrading)
  - [Known Issues](#known-issues)
  - [Deprecations](#deprecations)
    - [Apps](#apps)
    - [Auth](#auth)
    - [Autoscaling](#autoscaling)
    - [Cluster Lifecycle](#cluster-lifecycle)

    - [Client Binaries](#client-binaries-16)
    - [Server Binaries](#server-binaries-16)
    - [Node Binaries](#node-binaries-16)
  - [**Major Themes**](#major-themes)
  - [**Action Required Before Upgrading**](#action-required-before-upgrading)
    - [Network](#network)
    - [Storage](#storage)
    - [API Machinery](#api-machinery)
    - [Controller Manager](#controller-manager)
    - [kubectl (CLI)](#kubectl-cli)

    - [Instrumentation](#instrumentation)
    - [Network](#network)
    - [Node](#node)
    - [OpenStack](#openstack)
    - [Storage](#storage)
    - [Windows](#windows)
  - [Before Upgrading](#before-upgrading)
    - [**API Machinery**](#api-machinery-1)
    - [**Auth**](#auth-1)
    - [**CLI**](#cli)
    - [**Cluster Lifecycle**](#cluster-lifecycle-1)
    - [**Multicluster**](#multicluster)

    - [Windows](#windows)
    - [OpenStack](#openstack)
    - [API-machinery](#api-machinery)
    - [Auth](#auth)
    - [Azure](#azure)
    - [CLI](#cli)
    - [Network](#network)
  - [Before Upgrading](#before-upgrading)
  - [Known Issues](#known-issues)
  - [Deprecations](#deprecations)
  - [Other Notable Changes](#other-notable-changes-13)
    - [Apps](#apps)
    - [AWS](#aws)
    - [Auth](#auth-1)

## Urgent Upgrade Notes

### (No, really, you MUST do this before you upgrade)

Before upgrading to Kubernetes 1.13, you must keep the following in mind:

- kube-apiserver

    - [SIG OpenStack](#sig-openstack)
    - [SIG Scheduling](#sig-scheduling)
    - [SIG Storage](#sig-storage)
    - [SIG Windows](#sig-windows)
  - [Known Issues](#known-issues)
  - [Before Upgrading](#before-upgrading)
      - [New Deprecations](#new-deprecations)
      - [Removed Deprecations](#removed-deprecations)
      - [Graduated to Stable/GA](#graduated-to-stablega)
      - [Graduated to Beta](#graduated-to-beta)

  
  vSphere releases less than 67u3 are deprecated as of v1.21. Please consider upgrading vSphere to 67u3 or above. vSphere CSI Driver requires minimum vSphere 67u3.
  
  VM Hardware version less than 15 is deprecated as of v1.21. Please consider upgrading the Node VM Hardware version to 15 or above. vSphere CSI Driver recommends Node VM's Hardware version set to at least vmx-15.
  

dynamic resources to Pods.

  if and only if classic DRA was enabled in a cluster, remove all workloads (pods, app deployments, etc. ) which depend on classic DRA and make sure that all PodSchedulingContext resources are gone before upgrading. PodSchedulingContext resources cannot be removed through the apiserver after an upgrade and workloads would not work properly. ([#128003](https://github.com/kubernetes/kubernetes/pull/128003), [@pohly](https://github.com/pohly)) [SIG API Machinery,...

- Multiple Kubernetes clusters can now share the same resource group
    - When upgrading from previous releases, issues will arise with public IPs if multiple clusters share the same resource group. To solve these problems, make the following changes to the cluster: