- [Additional Notable Feature Updates](#additional-notable-feature-updates)
  - [Known Issues](#known-issues)
  - [Urgent Upgrade Notes](#urgent-upgrade-notes)
    - [(No, really, you MUST read this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade)
      - [API Machinery](#api-machinery)
      - [Apps](#apps)
      - [Auth](#auth)
      - [AWS](#aws)
      - [Azure](#azure)

- kubeadm: `kubeadm upgrade node` now supports `addon` and `post-upgrade` phases. Users can use `kubeadm upgrade node phase addon` to execute the addon upgrade, or use `kubeadm upgrade node --skip-phases addon` to skip the addon upgrade. If you were previously skipping an addon subphase on `kubeadm init` you should now skip the same addon when calling `kubeadm upgrade apply` and `kubeadm upgrade node`. Currently, the `post-upgrade` phase is no-op, and it is mainly used to handle some release-specific...

* **Tolerations**

    - [Etcd v3.5.[0-2] data corruption](#etcd-v350-2-data-corruption)
  - [Urgent Upgrade Notes](#urgent-upgrade-notes)
    - [(No, really, you MUST read this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade)
  - [Changes by Kind](#changes-by-kind-16)
    - [Deprecation](#deprecation)
    - [API Change](#api-change-2)
    - [Feature](#feature-9)

  - [Urgent Upgrade Notes](#urgent-upgrade-notes)
    - [(No, really, you MUST read this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade)
  - [Changes by Kind](#changes-by-kind-15)
    - [Deprecation](#deprecation)
    - [API Change](#api-change-4)
    - [Feature](#feature-14)

are hence enabled by default. This will be fixed in a future release.
[#38593](https://github.com/kubernetes/kubernetes/issues/38593)
* Federation control plane can be upgraded by updating the image
fields in the `Deployment` specs of the control plane components.
However, federation control plane upgrades were not tested in this
release [38537](https://github.com/kubernetes/kubernetes/issues/38537)

## Notable Changes to Existing Behavior

        * Then upgrade nodes to Kubernetes v1.3.
            * As nodes are upgraded, they will automatically, by default, opt-in to attach/detach controller management, which will cause the controller to start managing attaches/detaches for volumes that get scheduled to those nodes.

### Other notable changes

* kubeadm upgrade no longer races leading to unexpected upgrade behavior on pod restarts ([#62655](https://github.com/kubernetes/kubernetes/pull/62655), [@stealthybox](https://github.com/stealthybox))
    * kubeadm upgrade now successfully upgrades etcd and the controlplane to use TLS
    * kubeadm upgrade now supports external etcd setups
    * kubeadm upgrade can now rollback and restore etcd after an upgrade failure

# shellcheck disable=SC2120
cleanup() {
	MINIO_VERSION=dev /tmp/gopath/bin/docker-compose \
		-f "buildscripts/upgrade-tests/compose.yml" \
		down || true

	MINIO_VERSION=dev /tmp/gopath/bin/docker-compose \
		-f "buildscripts/upgrade-tests/compose.yml" \
		rm || true

	for volume in $(docker volume ls -q | grep upgrade); do
		docker volume rm ${volume} || true
	done

	docker volume prune -f
	docker system prune -f || true

    - [Node Binaries](#node-binaries-14)
    - [Container Images](#container-images-14)
  - [Changelog since v1.30.0](#changelog-since-v1300)
  - [Urgent Upgrade Notes](#urgent-upgrade-notes)
    - [(No, really, you MUST read this before you upgrade)](#no-really-you-must-read-this-before-you-upgrade)
  - [Changes by Kind](#changes-by-kind-14)
    - [Deprecation](#deprecation-1)
    - [API Change](#api-change-2)
    - [Feature](#feature-10)