- [Server Binaries](#server-binaries-17)
    - [Node Binaries](#node-binaries-17)
- [Changes](#changes)
  - [What’s New (Major Themes)](#whats-new-major-themes)
    - [Cloud Provider Labels reach General Availability](#cloud-provider-labels-reach-general-availability)
    - [Volume Snapshot Moves to Beta](#volume-snapshot-moves-to-beta)
    - [CSI Migration Beta](#csi-migration-beta)
  - [Known Issues](#known-issues)

- Containers which specify a `startupProbe` but not a `readinessProbe` were previously considered "ready" before the `startupProbe` completed, but are now considered "not-ready". ([#92196](https://github.com/kubernetes/kubernetes/pull/92196), [@thockin](https://github.com/thockin)) [SIG Node]

    * To avoid that negative effect only use per pod metrics from pods that are:
    * - ready (so metrics about them should be valid), or
    * - unready but creation and last readiness change timestamps are apart more than 10s (pods that have formerly been ready and so metrics are in at least some cases (pod becoming unready because of overload) very useful).

* fix azure disk not available issue when device name changed ([#57549](https://github.com/kubernetes/kubernetes/pull/57549), [@andyzhangx](https://github.com/andyzhangx))
* Allow kubernetes components to react to SIGTERM signal and shutdown gracefully. ([#57756](https://github.com/kubernetes/kubernetes/pull/57756), [@mborsz](https://github.com/mborsz))

- The e2e suite can be instructed not to wait for pods in kube-system to be ready or for all nodes to be ready by passing `--allowed-not-ready-nodes=-1` when invoking the e2e.test program. This allows callers to run subsets of the e2e suite in scenarios other than perfectly healthy clusters. ([#98781](https://github.com/kubernetes/kubernetes/pull/98781), [@...

*   Taint key `unreachable` is now in GA.
*   Taint key `notReady` is changed to `not-ready`, and is also now in GA.
*   These changes are automatically updated for taints. Tolerations for these taints must be updated manually. Specifically, you must:
    *   Change `node.alpha.kubernetes.io/notReady` to `node.kubernetes.io/not-ready`
    *   Change `node.alpha.kubernetes.io/unreachable` to  `node.kubernetes.io/unreachable`

- github.com/coreos/go-systemd/v22: [v22.3.2 → v22.4.0](https://github.com/coreos/go-systemd/v22/compare/v22.3.2...v22.4.0)
- github.com/creack/pty: [v1.1.11 → v1.1.18](https://github.com/creack/pty/compare/v1.1.11...v1.1.18)
- github.com/docker/docker: [v20.10.18+incompatible → v20.10.21+incompatible](https://github.com/docker/docker/compare/v20.10.18...v20.10.21)

change to a valid value after the device plugin re-connects with the Kubelet. If the extended resource is exported by an external component through direct node status capacity patching, the component should repatch the field after kubelet becomes ready again. During the time gap, pods previously assigned with such resources may fail kubelet admission but their controller should create new pods in response to such failures. ([#64784](https://github.com/kubernetes/kubernetes/pull/64784), [@jiaying...

* A new `TaintNodesByCondition` admission plugin taints newly created Node objects as "not ready", to fix a race condition that could cause pods to be scheduled on new nodes before their taints were updated to accurately reflect their reported conditions. This admission plugin is enabled by default if the `TaintNodesByCondition` feature is enabled....

* `kubectl wait` is a new command that allows waiting for one or more resources to be deleted or to reach a specific condition. It adds a `kubectl wait --for=[delete|condition=condition-name] resource/string` command. ([#64034](https://github.com/kubernetes/kubernetes/pull/64034), [@deads2k](https://github.com/deads2k))