- [Container Storage Interface graduations](#container-storage-interface-graduations)
    - [SIG Windows development tools](#sig-windows-development-tools)
    - [Deploy a more secure control plane with kubeadm](#deploy-a-more-secure-control-plane-with-kubeadm)
    - [etcd moves to version 3.5.0](#etcd-moves-to-version-350)
    - [Kubernetes Node system swap support](#kubernetes-node-system-swap-support)

- Kubeadm: add a upgrade health check that deploys a Job ([#81319](https://github.com/kubernetes/kubernetes/pull/81319), [@neolit123](https://github.com/neolit123))

 - `kubeadm`: a separate "super-admin.conf" file is now deployed. The User in `admin.conf` is now bound to a new RBAC Group `kubeadm:cluster-admins` that has `cluster-admin` `ClusterRole` access. The User in `super-admin.conf` is now bound to the `system:masters` built-in super-powers / break-glass Group that...

## Changelog since v1.20.0

## What's New (Major Themes)

### Deprecation of PodSecurityPolicy

* Updated Dashboard add-on to version 1.8.0: The Dashboard add-on now deploys with https enabled. The Dashboard can be accessed via kubectl proxy at http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/. The /ui redirect is deprecated and will be removed in 1.10. ([#53046](https://...

## Prerequisites

- Install and configure MinIO Server from [here](https://docs.min.io/community/minio-object-store/operations/deployments/baremetal-deploy-minio-on-redhat-linux.html#procedure).
- Install and configure MinIO Client from [here](https://docs.min.io/community/minio-object-store/reference/minio-mc.html#quickstart).

```
$ mc admin config get myminio | grep notify

Note that snapshot support will require Kubernetes distributors to bundle the Snapshot controller, Snapshot CRDs, and validation webhook. In addition, a CSI driver supporting the snapshot functionality must also be deployed on the cluster.

### Non-recursive Volume Ownership (FSGroup) graduates to Beta

- Eliminated work when creating Services or understanding port purposes, especially for external resources deployed via Helm charts. ([#133018](https://github.com/kubernetes/kubernetes/pull/133018), [@rushmash91](https://github.com/rushmash91))

- CSIStorageCapacity.storage.k8s.io: The v1beta1 version of this API is deprecated in favor of v1, and will be removed in v1.27. If a CSI driver supports storage capacity tracking, then it must get deployed with a release of external-provisioner that supports the v1 API. ([#108445](https://github.com/kubernetes/kubernetes/pull/108445), [@pohly](https://github.com/pohly))

* Updated admission controller settings for Juju deployed Kubernetes clusters ([#61427](https://github.com/kubernetes/kubernetes/pull/61427), [@hyperbolic2346](https://github.com/hyperbolic2346))