Note that snapshot support will require Kubernetes distributors to bundle the Snapshot controller, Snapshot CRDs, and validation webhook. In addition, a CSI driver supporting the snapshot functionality must also be deployed on the cluster.

### Non-recursive Volume Ownership (FSGroup) graduates to Beta

The main themes of this release are:

- **Custom resources:** CRDs are in widespread use as a way to extend Kubernetes to persist and serve new resource types, and have been available in beta since the 1.7 release. The 1.16 release marks the graduation of CRDs to general availability (GA).

- Fixed a bug that prevents kubectl to validate CRDs with schema using x-kubernetes-preserve-unknown-fields on object fields.

pkg syscall (openbsd-386), const IP_IPSEC_LOCAL_AUTH = 27
pkg syscall (openbsd-386), const IP_IPSEC_LOCAL_CRED = 25
pkg syscall (openbsd-386), const IP_IPSEC_LOCAL_ID = 23
pkg syscall (openbsd-386), const IP_IPSEC_REMOTE_AUTH = 28
pkg syscall (openbsd-386), const IP_IPSEC_REMOTE_CRED = 26
pkg syscall (openbsd-386), const IP_IPSEC_REMOTE_ID = 24
pkg syscall (openbsd-386), const IP_MAXPACKET = 65535

in that cluster. Currently, the minimum we require is the security provider (node agent or citadel).

### Install Istio CRDs

This is the first step of the installation. Please do not remove or edit any CRD - config currently requires
all CRDs to be present. On each upgrade it is recommended to reapply the file, to make sure
you get all CRDs.  CRDs are separated by release and by component type in the CRD directory.

- Reduce API server memory when many CRDs are loaded by sharing a single etcd3 client logger across all clients ([#111648](https://github.com/kubernetes/kubernetes/pull/111648), [@negz](https://github.com/negz)) [SIG API Machinery]

- Reduce API server memory when many CRDs are loaded by sharing a single etcd3 client logger across all clients ([#111650](https://github.com/kubernetes/kubernetes/pull/111650), [@negz](https://github.com/negz)) [SIG API Machinery]

```
λ CRED_DIR=/media/builder/minio make hotfix-push
```

#### Builds the hotfix container and pushes to docker.io/minio/minio

```
λ CRED_DIR=/media/builder/minio make docker-hotfix-push
```

#### Builds the hotfix container and pushes to registry.min.dev/<customer>/minio

```
λ REPO="registry.min.dev/<customer>" CRED_DIR=/media/builder/minio make docker-hotfix-push
```

- Reduce API server memory when many CRDs are loaded by sharing a single etcd3 client logger across all clients ([#111649](https://github.com/kubernetes/kubernetes/pull/111649), [@negz](https://github.com/negz)) [SIG API Machinery]

Functionality offered by `kclient` includes:
* Typed clients (via generics) and more ergonomic APIs
* Ability to make a _delayed_ client. This is used when making clients based on CRDs that may not exist.
  In general, Istio does not fail on missing CRDs and prefers to treat these as if there were just zero resources for that client.
  The `NewDelayedInformer` abstracts that entirely, offering the same API as normal `kclient.Client`.