Note that snapshot support will require Kubernetes distributors to bundle the Snapshot controller, Snapshot CRDs, and validation webhook. In addition, a CSI driver supporting the snapshot functionality must also be deployed on the cluster.

### Non-recursive Volume Ownership (FSGroup) graduates to Beta

  - example manifests for the kube-apiserver, kube-controller-manager, kube-schedule, kubelet, and the cloud-controller-manager
  - labels/annotations that are consumed by any cloud specific controllers

In addition to the above, SIG Cloud Provider has been focusing on a long running effort to remove cloud provider code from kubernetes/kubernetes.

### SIG-cluster-lifecycle

- Kubeadm: when adding new control plane nodes with "kubeadm join", ensure that the etcd member addition is performed only if a given member URL does not already exist in the list of members. Similarly, on "kubeadm reset" only remove an etcd member if its ID exists. ([#127622](https://github.com/kubernetes/kubernetes/pull/127622), [@SataQiu](https:...

* fixes kubefed's ability to create RBAC roles in version-skewed clusters ([#50537](https://github.com/kubernetes/kubernetes/pull/50537), [@liggitt](https://github.com/liggitt))
* Fix data race during addition of new CRD ([#50098](https://github.com/kubernetes/kubernetes/pull/50098), [@nikhita](https://github.com/nikhita))

- `kubectl exec` and `kubectl attach` now honor the `--quiet` flag which suppresses output from the local binary that could be confused by a script with the remote command output (all non-failure output is hidden). In addition, print inline with exec and attach the list of alternate containers when we default to the first spec.container. ([#99004](https://github.com/kubernetes/kubernetes/pull/99004), [@smarterclayton](https://github.com/smarterclayton)) [SIG CLI]

* In addition to restricting GCE metadata requests to known APIs, the metadata-proxy now restricts query strings to known parameters. ([#71094](https://github.com/kubernetes/kubernetes/pull/71094), [@dekkagaijin](https://github.com/dekkagaijin))

- Scheduler now retries updating a pod's status on `ServiceUnavailable` and `InternalError` errors, in addition to `net.ConnectionRefused` error. ([#111809](https://github.com/kubernetes/kubernetes/pull/111809), [@Huang-Wei](https://github.com/Huang-Wei))

            super.prepareOptions(options);
            options.addOption(Option.builder(MODEL_VERSION)
                    .longOpt("model-version")
                    .hasArg()
                    .argName("version")
                    .desc("Target POM model version (4.0.0 or 4.1.0)")
                    .get());
            options.addOption(Option.builder(DIRECTORY)
                    .longOpt("directory")

            super.prepareOptions(options);
            options.addOption(Option.builder(ALTERNATE_POM_FILE)
                    .longOpt("file")
                    .hasArg()
                    .desc("Force the use of an alternate POM file (or directory with pom.xml)")
                    .get());
            options.addOption(Option.builder(NON_RECURSIVE)
                    .longOpt("non-recursive")

        options.addOption(Option.builder(Character.toString(VERSION))
                .longOpt("version")
                .desc("Display version information")
                .build());
        options.addOption(Option.builder(Character.toString(QUIET))
                .longOpt("quiet")
                .desc("Quiet output - only show errors")
                .build());