* [stable] Support Attach and Detach operations for ReadWriteOnce (RWO) volumes that use iSCSI and Fibre Channel plugins.

* [stable] Expose storage usage metrics

  * The available capacity of a given Persistent Volume (PV) is available by calling the Kubernetes metrics API.

* [stable] Volume plugin metrics

cesena-forlì.it
cesenaforli.it
cesenaforlì.it
cf
cf-ipfs.com
cfa
cfd
cfolks.pl
cg
ch
ch.eu.org
ch.it
ch.trendhosting.cloud
chambagri.fr
championship.aero
chanel
channel
channelsdvr.net
charity
charter.aero
chase
chat
cheap
cheap.jp
cherkassy.ua
cherkasy.ua
chernigov.ua
chernihiv.ua
chernivtsi.ua
chernovtsy.ua
chiba.jp

	wg.Add(2)
	go publish(ReportMetrics(GlobalContext, c.metricsGroups))
	go publish(globalNotificationSys.GetClusterMetrics(GlobalContext))
	wg.Wait()
}

// ReportMetrics reports serialized metrics to the channel passed for the metrics generated.
func ReportMetrics(ctx context.Context, metricsGroups []*MetricsGroupV2) <-chan MetricV2 {
	ch := make(chan MetricV2)
	go func() {
		defer xioutil.SafeClose(ch)

* etcd: watcher centralize error handling ([#32907](https://github.com/kubernetes/kubernetes/pull/32907), [@hongchaodeng](https://github.com/hongchaodeng))
* etcd: stop watcher when watch channel is closed ([#33003](https://github.com/kubernetes/kubernetes/pull/33003), [@hongchaodeng](https://github.com/hongchaodeng))

- Fixed a storage bug related to multipath. iSCSI and Fibre Channel devices attached to nodes via multipath now resolve correctly when partitioned. ([#128086](https://github.com/kubernetes/kubernetes/pull/128086), [@RomanBednar](https://github.com/RomanBednar))

- cloud.google.com/go/bigquery: v1.58.0
- cloud.google.com/go/billing: v1.18.0
- cloud.google.com/go/binaryauthorization: v1.8.0
- cloud.google.com/go/certificatemanager: v1.7.4
- cloud.google.com/go/channel: v1.17.4
- cloud.google.com/go/cloudbuild: v1.15.0
- cloud.google.com/go/clouddms: v1.7.3
- cloud.google.com/go/cloudtasks: v1.12.4
- cloud.google.com/go/compute: v1.23.3

- `DelayingQueue.ShutDown()` can now be invoked multiple times without causing a closed channel panic. ([#77170](https://github.com/kubernetes/kubernetes/pull/77170), [@smarterclayton](https://github.com/smarterclayton))

* feat(fakeclient): push event on watched channel on add/update/delete ([#57504](https://github.com/kubernetes/kubernetes/pull/57504), [@yue9944882](https://github.com/yue9944882))

- cloud.google.com/go/beyondcorp: v0.5.0
- cloud.google.com/go/billing: v1.13.0
- cloud.google.com/go/binaryauthorization: v1.5.0
- cloud.google.com/go/certificatemanager: v1.6.0
- cloud.google.com/go/channel: v1.12.0
- cloud.google.com/go/cloudbuild: v1.9.0
- cloud.google.com/go/clouddms: v1.5.0
- cloud.google.com/go/cloudtasks: v1.10.0
- cloud.google.com/go/compute/metadata: v0.2.3

*   [alpha] Raw block volumes
    *   Support for surfacing volumes as raw block devices added to Kubernetes storage system.
    *   Only Fibre Channel volume plugin supports exposes this functionality, in this release.
*   [alpha] Volume resizing
    *   Added file system resizing for the following volume plugins: GCE PD, Ceph RBD, AWS EBS, OpenStack Cinder