* **alpha**: NVIDIA GPU support ([#24836](https://github.com/kubernetes/kubernetes/pull/24836), [@therc](https://github.com/therc))
  * Adding loadBalancer services and nodeports services to quota system

## Known Issues and Important Steps before Upgrading

	},
	ErrAdminBucketQuotaExceeded: {
		Code:           "XMinioAdminBucketQuotaExceeded",
		Description:    "Bucket quota exceeded",
		HTTPStatusCode: http.StatusBadRequest,
	},
	ErrAdminNoSuchQuotaConfiguration: {
		Code:           "XMinioAdminNoSuchQuotaConfiguration",
		Description:    "The quota configuration does not exist",
		HTTPStatusCode: http.StatusNotFound,
	},
	ErrInsecureClientRequest: {

| `minio_cluster_usage_buckets_quota_total_bytes`                 | Total bucket quota in bytes. <br><br>Type: gauge                                     | `bucket`          |

The CustomCFSQuotaPeriod alpha feature enables node administrators to change the default period used to enforce CFS quota on a node.  This can improve performance for some workloads that experience latency while using CFS quota with the default measurement period.  Finally, the SIG continues to focus on improving reliability by fixing bugs while working out design and implementation of future features.

					VersioningSuspended: globalBucketVersioningSys.Suspended(bucket.Name),
					Replication:         rcfg != nil,
					Locking:             lcfg.LockEnabled,
					Quota:               quota,
					Tagging:             tcfg,
				},
				Access: madmin.AccountAccess{
					Read:  rd,
					Write: wr,
				},
			})
		}
	}

* Add an Azure CloudProvider Implementation ([#28821](https://github.com/kubernetes/kubernetes/pull/28821), [@colemickens](https://github.com/colemickens))

| `minio_bucket_usage_total_bytes`        | Total bucket size in bytes.                       |
| `minio_bucket_quota_total_bytes`        | Total bucket quota size in bytes.                 |

## Requests Metrics

| Name                                              | Description                                                     |

For more information, see [#53040](https://github.com/kubernetes/kubernetes/issues/53040).

* Object quotas are not consistently charged or updated. Specifically, the object count quota does not reliably account for uninitialized objects. Some quotas are charged only when an object is initialized. Others are charged when an object is created, whether it is initialized or not. We plan to fix this issue in a future release.

- Fixed a potential deadlock in the resource quota controller. Enabled recording partial usage info for quota objects specifying multiple resources, when only some of the resources' usage can be determined. ([#74747](https://github.com/kubernetes/kubernetes/pull/74747), [@liggitt](https://github.com/liggitt))

	// Create new bucket encryption subsystem
	globalBucketSSEConfigSys = NewBucketSSEConfigSys()

	// Create new bucket object lock subsystem
	globalBucketObjectLockSys = NewBucketObjectLockSys()

	// Create new bucket quota subsystem
	globalBucketQuotaSys = NewBucketQuotaSys()

	// Create new bucket versioning subsystem
	if globalBucketVersioningSys == nil {
		globalBucketVersioningSys = NewBucketVersioningSys()
	}