- Added new metric `job_controller_terminated_pods_tracking_finalizer` which can be used to monitor whether the job controller is removing Pod finalizers from terminated Pods after accounting them in Job status. ([#113176](https://github.com/kubernetes/kubernetes/pull/113176), [@alculquicondor](https://github.com/alculquicondor))

servebolt.cloud

// Service Online LLC : http://drs.ua/
// Submitted by Serhii Bulakh <******@****.***>
biz.ua
co.ua
pp.ua

// Shanghai Accounting Society : https://www.sasf.org.cn
// Submitted by Information Administration <******@****.***>
as.sh.cn

// Sheezy.Art : https://sheezy.art
// Submitted by Nyoom <******@****.***>
sheezy.games

- Fix JobTrackingWithFinalizers that:
  - was declaring a job finished before counting all the created pods in the status
  - was leaving pods with finalizers, blocking pod and job deletions
  

- To avoid soft-deleted pods incorrectly affecting scale up replica count calculations, the HPA controller will stop counting soft-deleted pods for scaling purposes. ([#67067](https://github.com/kubernetes/kubernetes/pull/67067), [@moonek](https://github.com/moonek))

- Fix cgroup handling for systemd with cgroup v2 ([#98365](https://github.com/kubernetes/kubernetes/pull/98365), [@odinuge](https://github.com/odinuge)) [SIG Node]
- Fix counting error in service/nodeport/loadbalancer quota check ([#97451](https://github.com/kubernetes/kubernetes/pull/97451), [@pacoxu](https://github.com/pacoxu)) [SIG API Machinery, Network and Testing]

## Metrics Changes

### Added metrics

- Added metrics `aggregator_openapi_v2_regeneration_count`, `aggregator_openapi_v2_regeneration_gauge` and `apiextension_openapi_v2_regeneration_count` counting the triggering APIService and CRDs and the reason (add, update, delete) when kube-apiserver regenerates the OpenAPI spec. ([#81786](https://github.com/kubernetes/kubernetes/pull/81786), [@sttts](https://github.com/sttts))

- Fixed `JobTrackingWithFinalizers` that:
  - was declaring a job finished before counting all the created pods in the status
  - was leaving pods with finalizers, blocking pod and job deletions

## Changelog since v1.17.16

## Changes by Kind

### Bug or Regression

- Fix counting error in service/nodeport/loadbalancer quota check ([#97830](https://github.com/kubernetes/kubernetes/pull/97830), [@pacoxu](https://github.com/pacoxu)) [SIG API Machinery and Network]

- The kube-proxy `sync_proxy_rules_iptables_total` metric has now reverted back
  to its pre-1.27 behavior of tracking the total number of iptables rules that
  kube-proxy is responsible for, rather than only counting the number of rules
  that it re-synced on the last sync. The new `sync_proxy_rules_iptables_last`

- Fix Azure file share not deleted issue when the namespace is deleted ([#97417](https://github.com/kubernetes/kubernetes/pull/97417), [@andyzhangx](https://github.com/andyzhangx)) [SIG Cloud Provider and Storage]
- Fix counting error in service/nodeport/loadbalancer quota check ([#97829](https://github.com/kubernetes/kubernetes/pull/97829), [@pacoxu](https://github.com/pacoxu)) [SIG API Machinery and Network]