- Kubeadm: `kubeadm upgrade apply` now supports phase sub-command, user can use `kubeadm upgrade apply phase <phase-name>` to execute the specified phase, or use `kubeadm upgrade apply --skip-phases <phase-names>` to skip some phases during cluster upgrade. ([#126032](https://github.com/kubernetes/kubernetes/pull/126032), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle]...

    /** The key of the message: Result */
    public static final String LABELS_SCRIPT_RESULT = "{labels.scriptResult}";

    /** The key of the message: Executor */
    public static final String LABELS_SCRIPT_TYPE = "{labels.scriptType}";

    /** The key of the message: Segmentation */
    public static final String LABELS_SEGMENTATION = "{labels.segmentation}";

- Kubeadm: fix a bug where file copy(backup) could not be executed correctly on Windows platform during upgrade ([#117861](https://github.com/kubernetes/kubernetes/pull/117861), [@SataQiu](https://github.com/SataQiu)) [SIG Cluster Lifecycle]

- For volumes that allow attaches across multiple nodes, attach and detach operations across different nodes are now executed in parallel. ([#88678](https://github.com/kubernetes/kubernetes/pull/88678), [@verult](https://github.com/verult)) [SIG Storage]

	sum := sha256.Sum256(buf)
	req.Header.Set("X-Amz-Content-Sha256", hex.EncodeToString(sum[:]))
	req.Body = io.NopCloser(bytes.NewReader(buf))
	req = signer.SignV4(*req, accessKey, secretKey, "", "")

	// 3.1 Execute the request.
	resp, err := s.TestSuiteCommon.client.Do(req)
	if err != nil {
		c.Fatalf("unexpected request err: %v", err)
	}
	if resp.StatusCode != 200 {
		c.Fatalf("got unexpected response: %#v\n", resp)
	}

//    control_outputs: vector of TF_Operation objects to be marked as control
//      outputs of the function. Operations marked as control outputs are
//      guaranteed to execute.
//    control_output_names: Optional. If not nullptr, vector of strings, one
//      per control output, with their names to be added to the function's
//      OpDef.

- Kubeadm will cleanup the stale data on best effort basis. Stale data will be removed when each reset phase are executed, default etcd data directory will be cleanup when the `remove-etcd-member` phase are executed. ([#110972](https://github.com/kubernetes/kubernetes/pull/110972), [@chendave](https://github.com/chendave)) [SIG Cluster Lifecycle]

- For volumes that allow attaches across multiple nodes, attach and detach operations across different nodes are now executed in parallel. ([#89239](https://github.com/kubernetes/kubernetes/pull/89239), [@verult](https://github.com/verult)) [SIG Apps, Node and Storage]

		t.Fatalf("Unable to create file, %s", err.Error())
	}
	// Parent directory must have write permissions, this is read + execute.
	if err = os.Chmod(pathJoin(path, "no-permissions"), 0o555); err != nil {
		t.Fatalf("Unable to chmod directory, %s", err.Error())
	}
	t.Cleanup(func() {
		os.Chmod(pathJoin(path, "no-permissions"), 0o775)
	})

 * href="https://www.javadoc.io/doc/com.github.ben-manes.caffeine/caffeine/latest/com.github.benmanes.caffeine/com/github/benmanes/caffeine/cache/AsyncCacheLoader.html#asyncLoad(K,java.util.concurrent.Executor)">{@code
 * AsyncCacheLoader.asyncLoad}</a> must return a {@code CompletableFuture}. Users of Guava's {@link
 * com.google.common.util.concurrent.ListenableFuture} can adapt between the two {@code Future}