drive_workers  (int)       the number of workers per drive to heal a new disk replacement.
```

Example: The following settings will increase the heal operation speed by allowing healing operation to run without delay up to `100` concurrent requests, and the maximum delay between each heal operation is set to `300ms`.

```sh
~ mc admin config set alias/ heal max_sleep=300ms max_io=100
```

whereas in MinIO erasure code you can lose as many as half of drives and still the data remains safe. Further, MinIO's erasure code is at the object level and can heal one object at a time. For RAID, healing can be done only at the volume level which translates into high downtime. As MinIO encodes each object individually, it can heal objects incrementally. Storage servers once deployed should not require drive replacement or healing for the lifetime of the server. MinIO's erasure coded backend is...

#### Policy management permissions

- admin:CreatePolicy
- admin:DeletePolicy
- admin:GetPolicy
- admin:AttachUserOrGroupPolicy
- admin:ListUserPolicies

#### Heal management permissions

- admin:Heal

#### Service account management permissions

- admin:CreateServiceAccount
- admin:UpdateServiceAccount
- admin:RemoveServiceAccount
- admin:ListServiceAccounts

| `minio_heal_objects_errors_total`            | Objects for which healing failed in current self healing run.    |
| `minio_heal_objects_heal_total`              | Objects healed in current self healing run.                      |
| `minio_heal_objects_total`                   | Objects scanned in current self healing run.                     |

```
commit 886262e58af77ebc7c836ef587c08544e9a0c271
Author: Harshavardhana <******@****.***>
Date:   Wed Nov 17 15:49:12 2021 -0800

    heal legacy objects when versioning is enabled after upgrade (#13671)
```

	@(env bash $(PWD)/buildscripts/verify-healing.sh)
	@(env bash $(PWD)/buildscripts/verify-healing-empty-erasure-set.sh)
	@(env bash $(PWD)/buildscripts/heal-inconsistent-versions.sh)

verify-healing-with-root-disks: ## verify healing root disks
	@echo "Verify healing with root drives"

            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header X-NginX-Proxy true;

            # This is necessary to pass the correct IP to be hashed
            real_ip_header X-Real-IP;

            proxy_connect_timeout 300;
            

            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header X-NginX-Proxy true;

            # This is necessary to pass the correct IP to be hashed
            real_ip_header X-Real-IP;

            proxy_connect_timeout 300;
            

            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header X-NginX-Proxy true;

            # This is necessary to pass the correct IP to be hashed
            real_ip_header X-Real-IP;

            proxy_connect_timeout 300;
            

import org.junit.jupiter.api.AfterEach
import org.junit.jupiter.api.Tag
import org.junit.jupiter.api.Test

/**
 * Integration test to confirm that [TaskRunner] works with a real backend. Business logic is all
 * exercised by [TaskRunnerTest].
 *
 * This test is doing real sleeping with tolerances of 250 ms. Hopefully that's enough for even the
 * busiest of CI servers.
 */
@Tag("Slowish")
class TaskRunnerRealBackendTest {