"node(%s): Read/Write/Delete successful, bringing drive %s online", globalLocalNodeName, p.storage.String())
			p.health.status.Store(diskHealthOK)
			p.health.waiting.Add(-1)
			return
		}
	}
}

// monitorDiskStatus should be called once when a drive has been marked offline.
// Once the disk has been deemed ok, it will return to online status.
func (p *xlStorageDiskIDCheck) monitorDiskWritable(ctx context.Context) {

          "hide": false,
          "instant": true,
          "legendFormat": ".",
          "range": false,
          "refId": "B"
        }
      ],
      "title": "Total Online/Offline Drives",
      "type": "gauge"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "${DS_PROMETHEUS}"
      },
      "description": "",
      "fieldConfig": {

	// Return error when quorum unformatted disks - indicating we are
	// waiting for first server to be online.
	unformattedDisks := quorumUnformattedDisks(sErrs)
	if unformattedDisks && !firstDisk {
		return nil, errNotFirstDisk
	}

	// Return error when quorum unformatted disks but waiting for rest
	// of the servers to be online.
	if unformattedDisks && firstDisk {
		return nil, errFirstDiskWait
	}

		return
	}
	if dobj.VersionID != "" && rinfo.VersionPurgeStatus == replication.VersionPurgeComplete {
		return
	}
	if globalBucketTargetSys.isOffline(tgt.EndpointURL()) {
		replLogOnceIf(ctx, fmt.Errorf("remote target is offline for bucket:%s arn:%s", dobj.Bucket, tgt.ARN), "replication-target-offline-delete-"+tgt.ARN)
		sendEvent(eventArgs{
			BucketName: dobj.Bucket,
			Object: ObjectInfo{
				Bucket:       dobj.Bucket,

			// Since for majority of the cases erasure.Index matches with erasure.Distribution we can
			// consider the offline disks as consistent.
			continue
		}
		if !meta.Deleted {
			if len(meta.Erasure.Distribution) != len(onlineDisks) {
				// Erasure distribution seems to have lesser
				// number of items than number of online disks.
				inconsistent++
				continue
			}
			if meta.Erasure.Distribution[i] != meta.Erasure.Index {

			if retries >= 20 {
				logger.Info(fmt.Sprintf("Waiting for at least %d remote servers with valid configuration to be online", len(clnts)/2))
				if len(offlineEndpoints) > 0 {
					logger.Info(fmt.Sprintf("Following servers are currently offline or unreachable %s", offlineEndpoints))
				}
				if len(incorrectConfigs) > 0 {

import (
	"bytes"
	"crypto/rand"
	"io"
	"os"
	"testing"
)

var erasureHealTests = []struct {
	dataBlocks, disks int

	// number of offline disks is also number of staleDisks for
	// erasure reconstruction in this test
	offDisks int

	// bad disks are online disks which return errors
	badDisks, badStaleDisks int

	blocksize, size int64
	algorithm       BitrotAlgorithm
	shouldFail      bool
}{

	}
	idx := xxhash.Sum64String(s) % uint64(len(peerClients))
	return peerClients[idx]
}

// GetPeerOnlineCount gets the count of online and offline nodes.
func (sys *NotificationSys) GetPeerOnlineCount() (nodesOnline, nodesOffline int) {
	nodesOnline = 1 // Self is always online.
	nodesOffline = 0
	nodesOnlineIndex := make([]bool, len(sys.peerClients))
	var wg sync.WaitGroup
	for idx, client := range sys.peerClients {

   *       {@code consistentHash} will handle it. If, however, you are dividing traffic among
   *       servers {@code alpha}, {@code bravo}, and {@code charlie} and you occasionally need to
   *       take each of the servers offline, {@code consistentHash} will be a poor fit: It provides
   *       no way for you to specify which of the three buckets is disappearing. Thus, if your

			_, ok := disks[d.Endpoint]
			if !ok {
				continue
			}
		}

		if d.State != madmin.DriveStateOk && d.State != madmin.DriveStateUnformatted {
			metrics[d.Endpoint] = madmin.DiskMetric{NDisks: 1, Offline: 1}
			continue
		}

		var dm madmin.DiskMetric
		dm.NDisks = 1
		if d.Healing {
			dm.Healing++
		}
		if d.Metrics != nil {
			dm.LifeTimeOps = make(map[string]uint64, len(d.Metrics.APICalls))