There may be many routes for a single address. For example, a webserver that is hosted in multiple datacenters may yield multiple IP addresses in its DNS response.

In limited situations OkHttp will retry a route if connecting fails:

   * find an exchange to carry the request.
   *
   * Note that an exchange will not be needed if the request is satisfied by the cache.
   *
   * @param newRoutePlanner true if this is not a retry and new routing can be performed.
   */
  fun enterNetworkInterceptorExchange(
    request: Request,
    newRoutePlanner: Boolean,
    chain: RealInterceptorChain,
  ) {

		return Endpoints{}
	}
	return disksToHeal
}

var newDiskHealingTimeout = newDynamicTimeout(30*time.Second, 10*time.Second)

var errRetryHealing = errors.New("some items failed to heal, we will retry healing this drive again")

func healFreshDisk(ctx context.Context, z *erasureServerPools, endpoint Endpoint) error {
	poolIdx, setIdx := endpoint.PoolIdx, endpoint.SetIdx
	disk := getStorageViaEndpoint(endpoint)

	switch err.HTTPStatusCode {
	case http.StatusServiceUnavailable, http.StatusTooManyRequests:
		// Set retry-after header to indicate user-agents to retry request after 60 seconds.
		// https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After
		w.Header().Set(xhttp.RetryAfter, "60")
	}

	switch err.Code {
	case "InvalidRegion":

                if (pomFile.canRead()) {
                    return length != pomFile.length() || timestamp != pomFile.lastModified();
                } else {
                    // if the POM didn't exist, retry if any repo is configured to always update
                    boolean snapshot = pomArtifact.isSnapshot();
                    for (ArtifactRepository repository : remoteRepositories) {

the number of drives to 16 for erasure set because, erasure code shards more than 16 can become chatty and do not have any performance advantages. Additionally since 16 drive erasure set gives you tolerance of 8 drives per object by default which is plenty in any practical scenario.

- Choice of erasure set size is automatic based on the number of drives available, let's say for example if there are 32 servers and 32 drives which is a total of 1024 drives. In this scenario 16 becomes the erasure...

- `DeferredFileOutputStream` for large responses (temp files for >1MB)
- Connection pooling with limits
- Background temp file deletion via `FileUtil.deleteInBackground()`

**Fault Tolerance**:
- `FaultTolerantClient` wrapper (retry, circuit breaker)
- Graceful degradation (e.g., robots.txt parsing continues on errors)

---

## Key Components

### Crawler (`Crawler.java`)

Main orchestrator for crawling operations.

    /** Index name. */
    protected final String indexName;
    /** Supported fields. */
    protected final String[] supportedFields;

    /** Scroll size. */
    protected int scrollSize = 1;
    /** Maximum retry count. */
    protected int maxRetryCount = 5;
    /** Limit of document size. */
    protected long limitOfDocumentSize = 50000;
    /** Query builder. */

                if (StringUtil.isBlank(content)) {
                    if (resourceName != null) {
                        if (logger.isDebugEnabled()) {
                            logger.debug("retry without a resource name: {}", resourceName);
                        }
                        final Metadata metadata2 = createMetadata(null, contentType, contentEncoding, password);

         * Because the step is not guaranteed to run (e.g. build timeout), we need `KILL_LEAKED_PROCESSES_FROM_PREVIOUS_BUILDS` mode.
         */
        KILL_PROCESSES_STARTED_BY_GRADLE,
        /**
         * Run when we want to retry the build. Kill all Gradle processes, regardless of they're global or local.
         */
        KILL_ALL_GRADLE_PROCESSES
    }

    private static final Pattern UNIX_PID_PATTERN = Pattern.compile("([0-9]+)");