- Background temp file deletion via `FileUtil.deleteInBackground()`

**Fault Tolerance**:
- `FaultTolerantClient` wrapper (retry, circuit breaker)
- Graceful degradation (e.g., robots.txt parsing continues on errors)

---

## Key Components

### Crawler (`Crawler.java`)

Main orchestrator for crawling operations.

**Key Methods**:
```java

 *       Future} is "completed" even if it is cancelled while its underlying work continues on a
 *       thread, an RPC, etc. The {@code Future} is also "completed" if it fails "early" -- for
 *       example, if the deadline expires on a {@code Future} returned from {@link
 *       Futures#withTimeout} while the {@code Future} it wraps continues its underlying work. So

                if (line.length() == 0 || line.charAt(0) == '#') {
                    if (updater != null) {
                        updater.write(line);
                    }
                    continue; // ignore empty lines and comments
                }

                final String inputStrings = line;
                final String input = unescape(inputStrings);

                if (input.length() > 0) {

      while (true) {
        int q = c[j] + o[j];
        if (q < 0) {
          switchDirection();
          continue;
        }
        if (q == j + 1) {
          if (j == 0) {
            break;
          }
          s++;
          switchDirection();
          continue;
        }

        Collections.swap(list, j - c[j] + s, j - q + s);
        c[j] = q;
        break;
      }

                    if (logger.isDebugEnabled()) {
                        logger.debug("Configuration file (fess_ds++.xml) not found: path={}", jarFile.getAbsolutePath());
                    }
                    continue;
                }
                try (InputStream is = Files.newInputStream(xmlPath)) {
                    final DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();

                        this.authMethod = "NTLM";
                        break;
                    }
                    if (currentAuthMethod.indexOf(' ') != 4) {
                        continue;
                    }
                    this.authMethod = "NTLM";
                    authorization = currentAuthMethod.substring(5).trim();
                    break;
                }

  }

  /**
   * Drains the queue as {@linkplain #drain(BlockingQueue, Collection, int, Duration)}, but with a
   * different behavior in case it is interrupted while waiting. In that case, the operation will
   * continue as usual, and in the end the thread's interruption status will be set (no {@code
   * InterruptedException} is thrown).
   *
   * @param q the blocking queue to be drained

            final Object idValue = inputDoc.get(fessConfig.getIndexFieldId());
            if (idValue == null) {
                continue;
            }

            final Object configIdValue = inputDoc.get(fessConfig.getIndexFieldConfigId());
            if (configIdValue == null) {
                continue;
            }

            final QueryBuilder queryBuilder = QueryBuilders.boolQuery()

                    if (StringUtil.isNotBlank(responseData.getRedirectLocation())) {
                        u = responseData.getRedirectLocation();
                        continue;
                    }
                    if (StringUtil.isBlank(responseData.getUrl())) {
                        throw new ThumbnailGenerationException(

## Hospedagem Própria de JavaScript e CSS para a documentação { #self-hosting-javascript-and-css-for-docs }

Hospedar o JavaScript e o CSS pode ser útil se, por exemplo, você precisa que seu aplicativo continue funcionando mesmo offline, sem acesso aberto à Internet, ou em uma rede local.

Aqui você verá como providenciar esses arquivos você mesmo, na mesma aplicação FastAPI, e configurar a documentação para usá-los.