- **CrawlerContext** (`CrawlerContext.java`): Execution context - `sessionId`, `status`, `accessCount`, `numOfThread`, `maxDepth`, `maxAccessCount`
- **CrawlerThread** (`CrawlerThread.java`): Worker thread - Poll URL → Validate → Execute → Process → Queue children

### HTTP Client Architecture

```
SwitchableHttpClient (extends FaultTolerantClient)
    ├── Hc5HttpClient (default) - Apache HttpComponents 5.x

        }

        def stylesheetFile = new File(getTemporaryDir(), "dslHtml.xsl")
        def xslthlConfigFile = getStylesheetHighlightFile().get().asFile.toURI()

        // TODO: Implement this with the worker API
        workerLeaseService.runAsIsolatedTask({
            source.visit { FileVisitDetails fvd ->
                if (fvd.isDirectory()) {
                    return
                }

  }

  /**
   * Creates an empty {@code TreeBasedTable} that is ordered by the specified comparators.
   *
   * @param rowComparator the comparator that orders the row keys
   * @param columnComparator the comparator that orders the column keys
   */
  public static <R, C, V> TreeBasedTable<R, C, V> create(
      Comparator<? super R> rowComparator, Comparator<? super C> columnComparator) {

        failureConditions {
            // JavaExecDebugIntegrationTest.debug session fails without debugger might cause JVM crash
            // Some soak tests produce OOM exceptions
            // There are also random worker crashes for some tests.
            // We have test-retry to handle the crash in tests
            javaCrash = false
        }
    })

fun getTestTaskName(
    testCoverage: TestCoverage,

> NOTE: Server side replication is supported for idempotent versions on directory objects.

### Idempotent versions on delete markers

* der Task
* der Hintergrundtask
* die Taskfunktion

* das Template
* die Template-Engine

* die Typannotation
* der Typhinweis

* der Serverworker
* der Uvicorn-Worker
* der Gunicorn-Worker
* der Workerprozess
* die Workerklasse
* die Workload

* das Deployment
* deployen

* das SDK
* das Software Development Kit

* der `APIRouter`
* die `requirements.txt`

      purge:
          # retainVersions: 0 # (default) delete all versions of the object. This option is the fastest.
          # retainVersions: 5 # keep the latest 5 versions of the object including delete markers.

  notify:
    endpoint: https://notify.endpoint # notification endpoint to receive job completion status
    token: Bearer xxxxx # optional authentication token for the notification endpoint
  
  retry:

## NOTE

- Empty delete markers (such as for objects with no other successor versions) do not transition to the new pool to avoid creating empty metadata on the other pool(s). If you believe transitioning empty delete markers is required, open a GitHub issue.

## TODO

import java.nio.file.StandardOpenOption;

public class PrecommitTask extends DefaultTask {

    @OutputFile
    public File getSuccessMarker() {
        return new File(getProjectLayout().getBuildDirectory().getAsFile().get(), "markers/" + this.getName());
    }

    @TaskAction
    public void writeMarker() throws IOException {
        Files.write(getSuccessMarker().toPath(), new byte[] {}, StandardOpenOption.CREATE);
    }

    @Inject

comment          (sentence)  optionally add a comment to this setting
```

Configure MinIO to send audit logs to locally running Kafka brokers

```
mc admin config set myminio/ audit_kafka:target1 brokers=localhost:29092 topic=auditlog
mc admin service restart myminio/
```

On another terminal assuming you have `kafkacat` installed

```