* It manages the crawling lifecycle, including initialization, execution coordination,
 * monitoring, and cleanup operations.
 *
 * <p>The crawler can operate in different modes based on command-line options:
 * <ul>
 * <li>Web crawling - crawls web sites and web content</li>
 * <li>File system crawling - crawls file systems and documents</li>

   * semicolon</a>, rather than the ampersand, as a parameter delimiter. Nevertheless, we recommend
   * using the ampersand unless you must interoperate with systems that require semicolons.
   *
   * <p><b>Note:</b> Unlike other escapers, URL escapers produce <a
   * href="https://url.spec.whatwg.org/#percent-encode">uppercase</a> hexadecimal sequences.
   *
   */

Even if they are in different modules (Python files).

///

## Exclude from OpenAPI { #exclude-from-openapi }

To exclude a *path operation* from the generated OpenAPI schema (and thus, from the automatic documentation systems), use the parameter `include_in_schema` and set it to `False`:

{* ../../docs_src/path_operation_advanced_configuration/tutorial003_py39.py hl[6] *}

## Advanced description from docstring { #advanced-description-from-docstring }

```

Das bedeutet, dass das System nach Programmen in den Verzeichnissen suchen sollte:

* `/usr/local/bin`
* `/usr/bin`
* `/bin`
* `/usr/sbin`
* `/sbin`

////

//// tab | Windows

```plaintext
C:\Program Files\Python312\Scripts;C:\Program Files\Python312;C:\Windows\System32
```

Das bedeutet, dass das System nach Programmen in den Verzeichnissen suchen sollte:

```

##### Node (optional)

Optionally you can also collect per node metrics. This needs to be done on a per server instance.
The scrape configurations should use all the servers under `targets` so that graphing systems like
grafana can visualize them for all the nodes

```yaml
scrape_configs:
- job_name: minio-job
  metrics_path: /minio/v2/metrics/node
  scheme: http
  static_configs:

Fess basiert auf [OpenSearch](https://github.com/opensearch-project/OpenSearch), aber es ist kein Wissen oder Erfahrung mit OpenSearch erforderlich. Fess bietet eine einfach zu bedienende Administrations-GUI zur Konfiguration des Systems über Ihren Browser.

#       and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
#       see the in-line comments for details.
#
#       There are tweaks for specific operating systems such as AIX, CygWin,
#       Darwin, MinGW, and NonStop.
#
#   (3) This script is generated from the Groovy template

---

### 🛠 **Seamless IDE & CI Integration**

Gradle is built to work smoothly with a variety of Integrated Development Environments (IDEs) and Continuous Integration (CI) systems, providing extensive support for a streamlined workflow:

// Iterate through crawled data
dataService.iterate(sessionId, accessResult -> {
    System.out.println("URL: " + accessResult.getUrl());
    System.out.println("Status: " + accessResult.getHttpStatusCode());
    System.out.println("Content Type: " + accessResult.getMimeType());
    System.out.println("Content: " + accessResult.getContent());
    System.out.println("---");
});

// Get specific result

    * All this would also work for deeply nested JSON objects.
* Convert from and to JSON automatically.
* Document everything with OpenAPI, that can be used by:
    * Interactive documentation systems.
    * Automatic client code generation systems, for many languages.
* Provide 2 interactive documentation web interfaces directly.

---

We just scratched the surface, but you already get the idea of how it all works.