/// tip | Tipp

Eine Datei, die mit einem Punkt (`.`) beginnt, ist eine versteckte Datei in Unix-ähnlichen Systemen wie Linux und macOS.

Aber eine dotenv-Datei muss nicht unbedingt genau diesen Dateinamen haben.

///

        <include>mvnenc</include>
        <include>mvnsh</include>
        <include>mvnup</include>
        <include>mvnDebug</include>
        <include>mvnencDebug</include>
        <!-- This is so that CI systems can periodically run the profiler -->
        <include>mvnyjp</include>
      </includes>
      <lineEnding>unix</lineEnding>
      <fileMode>0755</fileMode>
    </fileSet>
    <fileSet>

import java.io.OutputStream;
import java.util.List;
import java.util.Map;

/**
 * Interface for cloud storage operations.
 * Implementations provide access to S3-compatible storage, GCS, or other cloud storage systems.
 */
public interface StorageClient extends AutoCloseable {

    /**
     * Uploads an object to storage.
     *
     * @param objectName the name/path for the object

These environment variables can only handle text strings, as they are external to Python and have to be compatible with other programs and the rest of the system (and even with different operating systems, as Linux, Windows, macOS).

That means that any value read in Python from an environment variable will be a `str`, and any conversion to a different type or any validation has to be done in code.

- **Testing**: JUnit 4, UTFlute, Mockito 5.7.0
- **Storage**: In-memory (default), OpenSearch (optional)

### Protocols

- **HTTP/HTTPS**: Full crawling, cookies, auth, robots.txt
- **File**: Local/network file systems
- **FTP**: With authentication
- **SMB/CIFS**: Windows shares (SMB1/SMB2+)
- **Storage**: MinIO/S3-compatible

### Content Formats

import software.amazon.awssdk.services.s3.model.Tag;

/**
 * A crawler client implementation for accessing and retrieving content from S3-compatible storage systems
 * using AWS SDK v2. This client supports operations on Amazon S3 and S3-compatible storage systems.
 *
 * <p>This client requires the following initialization parameters:
 * <ul>
 *   <li>endpoint - The URL of the S3-compatible server</li>

import software.amazon.awssdk.services.s3.model.Tagging;

/**
 * S3-compatible storage client implementation using AWS SDK v2.
 * Supports Amazon S3, MinIO, and other S3-compatible storage systems.
 */
public class S3StorageClient implements StorageClient {

    private static final Logger logger = LogManager.getLogger(S3StorageClient.class);

    private final S3Client s3Client;

Even if they are in different modules (Python files).

///

## Exclude from OpenAPI { #exclude-from-openapi }

To exclude a *path operation* from the generated OpenAPI schema (and thus, from the automatic documentation systems), use the parameter `include_in_schema` and set it to `False`:

{* ../../docs_src/path_operation_advanced_configuration/tutorial003_py39.py hl[6] *}

## Advanced description from docstring { #advanced-description-from-docstring }

 * It manages the crawling lifecycle, including initialization, execution coordination,
 * monitoring, and cleanup operations.
 *
 * <p>The crawler can operate in different modes based on command-line options:
 * <ul>
 * <li>Web crawling - crawls web sites and web content</li>
 * <li>File system crawling - crawls file systems and documents</li>

    * All this would also work for deeply nested JSON objects.
* Convert from and to JSON automatically.
* Document everything with OpenAPI, that can be used by:
    * Interactive documentation systems.
    * Automatic client code generation systems, for many languages.
* Provide 2 interactive documentation web interfaces directly.

---

We just scratched the surface, but you already get the idea of how it all works.