* Fix: Use strict timeouts when aborting a download.
 * Fix: Support Shoutcast HTTP responses like `ICY 200 OK`.
 * Fix: Don't unzip if there isn't a response body.
 * Fix: Don't leak gzip streams on redirects.
 * Fix: Don't do DNS lookups on invalid hosts.
 * Fix: Exhaust the underlying stream when reading gzip streams.
 * Fix: Support the `PATCH` method.
 * Fix: Support request bodies on `DELETE` method.

This way the application would be able to know what is its own public URL, if it is using HTTPS, the domain, etc.

This would be useful for example to properly handle redirects.

/// tip

You can learn more about this in the documentation for [Behind a Proxy - Enable Proxy Forwarded Headers](../advanced/behind-a-proxy.md#enable-proxy-forwarded-headers){.internal-link target=_blank}

///

  fun isCanceled(): Boolean

  /**
   * Returns a timeout that spans the entire call: resolving DNS, connecting, writing the request
   * body, server processing, and reading the response body. If the call requires redirects or
   * retries all must complete within one timeout period.
   *
   * Configure the client's default timeout with [OkHttpClient.Builder.callTimeout].
   */
  fun timeout(): Timeout

  /**

 * This filter processes requests with specific encoding requirements and converts
 * character encodings according to configured mapping rules.
 *
 * <p>The filter intercepts requests matching configured path patterns and
 * redirects them with proper character encoding applied to parameters.</p>
 */
public class EncodingFilter implements Filter {
    /** Configuration key for encoding rules mapping */

```console
$ fastapi run --forwarded-allow-ips="*"

<span style="color: green;">INFO</span>:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
```

</div>

### Redirecionamentos com HTTPS { #redirects-with-https }

Por exemplo, suponha que você defina uma *operação de rota* `/items/`:

{* ../../docs_src/behind_a_proxy/tutorial001_01_py39.py hl[6] *}

 *   <li>Starting and finishing threads</li>
 *   <li>Starting and cleaning up crawling</li>
 *   <li>Handling unsupported URLs</li>
 *   <li>Checking last modified dates</li>
 *   <li>Getting content</li>
 *   <li>Handling redirects</li>
 *   <li>Processing responses</li>
 *   <li>Handling exceptions during crawling and child URL processing</li>
 *   <li>Handling cases where no URL is in the queue</li>

Auf diese Weise kann die Anwendung ihre eigene öffentliche URL, ob sie HTTPS verwendet, die Domain, usw. erkennen.

Das ist z. B. nützlich, um <abbr title="Redirect – Umleitung">Redirects</abbr> korrekt zu handhaben.

/// tip | Tipp

     * @param url the URL to process
     * @return a map containing the processed document data
     * @throws CrawlingAccessException if crawling fails or configuration is invalid
     * @throws ChildUrlsException if the URL redirects to another location
     * @throws CrawlerSystemException if data deserialization fails
     */
    public Map<String, Object> processRequest(final CrawlingConfig crawlingConfig, final String crawlingInfoId, final String url) {

 * canceling may break the entire connection.
 */
class RealCall(
  val client: OkHttpClient,
  /** The application's original request unadulterated by redirects or auth headers. */
  val originalRequest: Request,
  val forWebSocket: Boolean,
) : Call,
  Cloneable,
  Lockable {
  private val connectionPool: RealConnectionPool = client.connectionPool.delegate

  @Volatile

crawler.urlFilter.addExclude(".*\\.js$");
crawler.urlFilter.addExclude(".*login.*");
```

## Supported Protocols and Formats

### Protocols
- **HTTP/HTTPS**: Full web crawling support with cookies, authentication, redirects
- **File System**: Local and network file system access
- **FTP**: FTP server crawling with authentication
- **SMB/CIFS**: Windows network shares
- **Storage**: Cloud storage systems (MinIO, S3-compatible)