This way the application would be able to know what is its own public URL, if it is using HTTPS, the domain, etc.

This would be useful for example to properly handle redirects.

/// tip

You can learn more about this in the documentation for [Behind a Proxy - Enable Proxy Forwarded Headers](../advanced/behind-a-proxy.md#enable-proxy-forwarded-headers){.internal-link target=_blank}

///

    - skip_users: ../en/data/skip_users.yml
    - members: ../en/data/members.yml
    - sponsors_badge: ../en/data/sponsors_badge.yml
    - sponsors: ../en/data/sponsors.yml
    - topic_repos: ../en/data/topic_repos.yml
  redirects:
    redirect_maps:
      deployment/deta.md: deployment/cloud.md
      advanced/graphql.md: how-to/graphql.md
      advanced/custom-request-and-route.md: how-to/custom-request-and-route.md

 * This filter processes requests with specific encoding requirements and converts
 * character encodings according to configured mapping rules.
 *
 * <p>The filter intercepts requests matching configured path patterns and
 * redirects them with proper character encoding applied to parameters.</p>
 */
public class EncodingFilter implements Filter {
    /** Configuration key for encoding rules mapping */

 *   <li>Starting and finishing threads</li>
 *   <li>Starting and cleaning up crawling</li>
 *   <li>Handling unsupported URLs</li>
 *   <li>Checking last modified dates</li>
 *   <li>Getting content</li>
 *   <li>Handling redirects</li>
 *   <li>Processing responses</li>
 *   <li>Handling exceptions during crawling and child URL processing</li>
 *   <li>Handling cases where no URL is in the queue</li>

crawler.urlFilter.addExclude(".*\\.js$");
crawler.urlFilter.addExclude(".*login.*");
```

## Supported Protocols and Formats

### Protocols
- **HTTP/HTTPS**: Full web crawling support with cookies, authentication, redirects
- **File System**: Local and network file system access
- **FTP**: FTP server crawling with authentication
- **SMB/CIFS**: Windows network shares
- **Storage**: Cloud storage systems (MinIO, S3-compatible)

        });
        return redirectWith(getClass(), moreUrl("list/1").params("dictId", form.dictId));
    }

    /**
     * Delete a stemmer override item from the dictionary.
     * Removes the specified item and redirects to the list page.
     *
     * @param form The edit form containing the item ID to delete
     * @return HTML response redirecting to the list page on success
     */
    @Execute
    @Secured({ ROLE })

     * @param url the URL to process
     * @return a map containing the processed document data
     * @throws CrawlingAccessException if crawling fails or configuration is invalid
     * @throws ChildUrlsException if the URL redirects to another location
     * @throws CrawlerSystemException if data deserialization fails
     */
    public Map<String, Object> processRequest(final CrawlingConfig crawlingConfig, final String crawlingInfoId, final String url) {

 * canceling may break the entire connection.
 */
class RealCall(
  val client: OkHttpClient,
  /** The application's original request unadulterated by redirects or auth headers. */
  val originalRequest: Request,
  val forWebSocket: Boolean,
) : Call,
  Cloneable,
  Lockable {
  private val connectionPool: RealConnectionPool = client.connectionPool.delegate

    //                                                                           =========

    /**
     * Redirects to the dictionary index page.
     *
     * @return HTML response redirecting to the dictionary index
     */
    protected HtmlResponse asDictIndexHtml() {
        return redirect(AdminDictAction.class);
    }

    private HtmlResponse asListHtml(final String dictId) {

    /**
     * Constructs a new HtmlTransformer.
     */
    public HtmlTransformer() {
        // Default constructor
    }

    /** Header name for location redirects. */
    protected static final String LOCATION_HEADER = "Location";

    /** The crawler container for dependency injection. */
    @Resource
    protected CrawlerContainer crawlerContainer;