* <ul>
     *   <li>&amp; with &amp;amp;</li>
     *   <li>&lt; with &amp;lt;</li>
     *   <li>&gt; with &amp;gt;</li>
     *   <li>" with &amp;quot;</li>
     *   <li>' with &amp;apos;</li>
     * </ul>
     * Additionally, it strips invalid XML characters from the input string.
     *
     * @param value the input string to be escaped
     * @return the escaped XML string with invalid characters removed
     */

# Run tests for specific module
mvn test -pl fess-crawler
```

## Examples

### Web Crawling with Custom Rules

```java
// Create crawler with custom configuration
StandardCrawlerContainer container = new StandardCrawlerContainer();

// Configure HTTP client with custom settings
container.singleton("httpClient", HcHttpClient.class, client -> {
    client.setUserAgent("MyBot/1.0");

 * It is used to indicate exceptions related to MIME type handling during the crawling process.
 * This exception can be thrown with a message, a cause, or both.
 */
public class MimeTypeException extends CrawlerSystemException {

    private static final long serialVersionUID = 1L;

    /**
     * Creates a new MimeTypeException with the specified detail message and cause.
     *

/*
 * Copyright 2012-2025 CodeLibs Project and the Others.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,

- **Popular Words Analytics**: Track and analyze frequently searched terms
- **Flexible Text Processing**: Configurable converters and normalizers for text transformation
- **OpenSearch Integration**: Seamless integration with OpenSearch/Elasticsearch clusters
- **Asynchronous Operations**: Non-blocking suggestion requests with callback support

 * and registration with the ExtractorFactory.
 *
 * <p>
 * This class handles the retrieval of essential crawler components like
 * {@link MimeTypeHelper} and {@link ExtractorFactory} from the
 * {@link CrawlerContainer}. It also provides a convenient method for
 * registering the extractor with the {@link ExtractorFactory}.
 * </p>
 *
 * <p>

    /**
     * Constructs a new CrawlingAccessException with the specified detail message and cause.
     *
     * @param message the detail message
     * @param cause the cause
     */
    public CrawlingAccessException(final String message, final Throwable cause) {
        super(message, cause);
    }

    /**
     * Constructs a new CrawlingAccessException with the specified detail message.
     *

   END OF TERMS AND CONDITIONS

   APPENDIX: How to apply the Apache License to your work.

      To apply the Apache License to your work, attach the following
      boilerplate notice, with the fields enclosed by brackets "[]"
      replaced with your own identifying information. (Don't include
      the brackets!)  The text should be enclosed in the appropriate
      comment syntax for the file format. We also recommend that a

    /** Delay in milliseconds for waiting for new URLs */
    protected long delayMillisForWaitingNewUrl = 1000L;

    /**
     * Default constructor with default delay values.
     */
    public DefaultIntervalController() {
    }

    /**
     * Constructor with configurable delay parameters.
     *
     * @param params map containing delay parameters
     */

/*
 * Copyright 2012-2025 CodeLibs Project and the Others.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,