- **Comprehensive Content Extraction**: Office documents, PDFs, archives, images, audio/video files
- **Multi-Threading**: Configurable thread pools for high-performance crawling
- **Fault Tolerance**: Built-in retry mechanisms and error handling
- **Flexible Configuration**: XML-based dependency injection with LastaFlute DI

 * <p>
 * This class provides methods to configure the XPath expressions, parser features, and properties.
 * It also includes caching mechanism for XPathAPI instances to improve performance.
 * </p>
 * <p>
 * The extracted text is obtained from the nodes selected by the {@code targetNodePath} XPath expression.

- Confirm documents are indexed: `suggester.refresh()`
- Check query formatting and filters
- Verify analyzer configurations

**Performance Issues**
- Increase thread pool size in SuggesterBuilder
- Optimize batch sizes for indexing operations
- Review OpenSearch cluster performance

**Memory Usage**
- Configure appropriate JVM heap settings
- Monitor index sizes and optimize mappings

        for (int i = 0; i < 1000; i++) {
            urlFilter.addInclude("https://site" + i + ".com/.*");
            urlFilter.addExclude(".*\\.exclude" + i + "$");
        }

        // Test matching performance
        assertTrue(urlFilter.match("https://site500.com/page"));
        assertFalse(urlFilter.match("https://site500.com/file.exclude500"));
        assertFalse(urlFilter.match("https://unknown.com/page"));
    }

        assertEquals("In progress", statusMap.get(CrawlerStatus.RUNNING));
        assertEquals("Completed", statusMap.get(CrawlerStatus.DONE));

        // Test with EnumMap for better performance
        java.util.EnumMap<CrawlerStatus, String> enumMap = new java.util.EnumMap<>(CrawlerStatus.class);
        enumMap.put(CrawlerStatus.INITIALIZING, "Init");
        enumMap.put(CrawlerStatus.RUNNING, "Run");

 * The extracted data is then formatted into an XML structure suitable for indexing.
 * </p>
 *
 * <p>
 * The class uses a cache for XPathAPI objects to improve performance. The cache duration is configurable.
 * </p>
 *
 * <p>
 * The transform method takes a ResponseData object containing the XML content and returns a ResultData object with the extracted and formatted data.
 * </p>
 *

 * group, and symbolic links.
 *
 * <p>
 * The class uses Apache Commons Net library for FTP communication. It maintains a queue of FTPClient
 * instances to improve performance by reusing connections.
 * </p>
 *
 * <p>
 * The client can be configured with FTP-specific settings via init parameters, such as:
 * </p>
 * <ul>