}

    /**
     * Returns the include filtering pattern.
     * @return The include filtering pattern.
     */
    public String getIncludeFilteringPattern() {
        return includeFilteringPattern;
    }

    /**
     * Sets the include filtering pattern.
     * @param includeFilteringPattern The include filtering pattern.
     */

- **Extensible Architecture**: Plugin system for custom extractors, transformers, and clients
- **Rate Limiting**: Politeness policies and interval controllers
- **URL Filtering**: Regex-based inclusion/exclusion patterns
- **Data Persistence**: Multiple backend options including OpenSearch integration

## Technology Stack

- **Java**: 21+ (requires Java 21 or higher)

import org.codelibs.fess.crawler.service.UrlFilterService;

import jakarta.annotation.Resource;

/**
 * Implementation of the {@link UrlFilterService} interface.
 * This class provides methods for managing URL filtering rules,
 * including adding include and exclude URL patterns, deleting patterns,
 * and retrieving lists of compiled URL patterns. It utilizes a
 * {@link MemoryDataHelper} to store and manage the URL patterns in memory.
 *

    /** Map of session IDs to include URL patterns for filtering URLs. */
    protected volatile Map<String, List<Pattern>> includeUrlPatternMap = new HashMap<>();

    /** Map of session IDs to exclude URL patterns for filtering URLs. */
    protected volatile Map<String, List<Pattern>> excludeUrlPatternMap = new HashMap<>();

    /**

import jakarta.annotation.Resource;

/**
 * The Crawler class is the main class for web crawling. It manages the crawling process,
 * including adding URLs to the queue, filtering URLs, managing crawler threads,
 * and handling the overall crawling lifecycle.
 *
 * <p>It implements the Runnable interface to be executed in a separate thread,