* This class provides functionality to filter URLs based on include and exclude patterns.
 * It uses a {@link UrlFilterService} to manage the URL filtering rules.
 * The class supports caching of include and exclude patterns for scenarios where a session ID is not available.
 * It also provides methods to initialize the filter with a session ID, clear the filter,

 * exceptions during retries.</p>
 *
 * <p>Key features:</p>
 * <ul>
 *   <li>Configurable maximum retry attempts</li>
 *   <li>Adjustable interval between retries</li>
 *   <li>Exception tracking and aggregation</li>
 *   <li>Request lifecycle monitoring through listener</li>
 * </ul>
 *
 * <p>By default, it will:</p>
 * <ul>
 *   <li>Retry up to 5 times</li>
 *   <li>Wait 500ms between retries</li>

    /**
     * Lock object for synchronizing access to active thread count.
     */
    protected Object activeThreadCountLock = new Object();

    /**
     * Atomic counter for tracking the number of accesses made.
     */
    protected AtomicLong accessCount = new AtomicLong(0);

    /**
     * Current status of the crawler.
     */

     * archived URLs.
     *
     * Please note that the value of this tag is considered a hint and not a
     * command. Even though search engine crawlers may consider this information
     * when making decisions, they may crawl pages marked "hourly" less
     * frequently than that, and they may crawl pages marked "yearly" more
     * frequently than that. Crawlers may periodically crawl pages marked