* It uses a ConcurrentMap to store the last access time for each host.
 * The delayBeforeProcessing method is overridden to introduce a delay before processing a URL,
 * ensuring that requests to the same host are not made too frequently.
 * The delay is calculated based on the configured delayMillisBeforeProcessing parameter.
 * If the time since the last request to the host is less than the configured delay,

    /**
     * Lock object for synchronizing access to active thread count.
     */
    protected Object activeThreadCountLock = new Object();

    /**
     * Atomic counter for tracking the number of accesses made.
     */
    protected AtomicLong accessCount = new AtomicLong(0);

    /**
     * Current status of the crawler.
     */
    protected volatile CrawlerStatus status = CrawlerStatus.INITIALIZING;

    /**

      </match>
    </magic>
    <glob pattern="*.odt"/>
    <sub-class-of type="application/zip"/>
  </mime-type>

  <!-- can't tell from the root element whether this is fodt, fodp, etc.
     This is a made up general mime to cover the open office flat formats.
      Subtype detection can be done with the document:mime attribute in the root
      element and/or file extensions fodt...-->