- **Comprehensive Content Extraction**: Office documents, PDFs, archives, images, audio/video files
- **Multi-Threading**: Configurable thread pools for high-performance crawling
- **Fault Tolerance**: Built-in retry mechanisms and error handling
- **Flexible Configuration**: XML-based dependency injection with LastaFlute DI

     * the same site, so you can use this tag to increase the likelihood that
     * your most important pages are present in a search index.
     *
     * Also, please note that assigning a high priority to all of the URLs on
     * your site is not likely to help you. Since the priority is relative, it
     * is only used to select between URLs on your site.
     */
    private String priority;

    /**

     popular format
   * The main mime type should be the canonical one, use aliases for any
     other widely used forms
   * Where there's a hierarchy in the types, list it via a parent
   * Highly specific magic matches get a high priority
   * General magic matches which could trigger a false-positive need
     a low one
   * The priority for containers normally need to be higher than for
     the things they contain, so they don't accidently get detected