```

### Helpers

**RobotsTxtHelper**: RFC 9309 parsing, user-agent matching, crawl-delay, sitemaps
**SitemapsHelper**: Sitemap XML parsing, index handling
**MimeTypeHelper**: MIME detection via Tika
**EncodingHelper**: Charset detection with BOM
**UrlConvertHelper**: URL normalization

---

## Development Workflow

### Build Commands

```bash
mvn clean install              # Build all

 * This class implements AutoCloseable to properly handle resource cleanup.
 *
 * <p>The factory maintains a map of regular expression patterns to crawler clients,
 * allowing for URL-based client selection. Clients can be added with specific patterns
 * and optionally at specific positions in the processing order.</p>
 *
 * <p>This factory is typically initialized through dependency injection and can be

 * <li>Document expiration handling</li>
 * <li>Child URL extraction and queueing</li>
 * <li>Integration with Fess configuration and permission systems</li>
 * <li>Client selection based on URL patterns</li>
 * </ul>
 *
 * @see CrawlerThread
 * @see org.codelibs.fess.crawler.client.CrawlerClient
 */
public class FessCrawlerThread extends CrawlerThread {

    /**

fun Project.isNativeProject() = name.contains("native")

fun Project.isKotlinDslToolingBuilders() = name.contains("kotlin-dsl-tooling-builders")

/**
 * Whether the project supports running with predictive test selection.
 *
 * Our performance tests don't work with PTS, yet.
 * Smoke and soak tests are hard to grasp for PTS, that is why we run them without.

     *
     * This method prioritizes explicit language parameters over browser locale settings
     * and handles special cases like "all languages" selection.
     *
     * @param request The HTTP servlet request
     * @param params The search request parameters
     * @return Array of normalized language codes
     */

- Added DRA support for a "one-of" prioritized list of selection criteria to satisfy a device request in a resource claim. ([#128586](https://github.com/kubernetes/kubernetes/pull/128586), [@mortent](https://github.com/mortent)) [SIG API Machinery, Apps, Etcd, Node, Scheduling and Testing]

        final UrlQueue<?> urlQueue = new UrlQueueImpl<>();
        urlQueue.setUrl("http://example.com/");
        urlQueue.setDepth(1);

        when(urlFilter.match(anyString())).thenReturn(true);

        // Use reflection to access protected method
        final java.lang.reflect.Method method = CrawlerThread.class.getDeclaredMethod("isValid", UrlQueue.class);
        method.setAccessible(true);

     * This method first refreshes the index, then performs the deletion, and logs the result.
     *
     * @param searchEngineClient the search engine client to use for deletion
     * @param index the index name to delete documents from
     * @param queryBuilder the query to match documents for deletion
     * @return the number of documents that were deleted
     */

      break;
    }
    expectUnchanged();
  }

  @J2ktIncompatible
  @GwtIncompatible // reflection
  public static Method getContainsEntryWithIncomparableKeyMethod() {
    return getMethod(MapEntrySetTester.class, "testContainsEntryWithIncomparableKey");
  }

  @J2ktIncompatible
  @GwtIncompatible // reflection
  public static Method getContainsEntryWithIncomparableValueMethod() {

      break;
    }
    expectUnchanged();
  }

  @J2ktIncompatible
  @GwtIncompatible // reflection
  public static Method getContainsEntryWithIncomparableKeyMethod() {
    return getMethod(MapEntrySetTester.class, "testContainsEntryWithIncomparableKey");
  }

  @J2ktIncompatible
  @GwtIncompatible // reflection
  public static Method getContainsEntryWithIncomparableValueMethod() {