## Architecture

### Module Structure

```
fess-crawler-parent/
├── fess-crawler/              # Core framework
├── fess-crawler-lasta/        # LastaFlute DI integration
└── fess-crawler-opensearch/   # OpenSearch backend
```

### Key Design Patterns

**Factory**: `CrawlerClientFactory`, `ExtractorFactory` - protocol/format-specific component selection
**Strategy**: `CrawlerClient`, `Extractor`, `Transformer` - pluggable implementations

Here’s some libraries that work nicely with OkHttp.

 * [Chucker](https://github.com/ChuckerTeam/chucker): An in-app HTTP inspector for Android OkHttp clients.
 * [Coil](https://github.com/coil-kt/coil): An image loading library for Android backed by Kotlin Coroutines.
 * [Communicator](https://github.com/Taig/Communicator): An OkHttp wrapper for Scala built with Android in mind.

import java.util.ConcurrentModificationException;
import java.util.Iterator;
import java.util.NoSuchElementException;
import org.jspecify.annotations.Nullable;

/**
 * Basic implementation of {@code Multiset<E>} backed by an instance of {@code
 * ObjectCountHashMap<E>}.
 *
 * <p>For serialization to work, the subclass must specify explicit {@code readObject} and {@code
 * writeObject} methods.
 *
 * @author Kevin Bourrillion
 */

      unsoundlyCovariantArray[size] = null;
    }
    return array;
  }

  /**
   * Implementation of {@link Collection#toArray(Object[])} for collections backed by an object
   * array. the runtime type of the returned array is that of the specified array. If the collection
   * fits in the specified array, it is returned therein. Otherwise, a new array is allocated with

 * crawling and indexing documents, including incremental crawling capabilities, content
 * modification checking, and integration with the Fess search engine backend.
 *
 * <p>Key features include:</p>
 * <ul>
 * <li>Incremental crawling support with last-modified timestamp checking</li>
 * <li>Document expiration handling</li>
 * <li>Child URL extraction and queueing</li>

      unsoundlyCovariantArray[size] = null;
    }
    return array;
  }

  /**
   * Implementation of {@link Collection#toArray(Object[])} for collections backed by an object
   * array. the runtime type of the returned array is that of the specified array. If the collection
   * fits in the specified array, it is returned therein. Otherwise, a new array is allocated with

    }

    /**
     * Returns a new immutable array. The builder can continue to be used after this call, to append
     * more values and build again.
     *
     * <p><b>Performance note:</b> the returned array is backed by the same array as the builder, so
     * no data is copied as part of this step, but this may occupy more memory than strictly
     * necessary. To copy the data to a right-sized backing array, use {@code .build().trimmed()}.

    }

    /**
     * Returns a new immutable array. The builder can continue to be used after this call, to append
     * more values and build again.
     *
     * <p><b>Performance note:</b> the returned array is backed by the same array as the builder, so
     * no data is copied as part of this step, but this may occupy more memory than strictly
     * necessary. To copy the data to a right-sized backing array, use {@code .build().trimmed()}.

import java.util.Iterator;
import java.util.Map;
import java.util.NoSuchElementException;
import java.util.Objects;
import java.util.Set;
import org.jspecify.annotations.Nullable;

/**
 * A {@link BiMap} backed by two hash tables. This implementation allows null keys and values. A
 * {@code HashBiMap} and its inverse are both serializable.
 *
 * <p>This implementation guarantees insertion-based iteration order of its keys.
 *

    the implementation trade-offs. We can't use our HTTP client engine on Kotlin/JS, and we weren't
    prepared to build a TLS API for Kotlin/Native.

    We'd prefer a multiplatform HTTP client API that's backed by OkHttp on Android and JVM, and
    other engines on other platforms. [Ktor] does this pretty well today!

 *  Breaking: Use `kotlin.time.Duration` in APIs like `OkHttpClient.Builder.callTimeout()`. This