title: Auth, user management and more for your B2B product
    img: https://fastapi.tiangolo.com/img/sponsors/propelauth.png
  - url: https://zuplo.link/fastapi-gh
    title: 'Zuplo: Deploy, Secure, Document, and Monetize your FastAPI'
    img: https://fastapi.tiangolo.com/img/sponsors/zuplo.png
  - url: https://liblab.com?utm_source=fastapi
    title: liblab - Generate SDKs from FastAPI

    }

    // ===================================================================================
    //                                                                            Document
    //                                                                            ========
    /**
     * {@inheritDoc} <br>
     * Application Origin Methods:
     * <pre>

     * The script language.
     */
    protected String scriptLang = "painless";

    /**
     * The script code.
     */
    protected String scriptCode = null;

    /**
     * A function to find document IDs.
     */
    protected Function<Map<String, Object>, String[]> idFinder = params -> {
        final FessConfig fessConfig = ComponentUtil.getFessConfig();

 *
 * <p>Key features:</p>
 * <ul>
 *   <li>Concurrent crawler process management with configurable limits</li>
 *   <li>Selective crawling based on configuration IDs</li>
 *   <li>Document expiration handling</li>
 *   <li>Hot thread monitoring for performance analysis</li>
 *   <li>Process isolation with separate JVM</li>
 * </ul>
 */
public class CrawlJob extends ExecJob {

 * context (such as an XML document). Typically (but not always), the inverse process of
 * "unescaping" the text is performed automatically by the relevant parser.
 *
 * <p>For example, an XML escaper would convert the literal string {@code "Foo<Bar>"} into {@code
 * "Foo&lt;Bar&gt;"} to prevent {@code "<Bar>"} from being confused with an XML tag. When the

 * context (such as an XML document). Typically (but not always), the inverse process of
 * "unescaping" the text is performed automatically by the relevant parser.
 *
 * <p>For example, an XML escaper would convert the literal string {@code "Foo<Bar>"} into {@code
 * "Foo&lt;Bar&gt;"} to prevent {@code "<Bar>"} from being confused with an XML tag. When the

   *
   * <p>Some algorithms that operate on a {@code PredecessorsFunction} may produce undesired results
   * if the returned {@link Iterable} contains duplicate elements. Implementations of such
   * algorithms should document their behavior in the presence of duplicates.
   *
   * <p>The elements of the returned {@code Iterable} must each be:
   *
   * <ul>
   *   <li>Non-null

   *
   * <p>Some algorithms that operate on a {@code SuccessorsFunction} may produce undesired results
   * if the returned {@link Iterable} contains duplicate elements. Implementations of such
   * algorithms should document their behavior in the presence of duplicates.
   *
   * <p>The elements of the returned {@code Iterable} must each be:
   *
   * <ul>
   *   <li>Non-null

    String getModelId();

    @Nullable
    String getLocation();

    boolean isAddDefaultEntities();

    interface Transformer {
        /**
         * Interpolate the value read from the xml document
         *
         * @param source    The source value
         * @param fieldName A description of the field being interpolated. The implementation may use this to
         *                  log stuff.

 * entire "chunk" (of implementation-dependent length) is ready to be hashed.
 *
 * @author Kevin Bourrillion
 * @author Dimitris Andreou
 */
// TODO(kevinb): this class still needs some design-and-document-for-inheritance love
abstract class AbstractStreamingHasher extends AbstractHasher {
  /** Buffer via which we pass data to the hash algorithm (the implementor) */
  private final ByteBuffer buffer;