*
     * <p>This method iterates through the provided array of SuggestItem objects and merges
     * items that have the same ID. The merged items are added to a new list, which is then
     * converted back to an array and returned.</p>
     *
     * @param items an array of SuggestItem objects to be merged
     * @return an array of merged SuggestItem objects
     */

     *  The implementation of this is a little slow, since it creates
     *  a number of objects each time it is called.  This could be optimized
     *  to keep the same objects around, but then thread-safety issues would arise.
     *
     *  @param contextNode The node to start searching from.
     *  @param expression A valid XPath string.

    /**
     * Cache for session queues.
     */
    protected Map<String, QueueHolder> sessionCache = new ConcurrentHashMap<>();

    /**
     * The number of URLs to fetch when polling.
     */
    protected int pollingFetchSize = 1000;

    /**
     * The maximum size of the crawling queue.
     */
    protected int maxCrawlingQueueSize = 100;

    /**

 */
package org.codelibs.fess.crawler.util;

import java.io.IOException;
import java.io.InputStream;

/**
 * A wrapper for an {@link InputStream} that ignores the {@link #close()} method call.
 * This can be useful when you want to prevent the underlying input stream from being closed.
 *
 * <p>All other methods delegate to the wrapped input stream.</p>
 *
 */
public class IgnoreCloseInputStream extends InputStream {

 *   <li>Checking if a URL has already been visited.</li>
 *   <li>Generating URL queues from a previous session's access results.</li>
 * </ul>
 *
 * <p>
 * The class uses synchronization to ensure thread safety when accessing and modifying the URL queue.
 * It also checks for duplicate URLs before adding them to the queue,
 * both within the queue itself and against previously accessed URLs stored in the {@link MemoryDataHelper}.
 * </p>
 *

      cross-claim or counterclaim in a lawsuit) alleging that the Work
      or a Contribution incorporated within the Work constitutes direct
      or contributory patent infringement, then any patent licenses
      granted to You under this License for that Work shall terminate
      as of the date such litigation is filed.

   4. Redistribution. You may reproduce and distribute copies of the

 *   <li>{@code Client} - The client used for accessing the settings.</li>
 * </ul>
 *
 * <p>Logging:</p>
 * <ul>
 *   <li>Uses {@code Logger} to log debug information when adding, deleting, or deleting all elevate words.</li>
 * </ul>
 */
public class ElevateWordSettings {
    private static final Logger logger = LogManager.getLogger(ElevateWordSettings.class);

 *   <li>Stores sitemap URLs listed in robots.txt</li>
 * </ul>
 *
 * <p>The class uses case-insensitive pattern matching for user agents and supports
 * wildcard characters (*) in user agent strings. When multiple directives match a user agent,
 * the most specific (longest) match is used.</p>
 *
 */
public class RobotsTxt {
    private static final String ALL_BOTS = "*";

 * It also includes a builder for creating extractors.
 *
 * <p>
 * The factory maintains a map of keys to an array of {@link Extractor} objects.
 * When multiple extractors are associated with a single key, they are sorted by weight
 * in descending order. The {@link #getExtractor(String)} method returns a composite

        dataService.delete(sessionId);
        urlFilter.clear();
    }

    /**
     * Closes the crawler, releasing resources.
     * This method is called automatically when the crawler is used in a try-with-resources statement.
     */
    @Override
    public void close() {
        clientFactory.close();
    }

    /**
     * Adds an include filter for URLs.