* passed to a {@link ChildUrlsException} to be processed by the crawler.
 * </p>
 *
 * <p>
 * The class also handles potential {@link IOException}s that may occur during
 * the parsing of the response body.
 * </p>
 */
public class SitemapsResponseProcessor implements ResponseProcessor {
    /** The crawler container for component lookup. */
    @Resource
    protected CrawlerContainer crawlerContainer;

 * HcHttpClient is an HTTP client implementation that extends AbstractCrawlerClient.
 * It provides various configurations and settings for making HTTP requests, including
 * connection timeouts, proxy settings, user agent, request headers, cookie management,
 * and SSL configurations. The client also supports robots.txt parsing and form-based
 * authentication schemes.
 *
 * <p>Key properties and configurations:</p>
 * <ul>

 * <p>It also handles the exclusion of search words based on certain criteria and manages the
 * maximum length of analyzed content.</p>
 *
 * <p>Methods in this class may throw SuggesterException in case of failures during the parsing process.</p>
 *
 * <p>Usage example:</p>
 * <pre>
 * {@code
 * DefaultContentsParser parser = new DefaultContentsParser();

    public CrawlerClientFactory getParent() {
        return factory;
    }

    /**
     * Sets the initialization parameter map for this wrapper.
     * These parameters will be applied to the wrapped factory upon calling initParameterMap().
     * @param params The map of parameters.
     */
    @Override
    public void setInitParameterMap(final Map<String, Object> params) {
        this.params.putAll(params);
    }

    /**

import org.codelibs.fess.suggest.entity.SuggestItem;
import org.codelibs.fess.suggest.index.contents.querylog.QueryLog;
import org.codelibs.fess.suggest.normalizer.Normalizer;

/**
 * Interface for parsing content and creating suggest items.
 */
public interface ContentsParser {
    /**
     * Parses the given search words and creates a SuggestItem.
     *
     * @param words the array of search words

        urlFilter.init(sessionId);

        // Add some patterns
        urlFilter.addInclude("https://example.com/.*");
        urlFilter.addExclude(".*\\.jpg$");

        // Verify patterns are working
        assertTrue(urlFilter.match("https://example.com/page"));
        assertFalse(urlFilter.match("https://other.com/page"));

        // Clear the filter
        urlFilter.clear();

         * @throws InvalidResponseException If the response is invalid
         * @throws NoSuchAlgorithmException If the algorithm is not available
         * @throws XmlParserException If XML parsing fails
         * @throws IOException If an I/O error occurs
         * @throws ServerException If a server error occurs
         */
        private StatObjectResponse getStatObject()

## Technology Stack

- **Java**: 21+ (configured via parent POM)
- **OpenSearch**: Latest (provided scope)
- **Apache Lucene**: Query parsing and text analysis
- **ICU4J**: Unicode text processing and normalization
- **JUnit 4**: Testing framework
- **Maven**: Build and dependency management

## Quick Start

### Prerequisites

- Java 21 or higher

     * archived URLs.
     *
     * Please note that the value of this tag is considered a hint and not a
     * command. Even though search engine crawlers may consider this information
     * when making decisions, they may crawl pages marked "hourly" less
     * frequently than that, and they may crawl pages marked "yearly" more
     * frequently than that. Crawlers may periodically crawl pages marked

            return (T) Long.valueOf(value.longValue());
        }
        return (T) field;
    }

    /**
     * Iterates through all access results for a session, calling the callback for each result.
     * Uses OpenSearch scroll API for efficient iteration over large result sets.
     *
     * @param sessionId The session ID.