* governing permissions and limitations under the License.
 */
package org.codelibs.fess.crawler.exception;

/**
 * RobotsTxtException is an exception class that represents an error related to robots.txt processing during web crawling.
 * It extends CrawlerSystemException and provides constructors to create instances with a message and/or a cause.
 *
 */
public class RobotsTxtException extends CrawlerSystemException {

## Key Features

- **Smart Query Suggestions**: Real-time auto-completion and search suggestions
- **Multi-language Support**: Built-in support for Japanese text processing with Kuromoji analyzer
- **Popular Words Analytics**: Track and analyze frequently searched terms
- **Flexible Text Processing**: Configurable converters and normalizers for text transformation
- **OpenSearch Integration**: Seamless integration with OpenSearch/Elasticsearch clusters

 * governing permissions and limitations under the License.
 */
package org.codelibs.fess.crawler.exception;

/**
 * Exception thrown during sitemaps processing in the crawler.
 * This exception extends {@link org.codelibs.fess.crawler.exception.CrawlerSystemException}
 * and provides constructors for wrapping other exceptions or creating a new exception with a message.
 */

- **SmbClient**: SMB/CIFS network shares
- **StorageClient**: Cloud storage integration

#### Content Processing Pipeline
- **Extractors**: Content extraction from various formats
- **Transformers**: Data transformation and enrichment
- **Filters**: URL filtering with regex patterns
- **Rules**: Content processing rules and validation

## Building and Testing

### Build Commands

```bash
# Build all modules

import org.apache.logging.log4j.Logger;
import org.codelibs.core.lang.StringUtil;

/**
 * Utility class for text normalization and processing.
 *
 * This class provides methods to normalize text by reading characters from a provided Reader
 * and processing them according to specific rules. The main functionality is encapsulated
 * within the nested {@link TextNormalizeContext} class.
 *

 */
package org.codelibs.fess.crawler.processor;

import org.codelibs.fess.crawler.entity.ResponseData;

/**
 * The ResponseProcessor interface defines a contract for processing response data.
 * Implementations of this interface are responsible for handling the response data
 * obtained during a crawling process.
 */
public interface ResponseProcessor {

    /**

            } catch (final Exception e) {
                if (logger.isDebugEnabled()) {
                    logger.debug("Exception on SID processing.", e);
                }
            }
        }
    }

    /**
     * Preprocesses the URI before processing the request.
     *
     * @param uri the URI to preprocess
     * @return the preprocessed URI

 * governing permissions and limitations under the License.
 */
package org.codelibs.fess.crawler.util;

import org.codelibs.fess.crawler.entity.AccessResult;

/**
 * A callback interface for processing access results.
 *
 * @param <RESULT> the type of access result that extends {@link AccessResult}
 */
public interface AccessResultCallback<RESULT extends AccessResult<?>> {
    /**

 * It maintains a list of ReadingConverter instances and iterates through them, applying each converter to the input text and accumulating the results.
 * The chain stops processing when the maximum number of readings is reached.
 */
public class ReadingConverterChain implements ReadingConverter {
    /**
     * Constructs a new {@link ReadingConverterChain}.
     */

 */
package org.codelibs.fess.suggest.analysis;

import java.util.List;

import org.opensearch.action.admin.indices.analyze.AnalyzeAction.AnalyzeToken;

/**
 * Interface for analyzing and processing suggestion tokens.
 */
public interface SuggestAnalyzer {

    /**
     * Analyzes the given text and returns a list of tokens.
     *
     * @param text the text to analyze