* and performs bulk indexing operations with proper error handling.
     *
     * @param searchEngineClient the search engine client to use for indexing
     * @param docList the list of documents to be indexed
     * @throws SearchEngineClientException if the bulk indexing operation fails
     */
    public void sendDocuments(final SearchEngineClient searchEngineClient, final DocList docList) {
        if (docList.isEmpty()) {

 * <li>Transforms document data using appropriate transformers</li>
 * <li>Applies document boosting rules and click/favorite count enhancements</li>
 * <li>Sends processed documents to the search engine for indexing</li>
 * <li>Manages cleanup of processed crawler session data</li>
 * </ul>
 *
 * <p>The updater runs continuously until crawling is finished and all documents are processed.

        // Add first document - should trigger indexing because cache size is 1
        Map<String, Object> dataMap1 = new HashMap<>();
        dataMap1.put("url", "http://example.com/test1");
        indexUpdateCallback.store(paramMap, dataMap1);
        assertEquals(1, indexingHelper.sendDocumentsCalled);

        // Add second document - should trigger indexing again
        Map<String, Object> dataMap2 = new HashMap<>();

import org.codelibs.fess.util.DocumentUtil;
import org.codelibs.fess.util.MemoryUtil;

import jakarta.annotation.PostConstruct;

/**
 * Implementation of IndexUpdateCallback for handling document indexing operations.
 * This class manages the process of updating the search index with documents from
 * data stores, including bulk operations, document transformation, and error handling.
 */

1. **Full UI Translation** (13 languages): Complete user interface translation with label and message files
2. **Search/Analysis Support** (40+ languages): Language-specific text analysis for search indexing

Currently supported UI languages: German, English, Spanish, French, Italian, Japanese, Korean, Dutch, Polish, Brazilian Portuguese, Russian, Simplified Chinese, and Traditional Chinese.

## File Structure

            }
        });
        refresh();
    }

    /**
     * Indexes suggest data from documents.
     *
     * @param success A callback for successful indexing.
     * @param error   A callback for indexing errors.
     */
    public void indexFromDocuments(final Consumer<Boolean> success, final Consumer<Throwable> error) {
        final FessConfig fessConfig = ComponentUtil.getFessConfig();

/**
 * Helper class for managing data crawling operations in Fess.
 * This class coordinates the execution of data store crawling processes,
 * managing multiple concurrent crawling threads and handling the indexing
 * of crawled documents into the search engine.
 *
 * <p>The DataIndexHelper supports:</p>
 * <ul>
 *   <li>Concurrent crawling of multiple data configurations</li>

 * Helper class for managing duplicate host configurations in the Fess search system.
 * This class handles URL conversion based on duplicate host rules, allowing multiple
 * hostnames or URLs to be treated as equivalent for crawling and indexing purposes.
 * It maintains a list of DuplicateHost rules and applies them to URLs.
 *
 */
public class DuplicateHostHelper {
    private static final Logger logger = LogManager.getLogger(DuplicateHostHelper.class);

    protected Set<String> sortFieldSet;

    /** Set of fields that are allowed in API responses */
    protected Set<String> apiResponseFieldSet;

    /** Set of fields that are not analyzed during indexing */
    protected Set<String> notAnalyzedFieldSet;

    /** List of additional default fields with their boost values */
    protected List<Pair<String, Float>> additionalDefaultList = new ArrayList<>();

    /**

 * This class provides constants for system configuration, crawling and indexing defaults, user agent strings,
 * status values, field names, date/time formats, authentication types, and various reserved words and patterns.
 *
 * <p>Key constant categories include:</p>
 * <ul>
 * <li>System configuration property keys</li>
 * <li>Default values for crawling and indexing operations</li>
 * <li>User agent strings and patterns for web crawling</li>