* <li>Applies document boosting rules and click/favorite count enhancements</li>
 * <li>Sends processed documents to the search engine for indexing</li>
 * <li>Manages cleanup of processed crawler session data</li>
 * </ul>
 *
 * <p>The updater runs continuously until crawling is finished and all documents are processed.
 * It includes error handling, retry logic, and performance monitoring capabilities.
 *
 */

        try {
            Thread.sleep(2000);  // 2 second grace period
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
    }
    
    // Proceed with normal error handling/retry
    super.handleConnectionLoss(error);
}
```

## 6. Configuration

### 6.1 Configuration Properties
```java
// In PropertyConfiguration.java

     *
     * @param responseData the response data from the crawled resource
     * @param resultData the result data to be processed
     * @return the access result containing the processed data
     */
    @Override
    protected AccessResult<?> createAccessResult(final ResponseData responseData, final ResultData resultData) {

     * Default implementation delegates to the basic process method.
     *
     * @param target the document data to process
     * @param accessResult the access result from crawling
     * @return the processed document data
     */
    public Map<String, Object> process(final Map<String, Object> target, final AccessResult<String> accessResult) {
        return process(target);
    }

    /**

     * Applies all available ingesters to transform the document data.
     *
     * @param paramMap the data store parameters
     * @param dataMap the document data to process
     * @return the processed document data
     */
    protected Map<String, Object> ingest(final DataStoreParams paramMap, final Map<String, Object> dataMap) {
        if (ingestFactory == null) {
            return dataMap;

     * Removes protocol, query parameters, and applies URL decoding based on encoding settings.
     *
     * @param u the URL string to process
     * @param encoding the character encoding to use for URL decoding
     * @return the processed site path, abbreviated if necessary
     */
    default String getSite(final String u, final String encoding) {
        if (StringUtil.isBlank(u)) {

 * Implementations of this interface are responsible for handling the response data
 * obtained during a crawling process.
 */
public interface ResponseProcessor {

    /**
     * Processes the given response data.
     *
     * @param responseData the response data to be processed
     */
    void process(ResponseData responseData);

/**
 * A thread that reads from an input stream line by line and maintains a buffer of recent lines.
 * This class provides functionality to read input stream data asynchronously,
 * optionally process each line with a callback function, and maintain a circular buffer
 * of recent lines for retrieval.
 */
public class InputStreamThread extends Thread {
    /** Logger instance for this class */

 * from each sitemap, and creates a new {@link RequestData} object for each URL.
 * These RequestData objects are added to a set of child URLs, which are then
 * passed to a {@link ChildUrlsException} to be processed by the crawler.
 * </p>
 *
 * <p>
 * The class also handles potential {@link IOException}s that may occur during
 * the parsing of the response body.
 * </p>
 */

 * This interface provides a contract for different script engine implementations
 * to process template strings with parameter substitution.
 */
public interface ScriptEngine {

    /**
     * Evaluates a template string with the provided parameter map.
     * The template is processed using the script engine's templating mechanism,
     * with parameters from the paramMap substituted into the template.
     *