Search Options

Results per page
Sort
Preferred Languages
Advance

Results 11 - 15 of 15 for intervalController (1.45 sec)

  1. fess-crawler/src/main/java/org/codelibs/fess/crawler/interval/impl/DefaultIntervalController.java

     */
    package org.codelibs.fess.crawler.interval.impl;
    
    import java.util.Map;
    
    import org.codelibs.core.lang.ThreadUtil;
    
    /**
     * Default implementation of the IntervalController.
     * This class provides a default way to manage delays between crawler operations.
     * It allows setting delays before processing, after processing, when no URLs are in the queue,
     * and when waiting for new URLs.
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Thu Nov 20 08:58:39 UTC 2025
    - 5.8K bytes
    - Viewed (0)
  2. fess-crawler/src/main/java/org/codelibs/fess/crawler/interval/impl/HostIntervalController.java

    import com.google.common.cache.Cache;
    import com.google.common.cache.CacheBuilder;
    
    /**
     * HostIntervalController is an implementation of {@link org.codelibs.fess.crawler.interval.IntervalController}
     * that controls the interval between requests to the same host.
     * It uses a Guava Cache to store the last access time for each host.
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Mon Nov 24 03:59:47 UTC 2025
    - 5.2K bytes
    - Viewed (0)
  3. CLAUDE.md

    void addUrl(String url)         // Add URL to queue
    void cleanup(String sessionId)  // Clean up session
    void stop()                     // Stop gracefully
    ```
    
    **Key Fields**: `crawlerContext`, `urlFilter`, `intervalController`, `clientFactory`, `ruleManager`
    
    ### CrawlerContext (`CrawlerContext.java`)
    
    Execution context and configuration.
    
    **Important Fields**:
    ```java
    String sessionId                // Format: yyyyMMddHHmmssSSS
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Fri Nov 28 17:31:34 UTC 2025
    - 10.7K bytes
    - Viewed (0)
  4. README.md

    String sessionId2 = crawler2.execute();
    
    crawler1.awaitTermination();
    crawler2.awaitTermination();
    ```
    
    ### Custom Interval Control
    
    ```java
    // Configure politeness policy
    container.singleton("intervalController", DefaultIntervalController.class, controller -> {
        controller.setDelayMillisForWaitingNewUrl(5000);
        controller.setDefaultIntervalTime(1000);
    });
    ```
    
    ### Sitemap Support
    
    ```java
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Sun Aug 31 05:32:52 UTC 2025
    - 15.3K bytes
    - Viewed (0)
  5. fess-crawler/src/test/java/org/codelibs/fess/crawler/CrawlerTest.java

                    .prototype("dataService", DataServiceImpl.class)
                    .prototype("urlFilter", UrlFilterImpl.class)
                    .singleton("urlConvertHelper", UrlConvertHelper.class)
                    .singleton("intervalController", DefaultIntervalController.class)
                    .singleton("sitemapsHelper", SitemapsHelper.class)
                    .singleton("logHelper", LogHelperImpl.class)
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Tue Nov 11 13:40:14 UTC 2025
    - 25.8K bytes
    - Viewed (0)
Back to top