Search Options

Results per page
Sort
Preferred Languages
Advance

Results 1 - 8 of 8 for CrawlerThread (0.06 sec)

  1. src/main/resources/crawler_opensearch+crawlerThread.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE components PUBLIC "-//DBFLUTE//DTD LastaDi 1.0//EN" 
    	"http://dbflute.org/meta/lastadi10.dtd">
    <components namespace="fessCrawler">
    	<!-- Crawler Thread -->
    	<component name="crawlerThread" class="org.codelibs.fess.crawler.FessCrawlerThread" instance="prototype" >
    	</component>
    Registered: Sat Dec 20 09:19:18 UTC 2025
    - Last Modified: Thu Oct 30 05:29:49 UTC 2025
    - 342 bytes
    - Viewed (0)
  2. fess-crawler/src/test/java/org/codelibs/fess/crawler/CrawlerThreadTest.java

            crawlerContext.ruleManager = ruleManager;
    
            crawlerThread.urlQueueService = urlQueueService;
            crawlerThread.dataService = dataService;
            crawlerThread.crawlerContainer = crawlerContainer;
            crawlerThread.logHelper = logHelper;
            crawlerThread.setClientFactory(clientFactory);
            crawlerThread.setCrawlerContext(crawlerContext);
    
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Mon Nov 24 03:59:47 UTC 2025
    - 18.3K bytes
    - Viewed (0)
  3. fess-crawler/src/main/java/org/codelibs/fess/crawler/Crawler.java

            for (int i = 0; i < crawlerContext.getNumOfThread(); i++) {
                final CrawlerThread crawlerThread = crawlerContainer.getComponent("crawlerThread");
                crawlerThread.setCrawlerContext(crawlerContext);
                crawlerThread.setClientFactory(clientFactory);
                threads[i] =
                        new Thread(crawlerThreadGroup, crawlerThread, "Crawler-" + crawlerContext.sessionId + "-" + Integer.toString(i + 1));
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Mon Nov 24 03:59:47 UTC 2025
    - 17K bytes
    - Viewed (0)
  4. CLAUDE.md

    int numOfThread = 10            // Crawler threads
    int maxDepth = -1               // Max depth (-1 = unlimited)
    long maxAccessCount = 0         // Max URLs (0 = unlimited)
    ```
    
    ### CrawlerThread (`CrawlerThread.java`)
    
    Worker thread for crawling.
    
    **Flow**: Poll URL → Validate → Get client → Delay → Check last-modified → Execute → Process → Extract children → Queue children → Delay
    
    ### CrawlerClientFactory
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Fri Nov 28 17:31:34 UTC 2025
    - 10.7K bytes
    - Viewed (0)
  5. src/main/java/org/codelibs/fess/exec/Crawler.java

         *
         * @param crawlerThread the thread to wait for; null threads are ignored
         */
        private void joinCrawlerThread(final Thread crawlerThread) {
            if (crawlerThread != null) {
                try {
                    crawlerThread.join();
                } catch (final Exception e) {
                    logger.info("Interrupted crawling process: name={}", crawlerThread.getName());
                }
            }
    Registered: Sat Dec 20 09:19:18 UTC 2025
    - Last Modified: Fri Nov 28 16:29:12 UTC 2025
    - 31.4K bytes
    - Viewed (0)
  6. src/main/java/org/codelibs/fess/crawler/FessCrawlerThread.java

     * <li>Integration with Fess configuration and permission systems</li>
     * <li>Client selection based on URL patterns</li>
     * </ul>
     *
     * @see CrawlerThread
     * @see org.codelibs.fess.crawler.client.CrawlerClient
     */
    public class FessCrawlerThread extends CrawlerThread {
    
        /**
         * Default constructor.
         */
        public FessCrawlerThread() {
            super();
        }
    
    Registered: Sat Dec 20 09:19:18 UTC 2025
    - Last Modified: Thu Dec 11 09:47:03 UTC 2025
    - 19.5K bytes
    - Viewed (0)
  7. fess-crawler/src/test/java/org/codelibs/fess/crawler/CrawlerTest.java

            })
                    .prototype("accessResult", AccessResultImpl.class)
                    .prototype("urlQueue", UrlQueueImpl.class)
                    .prototype("crawlerThread", CrawlerThread.class)
                    .prototype("crawler", Crawler.class)
                    .prototype("urlFilterService", UrlFilterServiceImpl.class)
                    .prototype("urlQueueService", UrlQueueServiceImpl.class)
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Tue Nov 11 13:40:14 UTC 2025
    - 25.8K bytes
    - Viewed (0)
  8. src/main/java/org/codelibs/fess/helper/DataIndexHelper.java

            while (startedCrawlerNum < dataCrawlingThreadList.size()) {
                // Force to stop crawl
                if (systemHelper.isForceStop()) {
                    for (final DataCrawlingThread crawlerThread : dataCrawlingThreadList) {
                        crawlerThread.stopCrawling();
                    }
                    break;
                }
    
                if (activeCrawlerNum < multiprocessCrawlingCount) {
                    // start crawling
    Registered: Sat Dec 20 09:19:18 UTC 2025
    - Last Modified: Fri Nov 28 16:29:12 UTC 2025
    - 19K bytes
    - Viewed (0)
Back to top