Search Options

Results per page
Sort
Preferred Languages
Advance

Results 11 - 14 of 14 for addExcludes (0.07 sec)

  1. CLAUDE.md

    ### Configuring URL Filtering
    
    ```java
    // Include patterns (must match)
    crawler.urlFilter.addInclude("https://example.com/.*");
    
    // Exclude patterns (must not match)
    crawler.urlFilter.addExclude(".*\\.(css|js|png|jpg)$");
    ```
    
    ### Setting Crawl Limits
    
    ```java
    context.setMaxAccessCount(1000);  // Max URLs (0 = unlimited)
    context.setMaxDepth(3);           // Max depth (-1 = unlimited)
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Fri Nov 28 17:31:34 UTC 2025
    - 10.7K bytes
    - Viewed (0)
  2. fess-crawler/src/test/java/org/codelibs/fess/crawler/CrawlerContextTest.java

            @Override
            public void init(String sessionId) {
            }
    
            @Override
            public void addInclude(String urlPattern) {
            }
    
            @Override
            public void addExclude(String urlPattern) {
            }
    
            @Override
            public boolean match(String url) {
                return true;
            }
    
            @Override
            public void processUrl(String url) {
            }
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Sat Sep 06 04:15:37 UTC 2025
    - 25.6K bytes
    - Viewed (0)
  3. fess-crawler/src/test/java/org/codelibs/fess/crawler/CrawlerTest.java

            crawler.crawlerContext.setMaxAccessCount(maxCount);
            crawler.crawlerContext.setNumOfThread(numOfThread);
            crawler.urlFilter.addInclude(url + ".*");
            crawler.urlFilter.addExclude(url + "/dir1/.*");
            final String sessionId = crawler.execute();
            assertEquals(maxCount, dataService.getCount(sessionId));
            dataService.delete(sessionId);
        }
    
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Tue Nov 11 13:40:14 UTC 2025
    - 25.8K bytes
    - Viewed (0)
  4. fess-crawler/src/main/java/org/codelibs/fess/crawler/client/http/HcHttpClient.java

                                            final String urlValue = hostUrl + urlPattern;
                                            crawlerContext.getUrlFilter().addExclude(urlValue);
                                            if (logger.isInfoEnabled()) {
                                                logger.info("Excluded URL: {}", urlValue);
                                            }
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Sun Nov 23 12:19:14 UTC 2025
    - 53.7K bytes
    - Viewed (0)
Back to top