- Sort Score
- Result 10 results
- Languages All
Results 1 - 1 of 1 for processRobotsTxt (0.11 sec)
-
fess-crawler/src/test/java/org/codelibs/fess/crawler/client/http/HcHttpClientTest.java
urlFilter.init(sessionId); crawlerContext.setUrlFilter(urlFilter); CrawlingParameterUtil.setCrawlerContext(crawlerContext); httpClient.init(); httpClient.processRobotsTxt(url); assertEquals(1, crawlerContext.getRobotsTxtUrlSet().size()); assertTrue(crawlerContext.getRobotsTxtUrlSet().contains("http://localhost:7070/robots.txt"));
Registered: Wed Jun 12 15:17:51 UTC 2024 - Last Modified: Thu May 09 09:28:25 UTC 2024 - 8.8K bytes - Viewed (0)