- Sort Score
- Result 10 results
- Languages All
Results 1 - 2 of 2 for getDisallows (0.16 sec)
-
fess-crawler/src/main/java/org/codelibs/fess/crawler/entity/RobotsTxt.java
Registered: Wed Jun 12 15:17:51 UTC 2024 - Last Modified: Thu Feb 22 01:36:27 UTC 2024 - 4.8K bytes - Viewed (0) -
fess-crawler/src/main/java/org/codelibs/fess/crawler/client/http/HcHttpClient.java
if (directive != null) { if (useRobotsTxtDisallows) { for (String urlPattern : directive.getDisallows()) { if (StringUtil.isNotBlank(urlPattern)) { urlPattern = convertRobotsTxtPathPattern(urlPattern);
Registered: Wed Jun 12 15:17:51 UTC 2024 - Last Modified: Thu May 09 09:28:25 UTC 2024 - 41K bytes - Viewed (0)