- Sort Score
- Result 10 results
- Languages All
Results 1 - 2 of 2 for getDisallows (0.07 sec)
-
fess-crawler/src/main/java/org/codelibs/fess/crawler/entity/RobotsTxt.java
Registered: Sun Sep 21 03:50:09 UTC 2025 - Last Modified: Sun Jul 06 02:13:03 UTC 2025 - 10K bytes - Viewed (0) -
fess-crawler/src/main/java/org/codelibs/fess/crawler/client/http/HcHttpClient.java
if (directive != null) { if (useRobotsTxtDisallows) { for (String urlPattern : directive.getDisallows()) { if (StringUtil.isNotBlank(urlPattern)) { urlPattern = convertRobotsTxtPatternToRegex(urlPattern);
Registered: Sun Sep 21 03:50:09 UTC 2025 - Last Modified: Thu Aug 07 02:55:08 UTC 2025 - 52.2K bytes - Viewed (0)