- Sort Score
- Result 10 results
- Languages All
Results 1 - 1 of 1 for getCrawlDelay (0.05 sec)
-
fess-crawler/src/main/java/org/codelibs/fess/crawler/entity/RobotsTxt.java
* or no crawl delay is specified for the matching directive. */ public int getCrawlDelay(final String userAgent) { final Directive directive = getMatchedDirective(userAgent); if (directive == null) { return 0; } return directive.getCrawlDelay(); } /** * Returns the most specific directive matching the given user agent.
Registered: Sun Sep 21 03:50:09 UTC 2025 - Last Modified: Sun Jul 06 02:13:03 UTC 2025 - 10K bytes - Viewed (0)