Search Options

Results per page
Sort
Preferred Languages
Advance

Results 1 - 1 of 1 for getDirectives (0.07 sec)

  1. fess-crawler/src/main/java/org/codelibs/fess/crawler/helper/RobotsTxtHelper.java

                            isGroupRecodeStarted = false;
                        }
                        final String userAgent = value.toLowerCase(Locale.ENGLISH);
                        Directive currentDirective = robotsTxt.getDirective(userAgent);
                        if (currentDirective == null) {
                            currentDirective = new Directive(userAgent);
                            robotsTxt.addDirective(currentDirective);
    Registered: Sun Nov 10 03:50:12 UTC 2024
    - Last Modified: Sat Oct 12 01:40:57 UTC 2024
    - 6.1K bytes
    - Viewed (0)
Back to top