Search Options

Results per page
Sort
Preferred Languages
Advance

Results 1 - 1 of 1 for stripComment (0.08 sec)

  1. fess-crawler/src/main/java/org/codelibs/fess/crawler/helper/RobotsTxtHelper.java

                final List<Directive> currentDirectiveList = new ArrayList<>();
                boolean isGroupRecordStarted = false;
                while ((line = reader.readLine()) != null) {
                    line = stripComment(line).trim();
                    if (StringUtil.isEmpty(line)) {
                        continue;
                    }
    
                    String value = getValue(USER_AGENT_RECORD, line);
    Registered: Sun Sep 21 03:50:09 UTC 2025
    - Last Modified: Sun Jul 06 02:13:03 UTC 2025
    - 7.7K bytes
    - Viewed (0)
Back to top