Search Options

Results per page
Sort
Preferred Languages
Advance

Results 1 - 10 of 15 for disallows (0.55 sec)

  1. fess-crawler/src/test/java/org/codelibs/fess/crawler/entity/RobotsTxtTest.java

            directive.addDisallow("/admin/");
            directive.addDisallow("/private/");
    
            String[] disallows = directive.getDisallows();
            assertEquals(2, disallows.length);
            assertEquals("/admin/", disallows[0]);
            assertEquals("/private/", disallows[1]);
        }
    
        public void test_directiveAddDisallowNoDuplicates() {
            // Test that addDisallow doesn't add duplicates
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Thu Nov 13 13:29:22 UTC 2025
    - 14.4K bytes
    - Viewed (0)
  2. fess-crawler/src/main/java/org/codelibs/fess/crawler/entity/RobotsTxt.java

                if (!exists) {
                    allowedPaths.add(pattern);
                }
            }
    
            /**
             * Adds a disallowed path pattern to this directive.
             * Supports wildcards (*) and end-of-path ($) according to RFC 9309.
             * @param path the path pattern to disallow
             */
            public void addDisallow(final String path) {
                final PathPattern pattern = new PathPattern(path);
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Mon Nov 24 03:59:47 UTC 2025
    - 18.5K bytes
    - Viewed (0)
  3. fess-crawler/src/test/java/org/codelibs/fess/crawler/helper/RobotsTxtHelperTest.java

            assertFalse(robotsTxt.allows("/store/public/sale", "PriorityBot")); // Most specific disallow
            assertFalse(robotsTxt.allows("/store/public/sale/item", "PriorityBot"));
    
            // Test SameLengthBot - Allow wins when same length as Disallow
            // Disallow: /page, Allow: /page
            assertTrue(robotsTxt.allows("/page", "SameLengthBot")); // Allow takes precedence
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Mon Nov 24 03:59:47 UTC 2025
    - 20.6K bytes
    - Viewed (0)
  4. fess-crawler/src/test/resources/org/codelibs/fess/crawler/helper/robots_wildcard.txt

    User-agent: PriorityBot
    Disallow: /store
    Allow: /store/public
    Disallow: /store/public/sale
    
    # Test Allow vs Disallow with same length (Allow wins)
    User-agent: SameLengthBot
    Disallow: /page
    Allow: /page
    
    # Test multiple wildcards
    User-agent: MultiWildcardBot
    Disallow: /*.cgi*
    Disallow: /*?*id=*
    
    # Test literal $ in middle of pattern
    User-agent: DollarBot
    Disallow: /price$info
    
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Thu Nov 13 14:03:41 UTC 2025
    - 910 bytes
    - Viewed (0)
  5. fess-crawler/src/test/resources/org/codelibs/fess/crawler/helper/robots_malformed.txt

    Disallow: /test/
    
    # Case 9: Special characters in paths
    User-agent: SpecialCharBot
    Disallow: /path with spaces/
    Disallow: /path%20encoded/
    Disallow: /path?query=value
    Disallow: /path#fragment
    Allow: /unicode/日本語/
    
    # Case 10: Multiple User-agents in sequence
    User-agent: Bot1
    User-agent: Bot2
    User-agent: Bot3
    Disallow: /shared/
    
    # Case 11: Sitemap with various formats
    Sitemap: http://example.com/sitemap.xml
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Fri Nov 14 12:52:01 UTC 2025
    - 2.6K bytes
    - Viewed (0)
  6. fess-crawler/src/main/java/org/codelibs/fess/crawler/helper/RobotsTxtHelper.java

     * <li>Disallow and Allow directives with pattern matching</li>
     * <li>Wildcard (*) in paths - matches any sequence of characters</li>
     * <li>End-of-path ($) matching - matches the end of URL path</li>
     * <li>Crawl-delay directive</li>
     * <li>Sitemap directive</li>
     * <li>Comment support (#)</li>
     * <li>Priority-based matching (longest match wins, Allow beats Disallow at equal length)</li>
     * </ul>
     *
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Fri Nov 14 12:52:01 UTC 2025
    - 11.4K bytes
    - Viewed (0)
  7. fess-crawler/src/main/java/org/codelibs/fess/crawler/client/http/HcHttpClient.java

            this.redirectHttpStatusPattern = redirectHttpStatusPattern;
        }
    
        /**
         * Sets whether to use robots.txt disallow rules.
         *
         * @param useRobotsTxtDisallows True to use disallow rules, false otherwise
         */
        public void setUseRobotsTxtDisallows(final boolean useRobotsTxtDisallows) {
            this.useRobotsTxtDisallows = useRobotsTxtDisallows;
        }
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Sun Nov 23 12:19:14 UTC 2025
    - 53.7K bytes
    - Viewed (0)
  8. fess-crawler/src/main/java/org/codelibs/fess/crawler/client/smb/SmbClient.java

                        if (logger.isDebugEnabled()) {
                            logger.debug("ACE:{}", ace);
                        }
                        processAllowedSIDs(file, ace.getSID(), ace.isAllow() ? sidAllowSet : sidDenySet);
                    }
                    responseData.addMetaData(SMB_ALLOWED_SID_ENTRIES, sidAllowSet.toArray(new SID[sidAllowSet.size()]));
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Thu Dec 11 08:38:29 UTC 2025
    - 23.4K bytes
    - Viewed (3)
  9. fess-crawler/src/main/java/org/codelibs/fess/crawler/client/smb1/SmbClient.java

                        if (logger.isDebugEnabled()) {
                            logger.debug("ACE:{}", ace);
                        }
                        processAllowedSIDs(file, ace.getSID(), ace.isAllow() ? sidAllowSet : sidDenySet);
                    }
                    responseData.addMetaData(SMB_ALLOWED_SID_ENTRIES, sidAllowSet.toArray(new SID[sidAllowSet.size()]));
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Thu Dec 11 08:38:29 UTC 2025
    - 23.3K bytes
    - Viewed (0)
  10. CHANGELOG/CHANGELOG-1.32.md

    ### Bug or Regression
    
    - Changed the node restrictions to disallow the node to change it's ownerReferences. ([#133469](https://github.com/kubernetes/kubernetes/pull/133469), [@natherz97](https://github.com/natherz97)) [SIG Auth]
    
    ## Dependencies
    
    ### Added
    _Nothing has changed._
    
    Registered: Fri Dec 26 09:05:12 UTC 2025
    - Last Modified: Tue Dec 16 18:27:41 UTC 2025
    - 448.1K bytes
    - Viewed (0)
Back to top