Search Options

Results per page
Sort
Preferred Languages
Advance

Results 1 - 6 of 6 for duplicates (0.2 sec)

  1. fess-crawler/src/main/java/org/codelibs/fess/crawler/service/impl/UrlQueueServiceImpl.java

     *   <li>Deleting URL queues associated with a session.</li>
     *   <li>Deleting all URL queues.</li>
     *   <li>Offering a list of URLs to the queue, ensuring duplicates are not added.</li>
     *   <li>Polling (retrieving and removing) a URL from the queue.</li>
     *   <li>Saving the session (currently a no-op).</li>
     *   <li>Checking if a URL has already been visited.</li>
    Registered: Sun Sep 21 03:50:09 UTC 2025
    - Last Modified: Sun Jul 06 02:13:03 UTC 2025
    - 9.3K bytes
    - Viewed (0)
  2. fess-crawler/src/main/java/org/codelibs/fess/crawler/transformer/impl/FileTransformer.java

        }
    
        /**
         * Gets the maximum number of duplicated paths to attempt.
         *
         * @return the maximum duplicated path count
         */
        public int getMaxDuplicatedPath() {
            return maxDuplicatedPath;
        }
    
        /**
         * Sets the maximum number of duplicated paths to attempt.
         *
         * @param maxDuplicatedPath the maximum duplicated path count to set
         */
    Registered: Sun Sep 21 03:50:09 UTC 2025
    - Last Modified: Thu Aug 07 02:55:08 UTC 2025
    - 11.7K bytes
    - Viewed (0)
  3. fess-crawler/src/main/java/org/codelibs/fess/crawler/extractor/impl/TikaExtractor.java

         */
        public void setInitialBufferSize(final int initialBufferSize) {
            this.initialBufferSize = initialBufferSize;
        }
    
        /**
         * Sets whether duplicated terms are replaced.
         * @param replaceDuplication If true, duplicated terms are replaced.
         */
        public void setReplaceDuplication(final boolean replaceDuplication) {
            this.replaceDuplication = replaceDuplication;
        }
    
        /**
    Registered: Sun Sep 21 03:50:09 UTC 2025
    - Last Modified: Thu Aug 07 02:55:08 UTC 2025
    - 30.7K bytes
    - Viewed (0)
  4. fess-crawler/src/main/java/org/codelibs/fess/crawler/util/TextUtil.java

     *   <li>Optionally removing duplicate terms based on a flag.</li>
     *   <li>Limiting the maximum size of alphanumeric and symbol terms.</li>
     * </ul>
     *
     * <p>The {@link TextNormalizeContext} class provides a fluent API to configure the text
     * normalization process, including setting initial buffer capacity, maximum term sizes,
     * duplicate term removal, and custom space characters.
     *
     * <p>Example usage:
    Registered: Sun Sep 21 03:50:09 UTC 2025
    - Last Modified: Sun Jul 06 02:13:03 UTC 2025
    - 12K bytes
    - Viewed (0)
  5. fess-crawler/src/main/java/org/codelibs/fess/crawler/transformer/impl/HtmlTransformer.java

                return matcher.group(1);
            }
            return null;
        }
    
        /**
         * Gets a duplicate URL by adding or removing a trailing slash.
         *
         * @param requestData the request data to create a duplicate for
         * @return the request data with the duplicate URL
         */
        protected RequestData getDuplicateUrl(final RequestData requestData) {
            final String url = requestData.getUrl();
    Registered: Sun Sep 21 03:50:09 UTC 2025
    - Last Modified: Sun Jul 06 02:13:03 UTC 2025
    - 28.5K bytes
    - Viewed (0)
  6. fess-crawler/src/test/java/org/codelibs/fess/crawler/rule/RuleManagerTest.java

            assertEquals("rule2", rules.get(1).getRuleId());
            assertEquals("rule3", rules.get(2).getRuleId());
            assertEquals("rule4", rules.get(3).getRuleId());
        }
    
        /**
         * Test adding duplicate rules
         */
        public void test_addRule_duplicates() {
            TestRule rule = new TestRule("rule1", true);
    
            ruleManager.addRule(rule);
            ruleManager.addRule(rule); // Add same rule again
    
    Registered: Sun Sep 21 03:50:09 UTC 2025
    - Last Modified: Sat Sep 06 04:15:37 UTC 2025
    - 23.8K bytes
    - Viewed (0)
Back to top