Search Options

Results per page
Sort
Preferred Languages
Advance

Results 1 - 5 of 5 for canonical (0.25 sec)

  2. github.com/codelibs/fess

    src/main/java/org/codelibs/fess/es/config/exentity/CrawlingConfig.java 

                public static final String CLEANUP_URL_FILTERS = "cleanup.urlFilters";
            public static final String JCIFS_PREFIX = "jcifs.";
            public static final String HTML_CANONICAL_XPATH = "html.canonical.xpath";
            public static final String HTML_PRUNED_TAGS = "html.pruned.tags";
            public static final String PIPELINE = "pipeline";
    Java
    - Registered: Mon Apr 29 08:04:11 GMT 2024
    - Last Modified: Thu Feb 22 01:37:57 GMT 2024
    - 5.5K bytes
    - Viewed (0)
  3. github.com/codelibs/fess

    src/main/resources/fess_config.properties 

    # html
crawler.document.html.content.xpath=//BODY
crawler.document.html.lang.xpath=//HTML/@lang
crawler.document.html.digest.xpath=//META[@name='description']/@content
crawler.document.html.canonical.xpath=//LINK[@rel='canonical'][1]/@href
crawler.document.html.pruned.tags=noscript,script,style,header,footer,aside,nav,a[rel=nofollow]
crawler.document.html.max.digest.length=120
crawler.document.html.default.lang=
    Properties
    - Registered: Mon Apr 29 08:04:11 GMT 2024
    - Last Modified: Thu Apr 11 02:34:53 GMT 2024
    - 30.6K bytes
    - Viewed (1)
  4. github.com/codelibs/fess

    src/test/java/org/codelibs/fess/crawler/transformer/FessXpathTransformerTest.java 

            data = "<html><head><link rel=\"canonical\" href=\"http://example.com/\"></head><body>aaa</body></html>";
        document = getDocument(data);
        value = transformer.getCanonicalUrl(responseData, document);
        assertEquals("http://example.com/", value);

        data = "<html><head><link rel=\"canonical\" href=\"http://example1.com/\"><link rel=\"canonical\" href=\"http://example2.com/\"></head><body>aaa</body></html>";
    Java
    - Registered: Mon Apr 29 08:04:11 GMT 2024
    - Last Modified: Thu Feb 22 01:37:57 GMT 2024
    - 38.6K bytes
    - Viewed (0)
  5. github.com/codelibs/fess

    src/main/java/org/codelibs/fess/crawler/transformer/FessXpathTransformer.java 

                    logger.debug("Invalid Canonical Url(https->http): {} -> {}", url, canonicalUrl);
            }
            return false;
        }
        return true;
    }

    protected void putAdditionalData(final Map<String, Object> dataMap, final ResponseData responseData, final Document document) {
        // canonical
        final String canonicalUrl = getCanonicalUrl(responseData, document);
    Java
    - Registered: Mon Apr 29 08:04:11 GMT 2024
    - Last Modified: Thu Feb 22 01:37:57 GMT 2024
    - 41.9K bytes
    - Viewed (0)
  6. github.com/codelibs/fess

    src/main/java/org/codelibs/fess/mylasta/direction/FessConfig.java 

        String CRAWLER_DOCUMENT_HTML_DIGEST_XPATH = "crawler.document.html.digest.xpath";

    /** The key of the configuration. e.g. //LINK[@rel='canonical'][1]/@href */
    String CRAWLER_DOCUMENT_HTML_CANONICAL_XPATH = "crawler.document.html.canonical.xpath";

    /** The key of the configuration. e.g. noscript,script,style,header,footer,aside,nav,a[rel=nofollow] */
    Java
    - Registered: Mon Apr 29 08:04:11 GMT 2024
    - Last Modified: Thu Apr 11 02:34:53 GMT 2024
    - 459.2K bytes
    - Viewed (4)
Back to top