Search Options

Results per page
Sort
Preferred Languages
Advance

Results 1 - 10 of 188 for nofollow (0.04 sec)

  2. github.com/codelibs/fess

    src/main/java/org/codelibs/fess/crawler/transformer/FessXpathTransformer.java 

                        }
                    if (content.contains(ROBOTS_TAG_NOFOLLOW)) {
                        nofollow = true;
                    }
                }
                if (noindex && nofollow) {
                    logger.info("META(robots=noindex,nofollow): {}", responseData.getUrl());
                    throw new ChildUrlsException(Collections.emptySet(), "#processMetaRobots");
                }
    Registered: Thu Sep 04 12:52:25 UTC 2025
    - Last Modified: Thu Aug 07 03:06:29 UTC 2025
    - 54.4K bytes
    - Viewed (0)
  3. github.com/codelibs/fess-crawler

    fess-crawler/src/main/java/org/codelibs/fess/crawler/entity/ResponseData.java 

         */
    public void setNoFollow(final boolean value) {
        noFollow = value;
    }

    /**
     * Checks if this response should not be followed for further crawling.
     *
     * @return true if this response should not be followed, false otherwise
     */
    public boolean isNoFollow() {
        return noFollow;
    }

    /**
     * Adds metadata to this response.
     *
    Registered: Sun Sep 21 03:50:09 UTC 2025
    - Last Modified: Sun Jul 06 02:13:03 UTC 2025
    - 11.6K bytes
    - Viewed (0)
  4. github.com/codelibs/fess

    src/test/java/org/codelibs/fess/mylasta/direction/FessPropTest.java 

            FessConfig fessConfig = new FessConfig.SimpleImpl() {
            @Override
            public String getCrawlerDocumentHtmlPrunedTags() {
                return "script,div#main,p.image,a[rel=nofollow],div[x-y=a-.:_0]";
            }
        };

        PrunedTag[] tags = fessConfig.getCrawlerDocumentHtmlPrunedTagsAsArray();
        assertTrue(matchesTag(tags[0], "<script></script>"));
    Registered: Thu Sep 04 12:52:25 UTC 2025
    - Last Modified: Tue Aug 19 14:09:36 UTC 2025
    - 13K bytes
    - Viewed (0)
  5. github.com/codelibs/fess

    src/test/java/org/codelibs/fess/crawler/transformer/FessXpathTransformerTest.java 

                };
        };

        final ResponseData responseData = new ResponseData();
        responseData.addMetaData("X-Robots-Tag", "nofollow");

        transformer.processXRobotsTag(responseData, new ResultData());
        assertTrue(responseData.isNoFollow());
    }

    public void test_processMetaRobots_no() throws Exception {
        final String data = "<html><body>foo</body></html>";
    Registered: Thu Sep 04 12:52:25 UTC 2025
    - Last Modified: Sat Jul 19 23:49:30 UTC 2025
    - 41.5K bytes
    - Viewed (0)
  6. github.com/codelibs/fess

    src/main/resources/fess_config.properties 

    crawler.document.html.canonical.xpath=//LINK[@rel='canonical'][1]/@href
# HTML tags to prune (remove) during document processing.
crawler.document.html.pruned.tags=noscript,script,style,header,footer,aside,nav,a[rel=nofollow]
# Maximum length of digest extracted from HTML documents.
crawler.document.html.max.digest.length=120
# Default language for HTML documents.
crawler.document.html.default.lang=
    Registered: Thu Sep 04 12:52:25 UTC 2025
    - Last Modified: Sat Jul 05 14:45:37 UTC 2025
    - 54.7K bytes
    - Viewed (0)
  7. github.com/codelibs/fess

    src/main/java/org/codelibs/fess/mylasta/direction/FessConfig.java 

        String CRAWLER_DOCUMENT_HTML_CANONICAL_XPATH = "crawler.document.html.canonical.xpath";

    /** The key of the configuration. e.g. noscript,script,style,header,footer,aside,nav,a[rel=nofollow] */
    String CRAWLER_DOCUMENT_HTML_PRUNED_TAGS = "crawler.document.html.pruned.tags";

    /** The key of the configuration. e.g. 120 */
    Registered: Thu Sep 04 12:52:25 UTC 2025
    - Last Modified: Thu Jul 17 08:28:31 UTC 2025
    - 525.6K bytes
    - Viewed (2)
  8. github.com/minio/minio

    docs/integrations/veeam/README.md 

    - Follow through the wizard as above for Veeam Backup and Replication as the steps are the same between both products

- Under Backup Infrastructure -> Backup Repositories, right click and "Add Backup Repository"

- Follow the wizard.  Under the "Object Storage Backup Repository" section, choose the MinIO object storage you created above
    Registered: Sun Sep 07 19:28:11 UTC 2025
    - Last Modified: Tue Aug 12 18:20:36 UTC 2025
    - 5.5K bytes
    - Viewed (0)
  9. github.com/square/okhttp

    okhttp/src/commonJvmAndroid/kotlin/okhttp3/internal/connection/RoutePlanner.kt 

    import okhttp3.HttpUrl

/**
 * Policy on choosing which connection to use for an exchange and any retries that follow. This uses
 * the following strategies:
 *
 *  1. If the current call already has a connection that can satisfy the request it is used. Using
 *     the same connection for an initial exchange and its follow-ups may improve locality.
 *
    Registered: Fri Sep 05 11:42:10 UTC 2025
    - Last Modified: Tue May 27 14:58:02 UTC 2025
    - 4.2K bytes
    - Viewed (0)
  10. github.com/square/okhttp

    okhttp/src/commonJvmAndroid/kotlin/okhttp3/EventListener.kt 

       *
   * Response header values like `Location` and `Retry-After` are also considered.
   *
   * Client configuration may be used to make follow-up decisions, such as:
   *
   *  * [OkHttpClient.followRedirects] must be true to follow redirects.
   *  * [OkHttpClient.followSslRedirects] must be true to follow redirects that add or remove HTTPS.
   *  * [OkHttpClient.authenticator] must respond to an authorization challenge.
   *
    Registered: Fri Sep 05 11:42:10 UTC 2025
    - Last Modified: Tue May 27 14:58:02 UTC 2025
    - 17.4K bytes
    - Viewed (0)
  11. github.com/tiangolo/fastapi

    docs/en/docs/help-fastapi.md 

    * News about FastAPI and friends 🚀
* Guides 📝
* Features ✨
* Breaking changes 🚨
* Tips and tricks ✅

## Follow FastAPI on X (Twitter) { #follow-fastapi-on-x-twitter }

<a href="https://x.com/fastapi" class="external-link" target="_blank">Follow @fastapi on **X (Twitter)**</a> to get the latest news about **FastAPI**. 🐦

## Star **FastAPI** in GitHub { #star-fastapi-in-github }
    Registered: Sun Sep 07 07:19:17 UTC 2025
    - Last Modified: Sun Aug 31 10:49:48 UTC 2025
    - 14K bytes
    - Viewed (0)
Back to top