* This class handles URL conversion based on duplicate host rules, allowing multiple
 * hostnames or URLs to be treated as equivalent for crawling and indexing purposes.
 * It maintains a list of DuplicateHost rules and applies them to URLs.
 *
 */
public class DuplicateHostHelper {
    private static final Logger logger = LogManager.getLogger(DuplicateHostHelper.class);

judu
judvi
judviejų
jųdviejų
judviem
judviese
jumis
jums
jumyse
juo
juodu
juodviese
juos
juose
jus
jūs
jūsų
ką
kad
kai
kaip
kas
kiek
kol
kur
kurie
kuris
man
mane
manęs
manimi
mano
manyje
mes
metu
mudu
mudvi
mudviejų
mudviem
mudviese
mumis
mums
mumyse
mus
mūsų
nei
nes
net
nors
nuo
o
pat
per

Das kann nützlich sein, wenn Sie beispielsweise in einem Land leben, in dem bestimmte URLs eingeschränkt sind.

### Die automatischen Dokumentationen deaktivieren

Der erste Schritt besteht darin, die automatischen Dokumentationen zu deaktivieren, da diese standardmäßig das Standard-CDN verwenden.

Um diese zu deaktivieren, setzen Sie deren URLs beim Erstellen Ihrer `FastAPI`-App auf `None`:

This could be useful if for example you live in a country that restricts some URLs.

### Disable the automatic docs { #disable-the-automatic-docs }

The first step is to disable the automatic docs, as by default, those use the default CDN.

To disable them, set their URLs to `None` when creating your `FastAPI` app:

{* ../../docs_src/custom_docs_ui/tutorial001.py hl[8] *}

 *
 * <p>
 * It provides methods to set and retrieve the server address, port, username,
 * password, and domain. Additionally, it offers a method to construct a path
 * prefix for SMB URLs based on the configured server and port.
 * </p>
 *
 * <p>
 * The path prefix is in the format "smb://server:port/", where the port is
 * included only if it's greater than 0. If the server is not set, the path

        public QueueHolder() {
            // Default constructor
        }

        /**
         * The queue for URLs waiting to be crawled.
         */
        protected Queue<OpenSearchUrlQueue> waitingQueue = new ConcurrentLinkedQueue<>();

        /**
         * The queue for URLs currently being crawled.
         */
        protected Queue<OpenSearchUrlQueue> crawlingQueue = new ConcurrentLinkedQueue<>();
    }

     */
    @Size(max = 10000)
    public String virtualHostValue;

    /**
     * Enable or disable appending query parameters to search URLs.
     * When enabled, additional parameters are added to search result URLs.
     */
    @Size(max = 10)
    public String appendQueryParameter;

    /**
     * Enable or disable login requirement for search access.

        }
    }

    /**
     * Registers a client component with a regular expression.
     * The component will be loaded into all registered CrawlerClientFactories.
     * @param regex The regular expression to match URLs.
     * @param componentName The name of the component to register.
     */
    public synchronized void register(final String regex, final String componentName) {
        clientMap.put(regex, componentName);

import com.google.common.annotations.GwtCompatible;
import com.google.common.escape.Escaper;

/**
 * {@code Escaper} instances suitable for strings to be included in particular sections of URLs.
 *
 * <p>If the resulting URLs are inserted into an HTML or XML document, they will require additional
 * escaping with {@link com.google.common.html.HtmlEscapers} or {@link
 * com.google.common.xml.XmlEscapers}.
 *
 * @author David Beaumont

    }

    /**
     * Retrieves a list of URLs that should be excluded from crawling based on failure counts.
     * URLs are excluded if they have failed more than the configured failure count threshold.
     * URLs can also be filtered by failure type using a regular expression pattern.
     *
     * @param configId the configuration ID to get excluded URLs for