if (userAgent == null) {
                Properties headers = new Properties();

                headers.put(
                        "User-Agent",
                        ConfigUtils.getString(
                                legacySupport.getRepositorySession(), "Maven", ConfigurationProperties.USER_AGENT));
                try {

> Host: localhost:9000
> User-Agent: curl/7.81.0
> Accept: */*
>

- `aws:username` - This is a string containing the friendly name of the current user, this value would point to STS temporary credential in `AssumeRole`ed requests, use `jwt:preferred_username` in case of OpenID connect and `ldap:username`...

     */
    public static WebApiManagerFactory getWebApiManagerFactory() {
        return getComponent(WEB_API_MANAGER_FACTORY);
    }

    /**
     * Gets the user agent helper component.
     * @return The user agent helper.
     */
    public static UserAgentHelper getUserAgentHelper() {
        return getComponent(USER_AGENT_HELPER);
    }

    /**
     * Gets the data store factory component.

extractorFactory.addExtractor("text/html", htmlExtractor, 2);  // Weight 2
extractorFactory.addExtractor("text/html", tikaExtractor, 1);  // Fallback
```

### Helpers

**RobotsTxtHelper**: RFC 9309 parsing, user-agent matching, crawl-delay, sitemaps
**SitemapsHelper**: Sitemap XML parsing, index handling
**MimeTypeHelper**: MIME detection via Tika
**EncodingHelper**: Charset detection with BOM
**UrlConvertHelper**: URL normalization

        .method("CONNECT", null)
        .header("Host", route.address.url.toHostHeader(includeDefaultPort = true))
        .header("Proxy-Connection", "Keep-Alive") // For HTTP/1.0 proxies like Squid.
        .header("User-Agent", USER_AGENT)
        .build()

    val fakeAuthChallengeResponse =
      Response
        .Builder()
        .request(proxyConnectRequest)
        .protocol(Protocol.HTTP_1_1)

Using OkHttp is easy. Its request/response API is designed with fluent builders and immutability. It
supports both synchronous blocking calls and async calls with callbacks.

A well behaved user agent
-------------------------

OkHttp follows modern HTTP specifications such as

* HTTP Semantics - [RFC 9110](https://datatracker.ietf.org/doc/html/rfc9110)

labels.regularName=Regular Name
labels.replacement=Replacement
labels.sessionId=Session ID
labels.sortOrder=Sort Order
labels.updatedBy=Updated By
labels.updatedTime=Updated Time
labels.urls=URLs
labels.userAgent=User Agent
labels.username=Username
labels.value=Value
labels.versionNo=Version No
labels.cronExpression=Schedule
labels.dayForCleanup=Delete documents older than specified days
labels.crawlingThreadCount=Concurrent Crawling Count

   * by the application:
   *
   * * It may be transformed by the user's interceptors. For example, an application interceptor
   *   may add headers like `User-Agent`.
   * * It may be the request generated in response to an HTTP redirect or authentication
   *   challenge. In this case the request URL may be different than the initial request URL.
   *

      Header("retry-after", ""),
      Header("server", ""),
      Header("set-cookie", ""),
      Header("strict-transport-security", ""),
      Header("transfer-encoding", ""),
      Header("user-agent", ""),
      Header("vary", ""),
      Header("via", ""),
      Header("www-authenticate", ""),
    )

  val NAME_TO_FIRST_INDEX = nameToFirstIndex()