* because unequal URLs may be equal because of how they are hosted.
 *
 * ### Equal URLs should be equal
 *
 * These two URLs are semantically identical, but `java.net.URI` disagrees:
 *
 *  * http://host:80/
 *
 *  * http://host
 *
 * Both the unnecessary port specification (`:80`) and the absent trailing slash (`/`) cause URI to

        } catch (final Exception e) {
            return url;
        }
    }

    /**
     * Extracts anchor URLs from the HTML document.
     *
     * @param document the parsed HTML document
     * @param responseData the response data from crawling
     * @return list of anchor URLs found in the document
     */
    protected List<String> getAnchorList(final Document document, final ResponseData responseData) {

            }
            break;
        }
        return url;
    }

    /**
     * Appends query parameters to URLs based on document type.
     * Adds search highlighting for HTML and PDF documents.
     *
     * @param document the document data map
     * @param url the base URL
     * @return the URL with appended query parameters

    the thread's `UncaughtExceptionHandler`.
 *  Fix: Don't evict incomplete entries when iterating the cache. We had a bug where iterating
    `Cache.urls()` would prevent in-flight entries from being written.


## Version 3.14.3

_2019-09-10_

 *  Fix: Don't lose HTTP/2 flow control bytes when incoming data races with a stream close. If this

    assertThat(parse("http://username:@host/path"))
      .isEqualTo(parse("http://username@host/path"))
  }

  @Test
  fun passwordWithEmptyUsername() {
    // Chrome doesn't mind, but Firefox rejects URLs with empty usernames and non-empty passwords.
    assertThat(parse("http://:@host/path"))
      .isEqualTo(parse("http://host/path"))
    assertThat(parse("http://:password@@host/path").encodedPassword)

            final String[] docIds = userInfoHelper.getResultDocIds(URLDecoder.decode(queryId, Constants.UTF_8));
            if (docIds == null) {
                throw new WebApiException(HttpServletResponse.SC_BAD_REQUEST, "No searched urls.");
            }

            searchHelper
                    .getDocumentByDocId(docId, new String[] { fessConfig.getIndexFieldUrl(), fessConfig.getIndexFieldLang() },
                            OptionalThing.empty())

     * protocols. The http/1.1 transport will never be dropped.
     *
     * If multiple protocols are specified, [ALPN][alpn] will be used to negotiate a transport.
     * Protocol negotiation is only attempted for HTTPS URLs.
     *
     * [Protocol.HTTP_1_0] is not supported in this set. Requests are initiated with `HTTP/1.1`. If
     * the server responds with `HTTP/1.0`, that will be exposed by [Response.protocol].
     *

// Excerpts from @lsegal - https://github.com/aws/aws-sdk-js/issues/659#issuecomment-120477258
//
//	User-Agent:
//
//	    This is ignored from signing because signing this causes problems with generating pre-signed URLs
//	    (that are executed by other agents) or when customers pass requests through proxies, which may
//	    modify the user-agent.
//
//	Authorization:
//
//	    Is skipped for obvious reasons

# Whether to ignore robots meta tags during crawling.
crawler.ignore.robots.tags=false
# Whether to ignore content exceptions during crawling.
crawler.ignore.content.exception=true
# HTTP status codes considered as failure URLs.
crawler.failure.url.status.codes=404
# Interval (seconds) for system monitor during crawling.
crawler.system.monitor.interval=60
# Whether to ignore idle threads in hot thread monitoring.

labels.regex=Регулярное выражение
labels.regularName=Обычное имя
labels.replacement=Замена
labels.sessionId=ID сессии
labels.sortOrder=Порядок сортировки
labels.updatedBy=Обновлено
labels.updatedTime=Время обновления
labels.urls=URL-адреса
labels.userAgent=User Agent
labels.username=Имя пользователя
labels.value=Значение
labels.versionNo=Номер версии
labels.cronExpression=Расписание
labels.dayForCleanup=Удалить документы старше указанных дней