* If the response body is stored as a file, returns a FileInputStream.
     *
     * @return the response body as an InputStream, or null if no response body is available
     * @throws IORuntimeException if an I/O error occurs while reading the response body file
     */
    public InputStream getResponseBody() {
        if (responseBodyBytes != null) {

 * It implements the {@link Sitemap} interface.
 *
 * <p>
 * A Sitemap file provides search engines with a list of URLs available for crawling.
 * This class encapsulates the essential attributes of a Sitemap entry, allowing for efficient management
 * and processing of Sitemap data.
 * </p>
 *
 * <p>

 * </ul>
 *
 * <p>Passwords are matched against URLs or resource names using regular expression patterns.
 * The extractor first tries to match against the URL, then falls back to the resource name if available.
 *
 * @author shinsuke
 */
public abstract class PasswordBasedExtractor extends AbstractExtractor {

    /** Logger instance for this class. */

 * When multiple extractors are associated with a single key, they are sorted by weight
 * in descending order. The {@link #getExtractor(String)} method returns a composite
 * extractor that iterates through the available extractors until one successfully
 * extracts the data.
 * </p>
 *
 * <p>
 * The class uses a {@link CrawlerContainer} for managing crawler components and

 * It uses a {@link UrlFilterService} to manage the URL filtering rules.
 * The class supports caching of include and exclude patterns for scenarios where a session ID is not available.
 * It also provides methods to initialize the filter with a session ID, clear the filter,
 * match a URL against the defined patterns, and process a URL to add include or exclude patterns based on predefined filtering patterns.
 *

     * @param tcCount The thread check count.
     * @return true if the crawling should continue, false otherwise.
     */
    protected boolean isContinue(final int tcCount) {
        if (!crawlerContainer.available()) {
            // system shutdown
            return false;
        }

        boolean isContinue = false;
        if (tcCount < crawlerContext.maxThreadCheckCount) {

      not limited to compiled object code, generated documentation,
      and conversions to other media types.

      "Work" shall mean the work of authorship, whether in Source or
      Object form, made available under the License, as indicated by a
      copyright notice that is included in or attached to the work
      (an example is provided in the Appendix below).

     * This method manages local caches and fetches from OpenSearch when needed.
     *
     * @param sessionId The session ID.
     * @return The next URL queue entry, or null if none available.
     */
    @Override
    public OpenSearchUrlQueue poll(final String sessionId) {
        final QueueHolder queueHolder = getQueueHolder(sessionId);

      not limited to compiled object code, generated documentation,
      and conversions to other media types.

      "Work" shall mean the work of authorship, whether in Source or
      Object form, made available under the License, as indicated by a
      copyright notice that is included in or attached to the work
      (an example is provided in the Appendix below).

                final ExtractData extractData = new ExtractData(content);
                final long contentLength;
                if (isByteStream) {
                    contentLength = ((ByteArrayInputStream) inputStream).available();
                } else {
                    contentLength = tempFile != null ? tempFile.length() : 0;
                }
                extractData.putValue("Content-Length", Long.toString(contentLength));