the original version of the Work and any modifications or additions
      to that Work or Derivative Works thereof, that is intentionally
      submitted to Licensor for inclusion in the Work by the copyright owner
      or by an individual or Legal Entity authorized to submit on behalf of
      the copyright owner. For the purposes of this definition, "submitted"

      the original version of the Work and any modifications or additions
      to that Work or Derivative Works thereof, that is intentionally
      submitted to Licensor for inclusion in the Work by the copyright owner
      or by an individual or Legal Entity authorized to submit on behalf of
      the copyright owner. For the purposes of this definition, "submitted"

      the original version of the Work and any modifications or additions
      to that Work or Derivative Works thereof, that is intentionally
      submitted to Licensor for inclusion in the Work by the copyright owner
      or by an individual or Legal Entity authorized to submit on behalf of
      the copyright owner. For the purposes of this definition, "submitted"

      the original version of the Work and any modifications or additions
      to that Work or Derivative Works thereof, that is intentionally
      submitted to Licensor for inclusion in the Work by the copyright owner
      or by an individual or Legal Entity authorized to submit on behalf of
      the copyright owner. For the purposes of this definition, "submitted"

      the original version of the Work and any modifications or additions
      to that Work or Derivative Works thereof, that is intentionally
      submitted to Licensor for inclusion in the Work by the copyright owner
      or by an individual or Legal Entity authorized to submit on behalf of
      the copyright owner. For the purposes of this definition, "submitted"

      the original version of the Work and any modifications or additions
      to that Work or Derivative Works thereof, that is intentionally
      submitted to Licensor for inclusion in the Work by the copyright owner
      or by an individual or Legal Entity authorized to submit on behalf of
      the copyright owner. For the purposes of this definition, "submitted"

- **Extensible Architecture**: Plugin system for custom extractors, transformers, and clients
- **Rate Limiting**: Politeness policies and interval controllers
- **URL Filtering**: Regex-based inclusion/exclusion patterns
- **Data Persistence**: Multiple backend options including OpenSearch integration

## Technology Stack

- **Java**: 21+ (requires Java 21 or higher)
- **Build System**: Maven 3.x

        this.skipMaskingForDebugLevel = enable;
        log.info("Skip masking for debug level {}", enable ? "enabled" : "disabled");
    }

    /**
     * Enable or disable stack trace inclusion
     *
     * @param enable true to include stack traces
     */
    public void setIncludeStackTrace(boolean enable) {
        this.includeStackTrace = enable;
    }

    /**

			<artifactId>poi-ooxml</artifactId>
			<version>${poi.version}</version>
			<exclusions>
				<exclusion>
					<groupId>stax</groupId>
					<artifactId>stax-api</artifactId>
				</exclusion>
				<exclusion>
					<groupId>xml-apis</groupId>
					<artifactId>xml-apis</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<dependency>
			<groupId>org.apache.poi</groupId>

        }
    }

    /**
     * Determines whether the specified URL is crawlable based on the exclusion pattern
     * provided in the {@code paramMap}. If the {@code URL_EXCLUDE_PATTERN} key exists in
     * the parameter map, its value is used as a regular expression pattern to match against
     * the given URL. If the URL matches the exclusion pattern, the method returns {@code false},