- **Fault Tolerance**: Built-in retry mechanisms and error handling
- **Flexible Configuration**: XML-based dependency injection with LastaFlute DI
- **Extensible Architecture**: Plugin system for custom extractors, transformers, and clients
- **Rate Limiting**: Politeness policies and interval controllers
- **URL Filtering**: Regex-based inclusion/exclusion patterns
- **Data Persistence**: Multiple backend options including OpenSearch integration

 * in a large index without loading all of them into memory at once.
 * </p>
 *
 * <p>
 * The reader supports limiting the number of documents read based on a percentage of the total documents
 * or a fixed number. It also allows filtering documents based on their size, using the {@code limitOfDocumentSize}
 * parameter.
 * </p>
 *
 * <p>

 *   <li>Appending alphanumeric characters (0-9, A-Z, a-z) to the buffer.</li>
 *   <li>Appending symbol characters (!-/, :-@, [-`, {-~) to the buffer.</li>
 *   <li>Optionally removing duplicate terms based on a flag.</li>
 *   <li>Limiting the maximum size of alphanumeric and symbol terms.</li>
 * </ul>
 *
 * <p>The {@link TextNormalizeContext} class provides a fluent API to configure the text

 * <li>{@link #addToArrayIndex(String, String, String, Map)}: Adds a map to the array index.</li>
 * <li>{@link #deleteKeyFromArray(String, String, String)}: Deletes all entries associated with the specified key from the array index.</li>
 * <li>{@link #deleteFromArray(String, String, String)}: Deletes a specific entry from the array index based on the ID.</li>

import jakarta.annotation.Resource;

/**
 * <p>
 * {@link DefaultResponseProcessor} is a default implementation of {@link ResponseProcessor}.
 * It processes the response data based on the HTTP status code and configured transformer.
 * </p>
 *
 * <p>
 * It handles successful responses by transforming the data using a {@link Transformer}

        private MinioClient minioClient;
        /** The name of the storage bucket */
        private String bucketName;
        /** The name of the storage object */
        private String objectName;
        /** Cached object statistics response */
        private StatObjectResponse statObject;

        /**
         * Constructs a new StorageURLConnection for the specified URL.

        String text = "test text";
        String field = "content";

        List<String> results = converter.convert(text, field);

        assertNotNull(results);
        // Results may vary based on analyzer configuration
    }

    @Test
    public void testConvertWithNullLanguages() throws IOException {
        // Test convert with null languages
        String text = "test text";

        protected StandardCrawlerContainer container;

        /**
         * The component instance.
         */
        private T instance;

        /**
         * Creates a new ComponentDef for a class-based component.
         * @param cls the class of the component
         * @param initializer the initializer function (can be null)
         * @param container the container instance for dependency injection
         */

 * </p>
 *
 * <p>
 * The conversion process involves iterating through the input string, identifying Katakana characters,
 * and replacing them with their corresponding Alphabet representations based on the internal mapping.
 * When a Katakana character has multiple Alphabet representations, the converter generates multiple
 * possible readings.
 * </p>
 *
 * <p>

            throw e;
        } catch (final Exception e) {
            throw new CrawlingAccessException("Could not parse Sitemaps: " + preloadDate, e);
        }
    }

    /**
     * Parses a text-based sitemap from the given input stream.
     * @param in the input stream to parse
     * @return the parsed sitemap set
     */
    protected SitemapSet parseTextSitemaps(final InputStream in) {