* The queue for URLs waiting to be crawled.
         */
        protected Queue<OpenSearchUrlQueue> waitingQueue = new ConcurrentLinkedQueue<>();

        /**
         * The queue for URLs currently being crawled.
         */
        protected Queue<OpenSearchUrlQueue> crawlingQueue = new ConcurrentLinkedQueue<>();
    }

    /**
     * Sets the polling fetch size.

     *
     * @param <T> the type of the component
     */
    protected static class ComponentHolder<T> {
        /**
         * The component instance being held.
         */
        protected T instance;

        /**
         * The destroyer function to be called when the component is destroyed.
         */
        protected Consumer<T> destroyer;

     * This method handles different file types (files, directories, symbolic links)
     * and populates the response data accordingly.
     *
     * @param uri The original URI being accessed
     * @param includeContent Whether to include the actual content in the response
     * @param responseData The response data to update
     * @param client The FTP client used for the operation