}

    /**
     * StorageURLConnection is a URL connection implementation for accessing storage objects.
     * It extends URLConnection to provide connectivity to MinIO-compatible storage services.
     * This class handles the authentication, connection management, and data retrieval
     * from storage buckets and objects.
     *
     * <p>

│   ├── client/               # Protocol clients (HTTP, FTP, SMB, etc.)
│   ├── extractor/           # Content extractors
│   ├── transformer/         # Data transformers
│   └── service/             # Core services
├── fess-crawler-lasta/       # LastaFlute DI integration
└── fess-crawler-opensearch/  # OpenSearch backend
```

### Key Components

#### Core Engine

 * </p>
 *
 * <p>
 * The class uses {@link CrawlerContainer} to access components like {@link AccessResult}
 * and {@link UrlQueue}. It also uses {@link CrawlingParameterUtil} to access services
 * like {@link UrlQueueService} and DataService, as well as the {@link CrawlerContext}.
 * </p>
 *
 * <p>
 * The class provides methods to check if a response is successful or not modified based on

 *
 * <p>It implements the Runnable interface to be executed in a separate thread,
 * and the AutoCloseable interface to ensure resources are properly released after use.
 *
 * <p>The crawler uses several services and components, such as UrlQueueService, DataService,
 * UrlFilter, RuleManager, CrawlerContainer, IntervalController, and CrawlerClientFactory,
 * to perform its tasks.
 *

 * processing the response, and extracting child URLs.
 *
 * <p>
 * This class implements the {@link Runnable} interface, allowing it to be executed in a separate thread.
 * It uses various services and components, such as {@link UrlQueueService}, {@link DataService},
 * {@link CrawlerContainer}, {@link LogHelper}, {@link CrawlerClientFactory}, and {@link CrawlerContext},
 * to perform its tasks.
 * </p>
 *
 * <p>

import org.opensearch.search.sort.SortBuilder;

import com.google.common.hash.HashFunction;
import com.google.common.hash.Hashing;

import jakarta.annotation.Resource;

/**
 * Abstract base class for crawler services that interact with OpenSearch.
 *
 * @author shinsuke
 *
 */
public abstract class AbstractCrawlerService {
    /**
     * Creates a new instance of AbstractCrawlerService.
     */