/**
 * A factory class for managing and creating crawler clients based on URL patterns.
 * This class implements AutoCloseable to properly handle resource cleanup.
 *
 * <p>The factory maintains a map of regular expression patterns to crawler clients,
 * allowing for URL-based client selection. Clients can be added with specific patterns
 * and optionally at specific positions in the processing order.</p>
 *

        private final int priorityLength;

        /**
         * Constructs a new PathPattern from the given robots.txt path pattern.
         * @param pattern the path pattern string from robots.txt (may contain * and $)
         */
        public PathPattern(final String pattern) {
            this.pattern = pattern;

                .getTotalHits()
                .value() > 0);

        // Verify pattern can be retrieved
        final List<Pattern> patterns = urlFilterService.getIncludeUrlPatternList(sessionId);
        assertEquals(1, patterns.size());
        assertTrue(patterns.get(0).matcher("http://example.com/page1").matches());
        assertFalse(patterns.get(0).matcher("http://other.com/page1").matches());

    }

    /**
     * Converts a GSA URL pattern into a regular expression pattern suitable for Fess.
     * Handles various GSA pattern formats including regexp, contains, and URL-based patterns.
     *
     * @param s the input GSA pattern string
     * @return a regular expression pattern string, or empty string for comments/invalid patterns
     */
    protected String getFilterPath(final String s) {

        Pattern[] patterns = (Pattern[]) propMap.get(CRAWLER_METADATA_CONTENT_EXCLUDES);
        if (patterns == null) {
            patterns = split(getCrawlerMetadataContentExcludes(), ",")
                    .get(stream -> stream.filter(StringUtil::isNotBlank).map(Pattern::compile).toArray(n -> new Pattern[n]));
            propMap.put(CRAWLER_METADATA_CONTENT_EXCLUDES, patterns);
        }

     * <p>
     * Patterns can be exact class names or use wildcards with '*' at the end.
     * For example: "com.example.*" allows all classes in the com.example package.
     * </p>
     *
     * @param allowedPatterns the patterns of classes to allow
     * @return an ObjectInputFilter configured with the specified patterns
     */

**Flow**: Poll URL → Validate → Get client → Delay → Check last-modified → Execute → Process → Extract children → Queue children → Delay

### CrawlerClientFactory

Pattern-based client selection using `LinkedHashMap<Pattern, CrawlerClient>`.

**Standard Patterns**:
```java
"^https?://.*"     → httpClient
"^file:.*"         → fileSystemClient
"^ftp://.*"        → ftpClient
"^smb://.*"        → smbClient

     * - Second: List of regex pattern matches (Pattern -> content template)
     */
    protected Map<String, Pair<Map<String, String>, List<Pair<Pattern, String>>>> relatedContentMap = Collections.emptyMap();

    /**
     * Prefix used to identify regex patterns in related content terms.
     * When a term starts with this prefix, it is treated as a regular expression
     * pattern rather than an exact match term.
     */

        }

        final String lowerEndpoint = endpoint.toLowerCase(Locale.ROOT);

        // GCS patterns
        if (lowerEndpoint.contains("storage.googleapis.com") || lowerEndpoint.contains(".storage.cloud.google.com")) {
            return StorageType.GCS;
        }

        // S3 patterns
        if (lowerEndpoint.contains(".amazonaws.com") || lowerEndpoint.contains("s3.") || lowerEndpoint.contains("s3-")) {

├── normalizer/                 # Text normalizers
├── converter/                  # Reading converters (katakana, romaji)
├── concurrent/                 # Async patterns (Deferred/Promise)
└── util/                      # Utilities
```

### Key Design Patterns

- **Builder**: SuggesterBuilder, SuggestRequestBuilder
- **Facade**: Suggester (main entry point)
- **Composite**: NormalizerChain, ReadingConverterChain