final List<SearchLog> searchLogList = new ArrayList<>();
        final Map<String, UserInfo> userInfoMap = new HashMap<>();
        while (!queue.isEmpty()) {
            final SearchLog searchLog = queue.poll();
            if (searchLog != null) {
                final String userAgent = searchLog.getUserAgent();
                final boolean isBot =

int maxDepth = -1               // Max depth (-1 = unlimited)
long maxAccessCount = 0         // Max URLs (0 = unlimited)
```

### CrawlerThread (`CrawlerThread.java`)

Worker thread for crawling.

**Flow**: Poll URL → Validate → Get client → Delay → Check last-modified → Execute → Process → Extract children → Queue children → Delay

### CrawlerClientFactory

Pattern-based client selection using `LinkedHashMap<Pattern, CrawlerClient>`.

            requestQueue.offer(new CrawlRequest(url, 0));
            while (!requestQueue.isEmpty() && (maxAccessCount < 0 || counter < maxAccessCount)) {
                final CrawlRequest crawlRequest = requestQueue.poll();
                if ((maxDepth != -1 && crawlRequest.getDepth() > maxDepth) || processedUrls.contains(crawlRequest.getUrl())) {
                    if (logger.isDebugEnabled()) {

            final List<Tuple3<String, String, String>> taskList = new ArrayList<>();
            while (generating) {
                try {
                    final Tuple3<String, String, String> task = thumbnailTaskQueue.poll(thumbnailTaskQueueTimeout, TimeUnit.MILLISECONDS);
                    if (task == null) {
                        if (!taskList.isEmpty()) {
                            storeQueue(taskList);
                        }