├─ MimeTypeException
  ├─ MultipleCrawlingAccessException
  ├─ RobotsTxtException
  └─ SitemapsException
```

### Thread-Local Storage

Use `CrawlingParameterUtil` to set/get `CrawlerContext` and `UrlQueue` in worker threads. Always clear in finally block with `CrawlingParameterUtil.clearAll()`.

### Resource Cleanup Pattern

Always use try-with-resources for `ResponseData` - temp files are auto-deleted on close.

        final Map<String, String> xpathConfigMap = crawlingConfig.getConfigParameterMap(ConfigName.XPATH);

        String urlEncoding;
        final UrlQueue<?> urlQueue = CrawlingParameterUtil.getUrlQueue();
        if (urlQueue != null && urlQueue.getEncoding() != null) {
            urlEncoding = urlQueue.getEncoding();
        } else {
            urlEncoding = responseData.getCharSet();
        }

        // cid

        }

        @Override
        public boolean visited(OpenSearchUrlQueue urlQueue) {
            return false;
        }

        @Override
        public void generateUrlQueues(String sessionId, String url) {
            // Do nothing in test
        }

        @Override
        public void insert(OpenSearchUrlQueue urlQueue) {
            // Do nothing in test
        }

        @Override