* <ol>
 *   <li>Initialization: Sets up the crawler context and initializes the URL filter.</li>
 *   <li>Thread Creation: Creates a group of crawler threads to perform the actual crawling.</li>
 *   <li>Execution: Starts the crawler threads and waits for them to complete.</li>
 *   <li>Cleanup: Deletes the crawled data and clears the URL filter.</li>
 * </ol>
 *

 * {@link CrawlerContainer}, {@link LogHelper}, {@link CrawlerClientFactory}, and {@link CrawlerContext},
 * to perform its tasks.
 * </p>
 *
 * <p>
 * The crawling process involves the following steps:
 * </p>
 * <ol>
 *   <li>Fetching a URL from the queue using {@link UrlQueueService#poll(String)}.</li>
 *   <li>Checking if the URL is valid using {@link #isValid(UrlQueue)}.</li>

 * <p>
 * The transformation process involves:
 * </p>
 * <ol>
 *   <li>Determining the character set encoding of the HTML content.</li>
 *   <li>Storing the HTML content as data in the {@link ResultData}.</li>
 *   <li>Extracting child URLs from the HTML content based on configured rules.</li>
 *   <li>Handling redirect URLs specified in the response headers.</li>
 * </ol>
 * <p>