*   <li>Processing the response using a {@link ResponseProcessor} associated with a {@link Rule}.</li>
 *   <li>Extracting child URLs and adding them to the queue using {@link #storeChildUrls(Set, String, int)}
 *       or {@link #storeChildUrl(String, String, float, int)}.</li>
 *   <li>Handling exceptions that may occur during the crawling process.</li>
 * </ol>
 *
 * <p>