*
 * <p>This class supports both GET and POST methods for token and login
 * requests. It also allows for the replacement of placeholders in URLs and
 * parameters with actual credentials.
 *
 * <p>Usage example:
 * <pre>
 * {@code
 * Map<String, String> params = new HashMap<>();
 * params.put("token_url", "http://example.com/token");
 * params.put("login_url", "http://example.com/login");

 * </ol>
 *
 * <p>The crawler can be configured with various parameters, such as the number of threads,
 * the maximum depth of crawling, and URL filters.
 *
 * <p>Example usage:
 * <pre>
 *   Crawler crawler = new Crawler();
 *   crawler.addUrl("http://example.com/");
 *   crawler.execute();
 *   crawler.close();
 * </pre>
 */
public class Crawler implements Runnable, AutoCloseable {

 * {@link MimeTypeHelper}, to handle content length checks and MIME type detection.
 * </p>
 *
 * <p>
 * The class uses JCIFS properties to configure the SMB connection.
 * </p>
 *
 * <p>
 * Usage example:
 * </p>
 *
 * <pre>
 * {@code
 * SmbClient smbClient = new SmbClient();
 * smbClient.init();
 * ResponseData responseData = smbClient.doGet("smb://example.com/share/file.txt");

 *       stream to determine the character set encoding.</li>
 *   <li><b>invalidUrlPattern:</b> A regular expression pattern used to identify
 *       invalid URLs.</li>
 * </ul>
 *
 * <p>
 * <b>Usage:</b>
 * </p>
 * <p>
 * The {@code transform} method is the main entry point for transforming an HTML
 * response. It takes a {@link ResponseData} object as input and returns a

  <mime-type type="application/vnd.oma.dd2+xml">
    <glob pattern="*.dd2"/>
  </mime-type>
  <mime-type type="application/vnd.oma.drm.risd+xml"/>
  <mime-type type="application/vnd.oma.group-usage-list+xml"/>
  <mime-type type="application/vnd.oma.poc.detailed-progress-report+xml"/>
  <mime-type type="application/vnd.oma.poc.final-report+xml"/>
  <mime-type type="application/vnd.oma.poc.groups+xml"/>