- **RobotsTxtHelper**: RFC 9309 parsing, user-agent matching, crawl-delay, sitemaps
- **SitemapsHelper**: Sitemap XML parsing, index handling
- **MimeTypeHelper**: MIME detection via Tika
- **EncodingHelper**: Charset detection with BOM
- **UrlConvertHelper**: URL normalization
- **ContentLengthHelper**: Content length limits per MIME type

---

## Development Workflow

### Build Commands

```bash

                    <dependencyManagement>
                      <dependencies>
                        <dependency>
                          <groupId>org.junit</groupId>
                          <artifactId>junit-bom</artifactId>
                          <version>5.11.1</version>
                          <type>pom</type>
                          <scope>import</scope>
                        </dependency>

                }
            }
        }

        if (trackingFile == null) {
            return; // parent or imported bom ?
        }
        try {
            Files.createDirectories(trackingDir);

            trackingData.add("");
            if (!missing) {
                if (event.getRepository() != null) {

    host
  }
}

/** Returns a [Locale.US] formatted [String]. */
internal fun format(
  format: String,
  vararg args: Any,
): String = String.format(Locale.US, format, *args)

/**
 * will also strip BOM from the source
 */
@Throws(IOException::class)
internal fun BufferedSource.readBomAsCharset(default: Charset): Charset =
  when (select(UNICODE_BOMS)) {

            logger.debug("Skipping child URL: url={}", u);
        }
    }

    /**
     * Checks if the byte array contains UTF-8 BOM (Byte Order Mark).
     *
     * @param b the byte array to check
     * @return true if the bytes represent UTF-8 BOM, false otherwise
     */
    private boolean isUtf8BomBytes(final byte[] b) {
        return b[0] == (byte) 0xEF && b[1] == (byte) 0xBB && b[2] == (byte) 0xBF;

    literals are now Huffman-encoded.
 *  New: Expose `Part` headers and body in `Multipart`.
 *  New: Make `ResponseBody.string()` and `ResponseBody.charStream()` BOM-aware. If your HTTP
    response body begins with a [byte order mark][bom] it will be consumed and used to select a
    charset for the remaining bytes. Most applications should not need a byte order mark.

 *  New: Upgrade to Okio 1.11.0.

     ```xml

    <developer>
      <id>cpovirk</id>
      <name>Chris Povirk</name>
      <email>******@****.***</email>
    </developer>
  </developers>
  <modules>
    <module>guava</module>
    <module>guava-bom</module>
    <module>guava-testlib</module>
    <module>guava-tests</module>
  </modules>
  <scm>
    <connection>scm:git:https://github.com/google/guava.git</connection>

**Implantar** uma aplicação significa executar as etapas necessárias para torná-la **disponível para os usuários**.

Para uma **web API**, isso normalmente envolve colocá-la em uma **máquina remota**, com um **programa de servidor** que ofereça bom desempenho, estabilidade, etc., de modo que seus **usuários** possam **acessar** a aplicação com eficiência e sem interrupções ou problemas.

    <maven.compiler.target>21</maven.compiler.target>
  </properties>

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>com.squareup.okhttp3</groupId>
        <artifactId>okhttp-bom</artifactId>
        <version>[5.2.0-SNAPSHOT,6.0.0-SNAPSHOT)</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <dependencies>

    public Collection<DefaultType> types() {
        return Arrays.asList(
                // Maven types
                new DefaultType(Type.POM, Language.NONE, "pom", null, false),
                new DefaultType(Type.BOM, Language.NONE, "pom", null, false),
                new DefaultType(Type.MAVEN_PLUGIN, Language.JAVA_FAMILY, "jar", null, false, JavaPathType.CLASSES),
                // Java types
                new DefaultType(