*/
    protected void processRobotsTxt(final String url) {
        if (StringUtil.isBlank(url)) {
            throw new CrawlerSystemException("HTTP URL is null or empty. Cannot process robots.txt without a valid URL.");
        }

        if (robotsTxtHelper == null || !robotsTxtHelper.isEnabled()) {
            // not support robots.txt
            return;
        }

        // crawler context

Para isso, nós atualizamos o modelo Pydantic `TokenData` com a nova propriedade `scopes`.

Ao validar os dados com o Pydantic nós podemos garantir que temos, por exemplo, exatamente uma `list` de `str` com os escopos e uma `str` com o `username`.

        lenient().when(response.getOutputStream()).thenReturn(servletOutputStream);
    }

    @Test
    void testInit_success() throws ServletException {
        // Test successful initialization with valid configuration
        Map<String, String> initParams = new HashMap<>();
        initParams.put("jcifs.smb.client.domain", "TEST_DOMAIN");
        initParams.put("jcifs.http.domainController", "dc.test.com");

        }
    }

    @Nested
    @DisplayName("Read Bytes Wire Format Tests")
    class ReadBytesWireFormatTests {

        @Test
        @DisplayName("Should read valid structure size of 4")
        void testReadBytesWireFormatValid() throws SMBProtocolDecodingException {
            byte[] buffer = new byte[1024];
            int bufferIndex = 100;

```JSON
{
  "detail": [
    {
      "type": "int_parsing",
      "loc": [
        "path",
        "item_id"
      ],
      "msg": "Input should be a valid integer, unable to parse string as an integer",
      "input": "foo"
    }
  ]
}
```

Çünkü burada `item_id` yol parametresi `int` tipinde bir değer beklerken `"foo"` yani `string` tipinde bir değer almıştı.

```JSON
{
  "detail": [
    {
      "type": "int_parsing",
      "loc": [
        "path",
        "item_id"
      ],
      "msg": "Input should be a valid integer, unable to parse string as an integer",
      "input": "foo"
    }
  ]
}
```
тому що параметр шляху має значення `"foo"`, яке не є типом `int`.

    }

    private void executeCommand(final File inputFile, final File outputFile) {

        if (StringUtil.isBlank(command)) {
            throw new CrawlerSystemException("External command is empty. Please configure a valid command for CommandExtractor.");
        }

        final Map<String, String> params = new HashMap<>();
        params.put("$INPUT_FILE", inputFile.getAbsolutePath());

        buffer[bufferIndex] = reservedValue;
        buffer[bufferIndex + 1] = reservedValue;
        buffer[bufferIndex + 2] = reservedValue;

        // Set valid data for other fields
        SMBUtil.writeInt4(100, buffer, bufferIndex + 3);
        for (int i = 7; i < 35; i++) {
            buffer[bufferIndex + i] = 0x00;
        }
        buffer[bufferIndex + 35] = 0x00;

            + "exposed by the superclass implementation. If the implementation has "
            + "changed, the code must be re-inspected to ensure that the "
            + "assumption is still valid.",
        24,
        ReentrantReadWriteLock.class.getMethods().length);
  }

  private enum MyOrder {
    FIRST,
    SECOND,
    THIRD;
  }

  private enum OtherOrder {
    FIRST,

```JSON
{
  "detail": [
    {
      "type": "int_parsing",
      "loc": [
        "path",
        "item_id"
      ],
      "msg": "Input should be a valid integer, unable to parse string as an integer",
      "input": "foo"
    }
  ]
}
```

porque el parámetro de path `item_id` tenía un valor de `"foo"`, que no es un `int`.