* is sent unencrypted to the proxy server, so tunnels include only the minimum set of headers.
   * This avoids sending potentially sensitive data like HTTP cookies to the proxy unencrypted.
   *
   * In order to support preemptive authentication we pass a fake "Auth Failed" response to the
   * authenticator. This gives the authenticator the option to customize the CONNECT request. It can

* *경로* 혹은 *쿼리* 매개변수를 전달하려면, URL 자체에 추가한다.
* JSON 본문을 전달하려면, 파이썬 객체 (예를들면 `dict`) 를 `json` 파라미터로 전달한다.
* JSON 대신 *폼 데이터* 를 보내야한다면, `data` 파라미터를 대신 전달한다.
* *헤더* 를 전달하려면, `headers` 파라미터에 `dict` 를 전달한다.
* *쿠키* 를 전달하려면, `cookies` 파라미터에 `dict` 를 전달한다.

백엔드로 데이터를 어떻게 보내는지 정보를 더 얻으려면 (`httpx` 혹은 `TestClient` 를 이용해서) <a href="https://www.python-httpx.org" class="external-link" target="_blank">HTTPX documentation</a> 를 확인하세요.

/// info | 정보

          recoveredFailures += e
          newRoutePlanner = false
          continue
        }

        // Clear out downstream interceptor's additional request headers, cookies, etc.
        response =
          response
            .newBuilder()
            .request(request)
            .priorResponse(priorResponse?.stripBody())
            .build()

        this.remaining = remaining;
        this.b = b;
        this.off = off;
        count = len;
        digest = null; /* otherwise recycled commands
                        * like writeandx will choke if session
                        * closes in between */
    }

    @Override
    int writeParameterWordsWireFormat(final byte[] dst, int dstIndex) {
        final int start = dstIndex;

        this.remaining = remaining;
        this.b = b;
        this.off = off;
        dataLength = len;
        digest = null; /* otherwise recycled commands
                        * like writeandx will choke if session
                        * closes in between */
    }

    @Override
    int getBatchLimit(final byte command) {
        if (command == SMB_COM_READ_ANDX) {
            return READ_ANDX_BATCH_LIMIT;

        this.b = b;
        this.off = off;
        this.count = len;
        this.digest = null; /*
                             * otherwise recycled commands
                             * like writeandx will choke if session
                             * closes in between
                             */
    }

    @Override
    protected int writeParameterWordsWireFormat(final byte[] dst, int dstIndex) {

crawler.urlFilter.addExclude(".*\\.js$");
crawler.urlFilter.addExclude(".*login.*");
```

## Supported Protocols and Formats

### Protocols
- **HTTP/HTTPS**: Full web crawling support with cookies, authentication, redirects
- **File System**: Local and network file system access
- **FTP**: FTP server crawling with authentication
- **SMB/CIFS**: Windows network shares

                }
                if (pcr[pc2[j + 24]] != 0) {
                    kn[n] |= bigbyte[j];
                }
            }
        }
        cookey(kn, KnL);
    }

    private void cookey(final int[] raw, final int KnL[]) {
        int raw0, raw1;
        int rawi, KnLi;
        int i;

        for (i = 0, rawi = 0, KnLi = 0; i < 16; ++i) {
            raw0 = raw[rawi];

        this.b = b;
        this.off = off;
        this.dataLength = len;
        this.digest = null; /*
                             * otherwise recycled commands
                             * like writeandx will choke if session
                             * closes in between
                             */
    }

    /**
     * Sets the write mode flags for this request.
     *
     * @param writeMode

      apply {
        this.followSslRedirects = followProtocolRedirects
      }

    /**
     * Sets the handler that can accept cookies from incoming HTTP responses and provides cookies to
     * outgoing HTTP requests.
     *
     * If unset, [no cookies][CookieJar.NO_COOKIES] will be accepted nor provided.
     */
    fun cookieJar(cookieJar: CookieJar) =
      apply {
        this.cookieJar = cookieJar