## Caveat { #caveat }

If you need to perform heavy background computation and you don't necessarily need it to be run by the same process (for example, you don't need to share memory, variables, etc), you might benefit from using other bigger tools like <a href="https://docs.celeryq.dev" class="external-link" target="_blank">Celery</a>.

### Protocols

- **HTTP/HTTPS**: Full crawling, cookies, auth, robots.txt
- **File**: Local/network file systems
- **FTP**: With authentication
- **SMB/CIFS**: Windows shares (SMB1/SMB2+)
- **Storage**: MinIO/S3-compatible

### Content Formats

Office (Word, Excel, PowerPoint), PDF, Archives (ZIP, TAR, GZ), HTML, XML, JSON, Media (audio/video metadata), Images (EXIF/IPTC/XMP)

---

    @Test
    public void testLeaseManager() {
        CIFSContext context = new BaseContext(new PropertyConfiguration());
        LeaseManager manager = new LeaseManager(context);
        
        String path = "/share/file.txt";
        int requestedState = Smb2LeaseState.SMB2_LEASE_FULL;
        
        Smb2LeaseKey key = manager.requestLease(path, requestedState);
        assertNotNull(key);
        

    synchronized void insert(final String path, final DfsReferral dr) {
        int s1, s2;
        String server, share, key;

        if (DISABLED) {
            return;
        }

        s1 = path.indexOf('\\', 1);
        s2 = path.indexOf('\\', s1 + 1);
        server = path.substring(1, s1);
        share = path.substring(s1 + 1, s2);

        key = path.substring(0, dr.pathConsumed).toLowerCase();

    }

    @Test
    void testToStringWithDifferentPaths() {
        // Test with various path formats
        String[] testPaths = { "simple.txt", "path/to/file.doc", "//server/share/file.txt", "folder\\windows\\style.txt", "" // empty path
        };

        for (String path : testPaths) {
            trans2QueryPathInfo = new Trans2QueryPathInformation(config, path, TEST_INFO_LEVEL);

    /**
     * SMB command to close a find operation.
     */
    public static final byte SMB_COM_FIND_CLOSE2 = (byte) 0x34;

    /**
     * SMB command to disconnect from a tree share.
     */
    public static final byte SMB_COM_TREE_DISCONNECT = (byte) 0x71;

    /**
     * SMB command to negotiate protocol dialect.
     */
    public static final byte SMB_COM_NEGOTIATE = (byte) 0x72;

      "control" means (i) the power, direct or indirect, to cause the
      direction or management of such entity, whether by contract or
      otherwise, or (ii) ownership of fifty percent (50%) or more of the
      outstanding shares, or (iii) beneficial ownership of such entity.

      "You" (or "Your") shall mean an individual or Legal Entity
      exercising permissions granted by this License.

        assertEquals(0x40000000, SmbConstants.GENERIC_WRITE);
        assertEquals(0x80000000, SmbConstants.GENERIC_READ);
    }

    @Test
    @DisplayName("Should define share access constants")
    void testShareAccessConstants() {
        assertEquals(0x00, SmbConstants.FILE_NO_SHARE);
        assertEquals(0x01, SmbConstants.FILE_SHARE_READ);

power, direct or indirect, to cause the direction or management of such 
entity, whether by contract or otherwise, or (ii) ownership of fifty percent 
(50%) or more of the outstanding shares, or (iii) beneficial ownership of 
such entity. 

"You" (or "Your") shall mean an individual or Legal Entity exercising 
permissions granted by this License. 

      "control" means (i) the power, direct or indirect, to cause the
      direction or management of such entity, whether by contract or
      otherwise, or (ii) ownership of fifty percent (50%) or more of the
      outstanding shares, or (iii) beneficial ownership of such entity.

      "You" (or "Your") shall mean an individual or Legal Entity
      exercising permissions granted by this License.