The MIT License

Copyright (c) 2014 Benedikt Lang <github at benediktlang.de>

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

= vendor/github.com/cespare/xxhash/v2 licensed under: =

Copyright (c) 2016 Caleb Spare

MIT License

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to

Guava has mostly not used the CONTRIBUTORS file, instead relying on Git
history. You can see a summary of contributions at
https://github.com/google/guava/graphs/contributors. However, Git history
over-counts some people's contributions because they were responsible for
mirroring out changes from our internal repo.

This files serves mainly to credit people who have not received proper credit
in the Git history.

Doug Lea, author of some concurrency libraries

above, provided that you also meet all of these conditions:

    a) The modified work must itself be a software library.

    b) You must cause the files modified to carry prominent notices
    stating that you changed the files and the date of any change.

    c) You must cause the whole of the work to be licensed at no
    charge to all third parties under the terms of this License.

      (d) If the Work includes a "NOTICE" text file as part of its
          distribution, then any Derivative Works that You distribute must
          include a readable copy of the attribution notices contained
          within such NOTICE file, excluding those notices that do not
          pertain to any part of the Derivative Works, in at least one
          of the following places: within a NOTICE text file distributed

import static org.codelibs.core.stream.StreamUtil.split;

import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.Proxy;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;

        try {
            final int maxCount = 50;
            final int numOfThread = 10;

            final File file = File.createTempFile("crawler-", "");
            file.delete();
            file.mkdirs();
            file.deleteOnExit();
            fileTransformer.setPath(file.getAbsolutePath());
            crawler.addUrl(url);
            crawler.getCrawlerContext().setMaxAccessCount(maxCount);

 * It extends CrawlerSystemException and provides functionality to set and check the log level for the exception.
 *
 * <p>
 * This exception can be thrown when there are problems accessing URLs, files, or any other resources needed for crawling.
 * It includes constructors to handle messages, causes, or both.
 * </p>
 *
 * <p>

/**
 * Filter interface for SMB file filtering.
 *
 * This interface allows selective filtering of files
 * when listing directory contents in SMB shares.
 */
public interface SmbFileFilter {

    /**
     * Tests whether the specified SMB file should be included in a file list.
     *
     * @param file the SMB file to test for inclusion
     * @return whether the given file should be included

Properties props = PropertiesUtil.load(resource);

// File operations with proper resource handling
try (InputStream input = ResourceUtil.getResourceAsStream("data.txt")) {
    String content = InputStreamUtil.getUTF8String(input);
}

// Resource traversal for processing multiple files
ResourceTraversalUtil.forEach("META-INF", (resource, is) -> {