## Overview

**Fess Crawler** is a powerful, flexible Java-based web crawling framework designed for enterprise-scale content extraction and processing. Built with a modular architecture, it supports multiple protocols (HTTP/HTTPS, File System, FTP, SMB, Cloud Storage) and provides extensive content extraction capabilities from various document formats.

### Key Features

            // Test file name extraction
            assertEquals("file.txt", new SmbFile("smb1://server/share/file.txt").getName());
            // Test directory name extraction (should include trailing slash)
            assertEquals("dir/", new SmbFile("smb1://server/share/dir/").getName());
            // Test share name extraction
            assertEquals("share/", new SmbFile("smb1://server/share/").getName());

#

[![GoDoc](https://godoc.org/github.com/bcicen/jstream?status.svg)](https://godoc.org/github.com/bcicen/jstream)


`jstream` is a streaming JSON parser and value extraction library for Go.

Unlike most JSON parsers, `jstream` is document position- and depth-aware -- this enables the extraction of values at a specified depth, eliminating the overhead of allocating encompassing arrays or objects; e.g:

Using the below example document:

**Fess Crawler** is a Java-based web crawling framework for enterprise content extraction.

### Essential Info

- **Language**: Java 21+
- **Build**: Maven 3.x
- **License**: Apache 2.0
- **DI**: LastaFlute DI
- **Repo**: https://github.com/codelibs/fess-crawler

### Tech Stack

- **HTTP**: Apache HttpComponents 4.5+ and 5.x (switchable)
- **Extraction**: Apache Tika, POI, PDFBox

        }
        return new URL(currentUrl);
    }

    /**
     * Gets child URL extraction rules from configuration.
     *
     * @param responseData the response data from crawling
     * @param resultData the result data
     * @return stream of tag-attribute pairs for URL extraction
     */
    @Override

import jakarta.annotation.PostConstruct;

/**
 * File transformer implementation for the Fess search engine.
 * This transformer handles file-based document transformation and content extraction
 * using the Fess file transformation process with support for various file types.
 *
 * <p>It extends AbstractFessFileTransformer to provide specialized file processing

import org.codelibs.fess.util.ComponentUtil;

import jakarta.annotation.PostConstruct;

/**
 * Standard transformer implementation for the Fess search engine.
 * This transformer handles document transformation and content extraction using
 * the standard Fess file transformation process with support for various content types.
 *
 * <p>It extends AbstractFessFileTransformer to provide file-specific transformation

    private static final Pattern ALLOWED_WRAPPER_VERSION =
        Pattern.compile("^[0-9.]+(-(rc|milestone|m)-[0-9]+)?$");

    // Keep the same extraction semantics as the old sed:
    //   sed 's/.*gradle-\(.*\)-[a-z]*\.[a-z]*/\1/'
    private static final Pattern WRAPPER_VERSION_EXTRACT =
        Pattern.compile(".*gradle-(.*)-[a-z]*\\.[a-z]*");

/**
 * Tests Artifact extraction, computation, and parent resolution functionality.
 */
@DisplayName("GAVUtils")
class GAVUtilsTest {

    @BeforeEach
    void setUp() {}

    private UpgradeContext createMockContext() {
        return TestUtils.createMockContext();
    }

    @Nested
    @DisplayName("Artifact Extraction")
    class GAVExtractionTests {

        @Test

import org.codelibs.fess.helper.PluginHelper.ArtifactType;
import org.codelibs.fess.util.ResourceUtil;

/**
 * Helper class for managing theme installation and uninstallation.
 * Handles the extraction and deployment of theme files from JAR artifacts.
 */
public class ThemeHelper {
    private static final Logger logger = LogManager.getLogger(ThemeHelper.class);

    /**
     * Default constructor for ThemeHelper.