# Extractor Implementation Tests

This directory contains comprehensive tests for the Extractor implementations, focusing on the improvements made to resource management, error handling, and input validation.

## Test Files Overview

### 1. ExtractorResourceManagementTest.java
**Purpose**: Verify proper resource management in Extractor implementations.

**Key Test Areas**:
- Resource closure on successful extraction (MS Office extractors)

## Overview

**Fess Crawler** is a powerful, flexible Java-based web crawling framework designed for enterprise-scale content extraction and processing. Built with a modular architecture, it supports multiple protocols (HTTP/HTTPS, File System, FTP, SMB, Cloud Storage) and provides extensive content extraction capabilities from various document formats.

### Key Features

            // Test file name extraction
            assertEquals("file.txt", new SmbFile("smb1://server/share/file.txt").getName());
            // Test directory name extraction (should include trailing slash)
            assertEquals("dir/", new SmbFile("smb1://server/share/dir/").getName());
            // Test share name extraction
            assertEquals("share/", new SmbFile("smb1://server/share/").getName());

    public void test_getText_nested() {
        final InputStream in = ResourceUtil.getResourceAsStream("extractor/json/test.json");
        final ExtractData extractData = jsonExtractor.getText(in, null);
        CloseableUtil.closeQuietly(in);

        final String content = extractData.getContent();

        // Verify nested content extraction
        assertTrue(content.contains("content.summary"));

/**
 * Extracts text content and metadata from CSV files.
 * This extractor provides better structured data extraction compared to Tika's generic text extraction.
 *
 * <p>Features:
 * <ul>
 *   <li>Automatic delimiter detection (comma, tab, semicolon, pipe)</li>
 *   <li>Header row detection and extraction</li>
 *   <li>Column name to data value association</li>
 *   <li>Quoted field handling</li>

---
title: Sample Markdown Document
author: John Doe
date: 2025-01-15
tags:
  - crawler
  - extractor
  - markdown
---

# Introduction

This is a sample Markdown document for testing the MarkdownExtractor.

## Features

The extractor should handle:

- YAML front matter extraction
- Heading structure
- **Bold text** and *italic text*
- Lists and other formatting

### Code Examples

 *   <li>Handling resource names and content types</li>
 *   <li>Retrying extraction without resource name or content type if the initial attempt fails</li>
 *   <li>Extracting text from metadata if the main content extraction fails</li>
 *   <li>Reading content as plain text if all other methods fail</li>
 *   <li>Applying post-extraction filters</li>
 *   <li>Handling Tika exceptions, including zip bomb exceptions</li>
 * </ul>
 *

#

[![GoDoc](https://godoc.org/github.com/bcicen/jstream?status.svg)](https://godoc.org/github.com/bcicen/jstream)


`jstream` is a streaming JSON parser and value extraction library for Go.

Unlike most JSON parsers, `jstream` is document position- and depth-aware -- this enables the extraction of values at a specified depth, eliminating the overhead of allocating encompassing arrays or objects; e.g:

Using the below example document:

 * governing permissions and limitations under the License.
 */
package org.codelibs.fess.crawler.exception;

/**
 * UnsupportedExtractException is thrown when the content extraction is not supported.
 * It extends ExtractException and indicates that the requested extraction operation cannot be performed.
 *
 */
public class UnsupportedExtractException extends ExtractException {

    private static final long serialVersionUID = 1L;

        }
        return URI.create(currentUrl);
    }

    /**
     * Gets child URL extraction rules from configuration.
     *
     * @param responseData the response data from crawling
     * @param resultData the result data
     * @return stream of tag-attribute pairs for URL extraction
     */
    @Override