## Overview

**Fess Crawler** is a powerful, flexible Java-based web crawling framework designed for enterprise-scale content extraction and processing. Built with a modular architecture, it supports multiple protocols (HTTP/HTTPS, File System, FTP, SMB, Cloud Storage) and provides extensive content extraction capabilities from various document formats.

### Key Features

  <mime-type type="application/macwriteii"/>

  <mime-type type="application/marc">
    <!-- todo add marc xml <marc:collection> -->
    <glob pattern="*.mrc"/>
    <magic priority="50">
      <!-- built from, e.g. https://www.loc.gov/marc/community/cileader.html -->
      <match value="[0-9]{5,5}" type="regex" offset="0">
        <match value="45" type="string" offset="20">
          <!-- bibliographic -->