}
        return getSupportedLanguage(result.getLanguage());
    }

    /**
     * Returns the text to be used for language detection.
     *
     * @param text The original text.
     * @return The text for language detection.
     */
    protected String getDetectText(final String text) {
        final String result;
        if (text.length() <= maxTextLength) {

```

### Helpers

**RobotsTxtHelper**: RFC 9309 parsing, user-agent matching, crawl-delay, sitemaps
**SitemapsHelper**: Sitemap XML parsing, index handling
**MimeTypeHelper**: MIME detection via Tika
**EncodingHelper**: Charset detection with BOM
**UrlConvertHelper**: URL normalization

---

## Development Workflow

### Build Commands

```bash
mvn clean install              # Build all

 * This extractor provides better structured data extraction compared to Tika's generic text extraction.
 *
 * <p>Features:
 * <ul>
 *   <li>Automatic delimiter detection (comma, tab, semicolon, pipe)</li>
 *   <li>Header row detection and extraction</li>
 *   <li>Column name to data value association</li>
 *   <li>Quoted field handling</li>
 *   <li>Column names as metadata</li>

                u = StringUtil.EMPTY;
            }
        }
        return u;
    }

    /**
     * Decodes a URL as a name using appropriate character encoding.
     * Handles encoding detection from parent URLs and configuration settings.
     *
     * @param url the URL to decode
     * @param escapePlus whether to escape plus signs before decoding

    }

    /**
     * Gets the preload size for charset detection.
     *
     * @return the preload size in bytes
     */
    public int getPreloadSizeForCharset() {
        return preloadSizeForCharset;
    }

    /**
     * Sets the preload size for charset detection.
     *
     * @param preloadSizeForCharset the preload size in bytes to set
     */

 * It also integrates with other Fess Crawler components, such as {@link ContentLengthHelper} and
 * {@link MimeTypeHelper}, to handle content length checks and MIME type detection.
 * </p>
 *
 * <p>
 * The class uses JCIFS properties to configure the SMB connection.
 * </p>
 *
 * <p>
 * Usage example:
 * </p>
 *
 * <pre>
 * {@code

 *
 * <p>Features:
 * <ul>
 *   <li>Automatic initialization of GCS client</li>
 *   <li>Support for HEAD and GET operations</li>
 *   <li>Content length validation</li>
 *   <li>MIME type detection</li>
 *   <li>Handling of large files through temporary file storage</li>
 *   <li>Object metadata retrieval</li>
 *   <li>Directory listing capabilities</li>
 * </ul>
 *

    }

    /**
     * Checks if the given URL is a file path protocol that requires directory and permission handling.
     * Used for incremental crawling directory detection and file permission processing.
     *
     * @param url the URL to check
     * @return true if the URL uses a file path protocol (smb, smb1, file, ftp, s3, gcs)
     */

 *
 * <p>Features:
 * <ul>
 *   <li>Automatic initialization of AWS S3 client</li>
 *   <li>Support for HEAD and GET operations</li>
 *   <li>Content length validation</li>
 *   <li>MIME type detection</li>
 *   <li>Handling of large files through temporary file storage</li>
 *   <li>Object metadata and tags retrieval</li>
 *   <li>Directory listing capabilities</li>
 * </ul>
 *

 * It also integrates with other Fess Crawler components, such as {@link ContentLengthHelper} and
 * {@link MimeTypeHelper}, to handle content length checks and MIME type detection.
 * </p>
 *
 * <p>
 * The class uses JCIFS properties to configure the SMB connection.
 * </p>
 *
 * <p>
 * Usage example:
 * </p>
 *
 * <pre>
 * {@code