* Sitemap were changed. The value for the lastmod tag should be in W3C
     * Datetime format.
     *
     * By providing the last modification timestamp, you enable search engine
     * crawlers to retrieve only a subset of the Sitemaps in the index i.e. a
     * crawler may only retrieve Sitemaps that were modified since a certain

    controller.setDelayMillisForWaitingNewUrl(5000);
    controller.setDefaultIntervalTime(1000);
});
```

### Sitemap Support

```java
// Enable sitemap processing
container.singleton("sitemapsRule", SitemapsRule.class, rule -> {
    rule.addRule("url", ".*sitemap.*");
});

// Add sitemap URL
crawler.addUrl("https://example.com/sitemap.xml");
```

        this.fieldRuleMap = fieldRuleMap;
    }

    /**
     * Returns whether space trimming is enabled.
     * @return True if space trimming is enabled, false otherwise.
     */
    public boolean isTrimSpace() {
        return trimSpaceEnabled;
    }

    /**
     * Sets whether space trimming is enabled.
     * @param trimSpace The trim space flag to set.
     */

     * Checks if robots.txt processing is enabled.
     * @return true if enabled, false otherwise
     */
    public boolean isEnabled() {
        return enabled;
    }

    /**
     * Sets whether robots.txt processing is enabled.
     * @param enabled true to enable, false to disable
     */
    public void setEnabled(final boolean enabled) {
        this.enabled = enabled;
    }

    /**
     * Checks if automatic UTF-8 encoding detection is enabled.
     *
     * @return True if auto-detection is enabled, false otherwise
     */
    public boolean isAutodetectEncoding() {
        return autodetectEncoding;
    }

    /**
     * Sets whether automatic UTF-8 encoding detection is enabled.
     *
     * @param autodetectEncoding True to enable auto-detection, false to disable
     */

- Review OpenSearch cluster performance

**Memory Usage**
- Configure appropriate JVM heap settings
- Monitor index sizes and optimize mappings
- Use streaming for large data imports

### Debug Logging

Enable debug logging for detailed troubleshooting:

```java
// Add to your logging configuration
logger.debug.org.codelibs.fess.suggest=DEBUG
```

## Contributing

1. Fork the repository

 *   <li>USER_AGENT_PROPERTY: User agent string.</li>
 *   <li>ROBOTS_TXT_ENABLED_PROPERTY: Enable or disable robots.txt parsing.</li>
 *   <li>AUTHENTICATIONS_PROPERTY: Web authentications.</li>
 *   <li>REQUEST_HEADERS_PROPERTY: Custom request headers.</li>
 *   <li>REDIRECTS_ENABLED: Enable or disable HTTP redirects.</li>
 *   <li>COOKIES_PROPERTY: Cookie settings.</li>

                offset += term.length();
            }
        }

        return kanaBuf.toString();
    }

    /**
     * Checks if the tokenizer is enabled.
     * @param factory The tokenizer factory.
     * @return True if the tokenizer is enabled, false otherwise.
     */
    protected boolean isEnableTokenizer(final TokenizerFactory factory) {
        // TODO return factory instanceof JapaneseTokenizerFactory;

	</build>
	<repositories>
		<repository>
			<id>snapshots.central.sonatype.com</id>
			<url>https://central.sonatype.com/repository/maven-snapshots</url>
			<releases>
				<enabled>false</enabled>
			</releases>
			<snapshots>
				<enabled>true</enabled>
			</snapshots>
		</repository>
	</repositories>

	</build>
	<repositories>
		<repository>
			<id>snapshots.central.sonatype.com</id>
			<url>https://central.sonatype.com/repository/maven-snapshots</url>
			<releases>
				<enabled>false</enabled>
			</releases>
			<snapshots>
				<enabled>true</enabled>
			</snapshots>
		</repository>
	</repositories>
	<dependencies>
		<dependency>
			<groupId>com.ibm.icu</groupId>
			<artifactId>icu4j</artifactId>