Search Options

Results per page
Sort
Preferred Languages
Advance

Results 1 - 1 of 1 for RobotsTxt (0.08 sec)

  2. github.com/codelibs/fess-crawler

    fess-crawler-lasta/src/main/resources/crawler/client.xml 

    <!DOCTYPE components PUBLIC "-//DBFLUTE//DTD LastaDi 1.0//EN"
	"http://dbflute.org/meta/lastadi10.dtd">
<components namespace="fessCrawler">
	<include path="crawler/container.xml" />
	<include path="crawler/robotstxt.xml" />
	<include path="crawler/contentlength.xml" />
	<include path="crawler/mimetype.xml" />

	<component name="internalHttpClient" class="org.codelibs.fess.crawler.client.http.HcHttpClient"
		instance="prototype">
    Registered: Sat Dec 20 11:21:39 UTC 2025
    - Last Modified: Thu Dec 11 08:38:29 UTC 2025
    - 3.3K bytes
    - Viewed (0)
Back to top