- Sort Score
- Result 10 results
- Languages All
Results 1 - 1 of 1 for robot (0.01 sec)
-
fess-crawler-opensearch/src/test/java/org/codelibs/fess/crawler/util/CrawlerWebServer.java
buf.append("User-agent: *").append('\n'); buf.append("Disallow: /admin/").append('\n'); buf.append("Disallow: /websvn/").append('\n'); final File robotTxtFile = new File(tempDir, "robots.txt"); FileUtil.writeBytes(robotTxtFile.getAbsolutePath(), buf.toString().getBytes("UTF-8")); robotTxtFile.deleteOnExit(); generateContents(tempDir, count);
Registered: Sun Nov 10 03:50:12 UTC 2024 - Last Modified: Thu Nov 07 04:44:10 UTC 2024 - 5K bytes - Viewed (0)