- Sort Score
- Result 10 results
- Languages All
Results 1 - 1 of 1 for webmaster (0.08 sec)
-
fess-crawler/src/main/java/org/codelibs/fess/crawler/helper/RobotsTxtHelper.java
import org.codelibs.fess.crawler.exception.RobotsTxtException; /** * Robots.txt Specifications: * <ul> * <li><a href= * "https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt" * >https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt * </a></li> * </ul> * * @author bowez * @author shinsuke * */ public class RobotsTxtHelper {
Registered: Wed Jun 12 15:17:51 UTC 2024 - Last Modified: Thu Feb 22 01:36:27 UTC 2024 - 6.1K bytes - Viewed (0)