Search Options

Display Count
Sort
Preferred Language
Advanced Search

Results 1 - 10 of 164 for Crawler (0.05 seconds)

  2. github.com/square/okhttp

    samples/crawler/src/main/java/okhttp3/sample/Crawler.java 

        OkHttpClient client = new OkHttpClient.Builder()
        .cache(cache)
        .callTimeout(5, TimeUnit.SECONDS)
        .build();

    Crawler crawler = new Crawler(client, queueLimit, hostLimit);
    crawler.queue.add(HttpUrl.get(args[1]));
    crawler.parallelDrainQueue(threadCount);
  }
    Created: Fri Apr 03 11:42:14 GMT 2026
    - Last Modified: Wed Jul 23 00:58:06 GMT 2025
    - 5K bytes
    - Click Count (0)
  3. github.com/codelibs/fess-crawler

    fess-crawler-lasta/src/main/resources/crawler.xml 

    <components namespace="fessCrawler">
	<include path="crawler/container.xml"/>
	<include path="crawler/client.xml"/>
	<include path="crawler/rule.xml"/>
	<include path="crawler/filter.xml"/>
	<include path="crawler/interval.xml"/>
	<include path="crawler/extractor.xml"/>
	<include path="crawler/mimetype.xml"/>
	<include path="crawler/encoding.xml"/>
	<include path="crawler/urlconverter.xml"/>
	<include path="crawler/log.xml"/>
    Created: Sun Apr 12 03:50:13 GMT 2026
    - Last Modified: Tue Nov 28 13:40:25 GMT 2017
    - 1.7K bytes
    - Click Count (0)
  4. github.com/codelibs/fess

    src/main/resources/mail/crawler.dfmail 

    /*
 [Crawler Notification]
 Crawler notification mail.
*/
subject: [FESS] Crawler completed: /*pmb.hostname*/
>>>
--- Server Info ---
Host Name: /*pmb.hostname:orElse('Unknown')*/
Job Name: /*pmb.jobname:orElse('Unknown')*/

--- Web/FileSystem Crawler ---
Start Time: /*pmb.webFsCrawlStartTime:orElse('-')*/
End Time:   /*pmb.webFsCrawlEndTime:orElse('-')*/
Exec Time:  /*pmb.webFsCrawlExecTime:orElse('-')*/ ms

--- Web/FileSystem Indexer ---
    Created: Tue Mar 31 13:07:34 GMT 2026
    - Last Modified: Wed Jan 15 22:05:20 GMT 2020
    - 1K bytes
    - Click Count (0)
  5. github.com/codelibs/fess-crawler

    fess-crawler-lasta/src/test/java/org/codelibs/fess/crawler/container/LastaCrawlerContainerTest.java 

            // Verify that multiple components can be retrieved
        final Crawler crawler1 = crawlerContainer.getComponent("crawler");
        final Crawler crawler2 = crawlerContainer.getComponent("crawler");

        // They should be different instances (prototype scope)
        assertNotNull(crawler1);
        assertNotNull(crawler2);
    }

    @Test
    public void test_isLastaCrawlerContainer() {
    Created: Sun Apr 12 03:50:13 GMT 2026
    - Last Modified: Fri Jan 16 13:52:11 GMT 2026
    - 2.7K bytes
    - Click Count (0)
  6. github.com/codelibs/fess

    src/main/resources/fess_indices/fess_config.scheduled_job/scheduled_job.bulk 

    {"index":{"_index":"fess_config.scheduled_job","_id":"default_crawler"}}
{"name":"Default Crawler","target":"all","cronExpression":"0 0 * * *","scriptType":"groovy","scriptData":"return container.getComponent(\"crawlJob\").logLevel(\"info\").gcLogging().execute(executor);","jobLogging":true,"crawler":true,"available":true,"sortOrder":1,"createdBy":"system","createdTime":0,"updatedBy":"system","updatedTime":0}
{"index":{"_index":"fess_config.scheduled_job","_id":"suggest_indexer"}}
    Created: Tue Mar 31 13:07:34 GMT 2026
    - Last Modified: Thu Mar 26 02:24:08 GMT 2026
    - 5K bytes
    - Click Count (0)
  7. github.com/codelibs/fess-crawler

    fess-crawler-opensearch/src/test/java/org/codelibs/fess/crawler/CrawlerTest.java 

                crawler1.getCrawlerContext().setMaxAccessCount(maxCount);
            crawler1.getCrawlerContext().setNumOfThread(numOfThread);

            final Crawler crawler2 = getComponent(Crawler.class);
            crawler2.setBackground(true);
            ((UrlFilterImpl) crawler2.urlFilter).setIncludeFilteringPattern("$1$2$3.*");
            crawler2.addUrl(url2);
            crawler2.getCrawlerContext().setMaxAccessCount(maxCount);
    Created: Sun Apr 12 03:50:13 GMT 2026
    - Last Modified: Thu Jan 15 01:11:43 GMT 2026
    - 7.8K bytes
    - Click Count (0)
  8. github.com/codelibs/fess-crawler

    CLAUDE.md 

    **DI Config**: `fess-crawler-lasta/src/main/resources/`
- `crawler.xml` (root), `crawler/client.xml`, `crawler/extractor.xml`, `crawler/rule.xml`, `crawler/transformer.xml`, `crawler/transformer_basic.xml`
- `crawler/mimetype.xml`, `crawler/encoding.xml`, `crawler/robotstxt.xml`, `crawler/sitemaps.xml`
    Created: Sun Apr 12 03:50:13 GMT 2026
    - Last Modified: Thu Mar 12 03:39:20 GMT 2026
    - 8.1K bytes
    - Click Count (0)
  9. github.com/codelibs/fess

    src/test/resources/plugin/repo1/index.html 

    <a href="fess-crawler-es/" title="fess-crawler-es/">fess-crawler-es/</a>                                                 -         -      
<a href="fess-crawler-lasta/" title="fess-crawler-lasta/">fess-crawler-lasta/</a>                                              -         -
    Created: Tue Mar 31 13:07:34 GMT 2026
    - Last Modified: Tue Aug 13 07:34:14 GMT 2019
    - 3.3K bytes
    - Click Count (0)
  10. github.com/codelibs/fess-crawler

    fess-crawler-opensearch/src/main/resources/crawler_opensearch.xml 

    <components namespace="fessCrawler">
    <include path="crawler/container.xml"/>
    <include path="crawler/client.xml"/>
    <include path="crawler/rule.xml"/>
    <include path="crawler/filter.xml"/>
    <include path="crawler/interval.xml"/>
    <include path="crawler/extractor.xml"/>
    <include path="crawler/mimetype.xml"/>
    <include path="crawler/encoding.xml"/>
    <include path="crawler/urlconverter.xml"/>
    <include path="crawler/log.xml"/>
    Created: Sun Apr 12 03:50:13 GMT 2026
    - Last Modified: Thu Nov 07 04:44:10 GMT 2024
    - 2.2K bytes
    - Click Count (0)
  11. github.com/codelibs/fess

    src/test/resources/plugin/repo3/index.html 

    <a href="fess-crawler-es/" title="fess-crawler-es/">fess-crawler-es/</a>                                                 -         -      
<a href="fess-crawler-lasta/" title="fess-crawler-lasta/">fess-crawler-lasta/</a>                                              -         -
    Created: Tue Mar 31 13:07:34 GMT 2026
    - Last Modified: Mon Jun 17 13:30:41 GMT 2024
    - 6.2K bytes
    - Click Count (0)
Back to Top