tempDir.delete();
            tempDir.mkdirs();

            // robots.txt
            StringBuilder buf = new StringBuilder();
            buf.append("User-agent: *").append('\n');
            buf.append("Disallow: /admin/").append('\n');
            buf.append("Disallow: /websvn/").append('\n');
            final File robotTxtFile = new File(tempDir, "robots.txt");

///

## 自動変換 { #automatic-conversion }

`Header`は`Path`や`Query`、`Cookie`が提供する機能に加え、少しだけ追加の機能を持っています。

ほとんどの標準ヘッダーは、「マイナス記号」（`-`）としても知られる「ハイフン」文字で区切られています。

しかし、`user-agent`のような変数はPythonでは無効です。

そのため、デフォルトでは、`Header`はパラメータ名の文字をアンダースコア（`_`）からハイフン（`-`）に変換して、ヘッダーを抽出して文書化します。

また、HTTPヘッダは大文字小文字を区別しないので、Pythonの標準スタイル（別名「スネークケース」）で宣言することができます。

`TikaExtractor`, `PdfExtractor`, `MsWordExtractor`, `MsExcelExtractor`, `MsPowerPointExtractor`, `ZipExtractor`, `HtmlExtractor`, `MarkdownExtractor`, `EmlExtractor`

### Helpers

- **RobotsTxtHelper**: RFC 9309 parsing, user-agent matching, crawl-delay, sitemaps
- **SitemapsHelper**: Sitemap XML parsing, index handling
- **MimeTypeHelper**: MIME detection via Tika
- **EncodingHelper**: Charset detection with BOM
- **UrlConvertHelper**: URL normalization

- `aws:username` - This is a string containing the friendly name of the current user, this value would point to STS temporary credential in `AssumeRole`ed requests, use `jwt:preferred_username` in case of OpenID connect and `ldap:username`...

  ':libs:elasticsearch-secure-sm',
  ':libs:elasticsearch-ssl-config',
  ':libs:elasticsearch-x-content',
  ':modules:analysis-common',
  ':modules:ingest-common',
  ':modules:ingest-geoip',
  ':modules:ingest-user-agent',
  ':modules:lang-expression',
  ':modules:lang-mustache',
  ':modules:lang-painless',
  ':modules:lang-painless:spi',
  ':modules:mapper-extras',
  ':modules:parent-join',
  ':modules:percolator',

mvn test -Dtest=SuggesterTest           # Run specific test
mvn package                              # Package JAR
mvn formatter:format license:format     # Format code and apply licenses
mvn clean jacoco:prepare-agent test jacoco:report  # Generate coverage report
```

### Adding New Features

1. Read related source files and tests
2. Write implementation following existing patterns
3. Add comprehensive tests

            if (userAgent == null) {
                Properties headers = new Properties();

                headers.put(
                        "User-Agent",
                        ConfigUtils.getString(
                                legacySupport.getRepositorySession(), "Maven", ConfigurationProperties.USER_AGENT));
                try {

            <option value="$PROJECT_DIR$/platforms/core-runtime/installation-beacon" />
            <option value="$PROJECT_DIR$/platforms/core-runtime/instrumentation-agent" />
            <option value="$PROJECT_DIR$/platforms/core-runtime/instrumentation-agent-services" />
            <option value="$PROJECT_DIR$/platforms/core-runtime/instrumentation-declarations" />

labels.regularName=Regular Name
labels.replacement=Replacement
labels.sessionId=Session ID
labels.sortOrder=Sort Order
labels.updatedBy=Updated By
labels.updatedTime=Updated Time
labels.urls=URLs
labels.userAgent=User Agent
labels.username=Username
labels.value=Value
labels.versionNo=Version No
labels.cronExpression=Schedule
labels.dayForCleanup=Delete documents older than specified days
labels.crawlingThreadCount=Concurrent Crawling Count

> Host: localhost:9000
> User-Agent: curl/7.81.0
> Accept: */*
>