## How the configuration works

We use Kotlin portable DSL to store TeamCity configuration, which means you can easily create a new pipeline
based on a specific branch. Currently, we have two pipelines: `master` and `release`, but you can easily create
and test another isolated pipeline from any branch. 

We'll explain everything via an example. Let's say you make some changes on your branch `myTestBranch`

            public static final String HTML_CANONICAL_XPATH = "html.canonical.xpath";
            public static final String HTML_PRUNED_TAGS = "html.pruned.tags";
            public static final String PIPELINE = "pipeline";
            public static final String IGNORE_ROBOTS_TAGS = "ignore.robots.tags";
            public static final String SCRIPT_TYPE = "script.type";

        }
        queryCommandMap.put(name, queryCommand);
    }

    /**
     * Adds a filter to the processing pipeline.
     * After adding a filter, the filter chain is recreated to include the new filter.
     *
     * @param filter the filter to add to the processing pipeline
     */
    public void addFilter(final Filter filter) {
        filterList.add(filter);
        createFilterChain();
    }

Example:      my-suggest.20250123120000

Aliases:
├── Search Alias: {baseIndex}          (read operations)
└── Update Alias: {baseIndex}.update   (write operations)
```

### Text Processing Pipeline

```
Input Text
    ↓
[Normalization] - NormalizerChain
    ↓
[Reading Conversion] - ReadingConverterChain
    ↓
[Analysis] - SuggestAnalyzer
```

    init {
        id("Promotion_AllBranchesStartReleaseCycleTest")
        name = "Start Release Cycle Test"
        description = "Test for Start Release Cycle pipeline"

        steps {
            gradleWrapper {
                name = "PromoteTest"
                tasks = "clean promoteStartReleaseCycle"
                useGradleWrapper = true
                gradleParams =

#!groovy

pipeline {
  agent none
  // save some io during the build
  options {
    skipDefaultCheckout()
    durabilityHint('PERFORMANCE_OPTIMIZED')
    disableRestartFromStage()
  }
  stages {
    stage("Build / Test - JDK17") {
      agent { node { label 'ubuntu' } }
      steps {
        timeout(time: 210, unit: 'MINUTES') {
          checkout scm
          mavenBuild("jdk_17_latest", "")
          script {

 *
 * Usage: `mvn compile exec:java@update-test-buckets -DinputTestClassDataJson=/path/to/test-class-data.json`.
 * You can get the JSON file as an artifacts of the "autoUpdateTestSplitJsonOnGradleMaster" pipeline in TeamCity.
 */
fun main() {
    val model =
        CIBuildModel(
            projectId = "Check",
            branch = VersionedSettingsBranch("master"),
            buildScanTags = listOf("Check"),

- **DataService**: Access result persistence, iteration
- Implementations: `UrlQueueServiceImpl`, `DataServiceImpl` (in-memory), `OpenSearchDataService` (persistent)

### Processing Pipeline

```
CrawlerThread → Client → ResponseProcessor → Transformer → Extractor → ExtractData
                                                                            ↓

import org.elasticsearch.search.aggregations.MultiBucketConsumerService;
import org.elasticsearch.search.aggregations.bucket.terms.StringTerms;
import org.elasticsearch.search.aggregations.pipeline.PipelineAggregator;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.search.query.QuerySearchResult;
import org.openjdk.jmh.annotations.Benchmark;

                "-PmaxParallelForks=%maxParallelForks%",
                "-Dorg.gradle.internal.plugins.portal.url.override=%gradle.plugins.portal.url%",
                "-Dscan.value.tcPipeline=master",
                "-s",
                "%additional.gradle.parameters%",
                "--continue",