*   <li>{@link #refresh()}: Refreshes the suggestion indices.</li>
 *   <li>{@link #shutdown()}: Shuts down the thread pool.</li>
 *   <li>{@link #createIndexIfNothing()}: Creates a new index if no index exists.</li>
 *   <li>{@link #createNextIndex()}: Creates a new index and replaces the current update alias with the new index.</li>
 *   <li>{@link #switchIndex()}: Switches the search alias to the current update index.</li>

            for (;;) {
                try {
                    final String line = br.readLine();
                    if (line == null) {
                        break;
                    }
                    if (logger.isDebugEnabled()) {
                        logger.debug(line);
                    }
                    list.add(line);
                    if (list.size() > maxLineBuffer) {

 *       </ul>
 *   </li>
 * </ul>
 *
 * <p>Fields:
 * <ul>
 *   <li>{@link #client}: The client used to perform analysis.</li>
 *   <li>{@link #settings}: The settings for suggestions.</li>
 *   <li>{@link #analyzerSettings}: The settings for the analyzer.</li>
 *   <li>{@link #transliterator}: The transliterator used to convert between Hiragana and Katakana.</li>
 * </ul>
 */

 *   <li>{@link #elevateWord()} - Returns an instance of ElevateWordSettings.</li>
 * </ul>
 *
 * <p>Timeout settings can be accessed via:</p>
 * <ul>
 *   <li>{@link #getScrollTimeout()}</li>
 *   <li>{@link #getSearchTimeout()}</li>
 *   <li>{@link #getIndexTimeout()}</li>
 *   <li>{@link #getIndicesTimeout()}</li>
 *   <li>{@link #getBulkTimeout()}</li>
 *   <li>{@link #getClusterTimeout()}</li>
 * </ul>
 *

 * </p>
 *
 * <p>
 * The class uses {@link CrawlerContainer} to access components like {@link AccessResult}
 * and {@link UrlQueue}. It also uses {@link CrawlingParameterUtil} to access services
 * like {@link UrlQueueService} and DataService, as well as the {@link CrawlerContext}.
 * </p>
 *
 * <p>

 * <li>{@link #loadIndexSettings()}: Loads the index settings from a JSON file.</li>
 * </ul>
 *
 * <p>Fields:
 * <ul>
 * <li>{@link #logger}: Logger instance for logging debug information.</li>
 * <li>{@link #client}: OpenSearch client instance.</li>
 * <li>{@link #arraySettingsIndexName}: Name of the array settings index.</li>
 * <li>{@link #settingsId}: ID of the settings.</li>

        try {
            final BufferedReader br = new BufferedReader(new InputStreamReader(in, Constants.UTF_8));
            String line;
            while ((line = br.readLine()) != null) {
                final String url = line.trim();
                if (StringUtil.isNotBlank(url) && (url.startsWith("http://") || url.startsWith("https://"))) {

 *   <li>PDF Parser configuration for PDF documents</li>
 * </ul>
 *
 * <p>
 * The {@link TikaDetectParser} inner class extends {@link CompositeParser} to provide auto-detection of the MIME type
 * of the document. It also handles zip bomb prevention and embedded document extraction.
 * </p>
 *
 * <p>
 * The {@link ContentWriter} functional interface is used to abstract the process of writing content to a writer.
 * </p>
 *

				"application/vnd.commonspace",
				"application/vnd.contact.cmsg",
				"application/vnd.cosmocaller",
				"application/vnd.crick.clicker",
				"application/vnd.crick.clicker.keyboard",
				"application/vnd.crick.clicker.palette",
				"application/vnd.crick.clicker.template",
				"application/vnd.crick.clicker.wordbank",
				"application/vnd.criticaltools.wbs+xml",
				"application/vnd.ctc-posml",
				"application/vnd.ctct.ws+xml",

    }

    /**
     * Strips comments from a line (everything after '#' character).
     * @param line the line to strip comments from
     * @return the line without comments
     */
    protected String stripComment(final String line) {
        final int commentIndex = line.indexOf('#');
        if (commentIndex != -1) {
            return line.substring(0, commentIndex);
        }
        return line;
    }