e.getMessage().contains("Failed to extract") || e.getMessage().contains("No entries could be processed"));
        }
    }

    /**
     * Test that TarExtractor handles invalid archive gracefully.
     * Invalid archives may either throw an exception or return empty content.
     */
    public void test_TarExtractor_invalidArchive_handlesGracefully() {

        final long elapsed = (System.nanoTime() - start) / 1000000;

        assertTrue("file:// URLs should not be delayed", elapsed < 50);
    }

    /**
     * Test that null URL queue is handled gracefully
     */
    public void test_nullUrlQueue() {
        final HostIntervalController controller = new HostIntervalController();
        controller.delayMillisBeforeProcessing = 1000L;

# Test file for malformed robots.txt parsing
# This file contains various malformed directives that should be handled gracefully

# Case 1: Directives before any User-agent (should be ignored)
Disallow: /orphaned1/
Allow: /orphaned2/

# Case 2: Valid user-agent with various malformed directives
User-agent: GoodBot
Disallow: /admin/
InvalidDirective: some-value
unknown-field: test
Disallow /missing-colon

- ZipExtractor
- TarExtractor

**Test Count**: 12 tests

**Key Scenarios**:
- ✅ Descriptive error messages for invalid archives
- ✅ Continues processing when some entries fail
- ✅ Handles empty archives gracefully
- ✅ Mixed valid/invalid entries processed correctly
- ✅ Null input stream validation

---

### 4. AbstractExtractorTest.java
**Purpose**: Test the base AbstractExtractor functionality.

**Key Test Areas**:

void addUrl(String url)         // Add URL to queue
void cleanup(String sessionId)  // Clean up session
void stop()                     // Stop gracefully
```

**Key Fields**: `crawlerContext`, `urlFilter`, `intervalController`, `clientFactory`, `ruleManager`

### CrawlerContext (`CrawlerContext.java`)

Execution context and configuration.

**Important Fields**:
```java

     */
    public void test_destroyObject_nullPooledObject() {
        try {
            factory.destroyObject(null);
            // Should not throw NPE - implementation should handle gracefully or throw
        } catch (NullPointerException e) {
            // Acceptable behavior
            assertTrue(true);
        } catch (Exception e) {
            // Other exceptions might be acceptable

         */
        public boolean isFinished() {
            return finished;
        }

        /**
         * Stops the crawling operation gracefully.
         * If a data store is currently active, this method calls
         * its stop method to halt the crawling process.
         */
        public void stopCrawling() {
            if (dataStore != null) {

        assertTrue(sitemaps.length >= 3); // At least the valid ones should be parsed

        // Test NumericBot - various crawl-delay formats
        // Should handle edge cases gracefully
        assertTrue(robotsTxt.getCrawlDelay("NumericBot") >= 0);

        // Test TabBot - tab characters should be treated as whitespace
        assertFalse(robotsTxt.allows("/tab1/", "TabBot"));

        assertEquals(4, attemptCount.get());
    }

    /**
     * Test: Null safety in connect() when targetIndices is null
     * Verifies that connect() handles null targetIndices gracefully
     */
    @Test
    public void testConnectWithNullTargetIndices() {
        final AdminClient mockAdminClient = mock(AdminClient.class);

     * It will parse valid directives and ignore invalid ones, ensuring that partial
     * content can be extracted even from poorly formatted files.</p>
     *
     * <p>The following errors are handled gracefully (line is skipped, parsing continues):</p>
     * <ul>
     * <li>Invalid directive formats</li>
     * <li>Unknown directives</li>
     * <li>Invalid crawl-delay values (non-numeric, negative)</li>