Crawl-delay: 1.23e10

# Case 14: Tab characters instead of spaces
User-agent:	TabBot
Disallow:	/tab1/
Allow:	/tab2/

# Case 15: Unicode and special characters in user-agent
User-agent: Bot™
Disallow: /trademark/

User-agent: Bot®
Disallow: /registered/

# Case 16: Multiple wildcards user agent
User-agent: *
Disallow: /default/

# Case 17: Empty file handling (just comments and whitespace after this)

        assertTrue(robotsTxt.allows("/tab2/", "TabBot"));

        // Test bots with special characters - should be normalized to lowercase
        assertFalse(robotsTxt.allows("/trademark/", "Bot™"));
        assertFalse(robotsTxt.allows("/registered/", "Bot®"));

        // Test wildcard user-agent
        assertFalse(robotsTxt.allows("/default/", "UnknownRandomBot"));
    }

--------------------------------

2.1. Grants

Each Contributor hereby grants You a world-wide, royalty-free,
non-exclusive license:

(a) under intellectual property rights (other than patent or trademark)
    Licensable by such Contributor to use, reproduce, make available,
    modify, display, perform, distribute, and otherwise exploit its
    Contributions, either on an unmodified basis, with Modifications, or

additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty....