Crawl-delay: 1.23e10

# Case 14: Tab characters instead of spaces
User-agent:	TabBot
Disallow:	/tab1/
Allow:	/tab2/

# Case 15: Unicode and special characters in user-agent
User-agent: Bot™
Disallow: /trademark/

User-agent: Bot®
Disallow: /registered/

# Case 16: Multiple wildcards user agent
User-agent: *
Disallow: /default/

# Case 17: Empty file handling (just comments and whitespace after this)

      the terms of any separate license agreement you may have executed
      with Licensor regarding such Contributions.

   6. Trademarks. This License does not grant permission to use the trade
      names, trademarks, service marks, or product names of the Licensor,
      except as required for reasonable and customary use in describing the

    d) Limiting the use for publicity purposes of names of licensors or
    authors of the material; or

    e) Declining to grant rights under trademark law for use of some
    trade names, trademarks, or service marks; or

    f) Requiring indemnification of licensors and authors of that
    material by anyone who conveys the material (or modified versions of

      the terms of any separate license agreement you may have executed
      with Licensor regarding such Contributions.

   6. Trademarks. This License does not grant permission to use the trade
      names, trademarks, service marks, or product names of the Licensor,
      except as required for reasonable and customary use in describing the

        assertTrue(robotsTxt.allows("/tab2/", "TabBot"));

        // Test bots with special characters - should be normalized to lowercase
        assertFalse(robotsTxt.allows("/trademark/", "Bot™"));
        assertFalse(robotsTxt.allows("/registered/", "Bot®"));

        // Test wildcard user-agent
        assertFalse(robotsTxt.allows("/default/", "UnknownRandomBot"));
    }

--------------------------------

2.1. Grants

Each Contributor hereby grants You a world-wide, royalty-free,
non-exclusive license:

(a) under intellectual property rights (other than patent or trademark)
    Licensable by such Contributor to use, reproduce, make available,
    modify, display, perform, distribute, and otherwise exploit its
    Contributions, either on an unmodified basis, with Modifications, or

additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty....

 * names to avoid confusing characters. This includes basic case folding: transforming shouting
 * `SQUARE.COM` into cool and casual `square.com`. It also handles more exotic characters. For
 * example, the Unicode trademark sign (™) could be confused for the letters "TM" in
 * `http://ho™ail.com`. To mitigate this, the single character (™) maps to the string (tm). There