END OF TERMS AND CONDITIONS

   APPENDIX: How to apply the Apache License to your work.

      To apply the Apache License to your work, attach the following
      boilerplate notice, with the fields enclosed by brackets "[]"
      replaced with your own identifying information. (Don't include
      the brackets!)  The text should be enclosed in the appropriate
      comment syntax for the file format. We also recommend that a

        ComponentUtil.register(crawler, Crawler.class.getCanonicalName());
    }

    @Override
    public void tearDown() throws Exception {
        super.tearDown();
        // Clear static fields
        clearStaticFields();
    }

    private void clearStaticFields() throws Exception {
        Field runningField = Crawler.class.getDeclaredField("running");
        runningField.setAccessible(true);

   * implementation. Experimentally determined.
   */
  private static final int MAX_HASH_BUCKET_LENGTH = 9;

  // See CompactHashMap for a detailed description of how the following fields work. That
  // description talks about `keys`, `values`, and `entries`; here the `keys` and `values` arrays
  // are replaced by a single `elements` array but everything else works similarly.

  /**

  // it doesn't escape this class
  @SuppressWarnings("Immutable")
  private final long[] array;

  /*
   * TODO(kevinb): evaluate the trade-offs of going bimorphic to save these two fields from most
   * instances. Note that the instances that would get smaller are the right set to care about
   * optimizing, because the rest have the option of calling `trimmed`.
   */

  private class Itr implements Iterator<E> {
    /** Index of element to be returned by next, or a negative number if no such. */
    private int nextIndex;

    /**
     * nextItem holds on to item fields because once we claim that an element exists in hasNext(),
     * we must return it in the following next() call even if it was in the process of being removed
     * when hasNext() was called.
     */

  // it doesn't escape this class
  @SuppressWarnings("Immutable")
  private final int[] array;

  /*
   * TODO(kevinb): evaluate the trade-offs of going bimorphic to save these two fields from most
   * instances. Note that the instances that would get smaller are the right set to care about
   * optimizing, because the rest have the option of calling `trimmed`.
   */

   END OF TERMS AND CONDITIONS

   APPENDIX: How to apply the Apache License to your work.

      To apply the Apache License to your work, attach the following
      boilerplate notice, with the fields enclosed by brackets "{}"
      replaced with your own identifying information. (Don't include
      the brackets!)  The text should be enclosed in the appropriate
      comment syntax for the file format. We also recommend that a

   END OF TERMS AND CONDITIONS

   APPENDIX: How to apply the Apache License to your work.

      To apply the Apache License to your work, attach the following
      boilerplate notice, with the fields enclosed by brackets "[]"
      replaced with your own identifying information. (Don't include
      the brackets!)  The text should be enclosed in the appropriate
      comment syntax for the file format. We also recommend that a

   END OF TERMS AND CONDITIONS

   APPENDIX: How to apply the Apache License to your work.

      To apply the Apache License to your work, attach the following
      boilerplate notice, with the fields enclosed by brackets "[]"
      replaced with your own identifying information. (Don't include
      the brackets!)  The text should be enclosed in the appropriate
      comment syntax for the file format. We also recommend that a

   END OF TERMS AND CONDITIONS

   APPENDIX: How to apply the Apache License to your work.

      To apply the Apache License to your work, attach the following
      boilerplate notice, with the fields enclosed by brackets "[]"
      replaced with your own identifying information. (Don't include
      the brackets!)  The text should be enclosed in the appropriate
      comment syntax for the file format. We also recommend that a