`element_spec` argument of `tf.data.experimental.load` when loading the
        previously saved dataset.
    *   Add `.element_spec` property to `tf.data.DatasetSpec` to access the
        inner spec. This can be used to extract the structure of nested
        datasets.
    *   Add `tf.data.experimental.AutoShardingPolicy.HINT` which can be used to

  // blocking collective operations at a time), and avoids some dynamic
  // allocation/scheduling.
  //
  // TODO(allenl): Keep a map from outer thread to list of inner threads rather
  // than a single list of threads so aliased nested parallel devices don't
  // re-use a thread.
  std::vector<std::unique_ptr<DeviceThread>> device_threads_;