// `op_type` (e.g. "Add"), the operation's inputs (`input_tensors`,
  // `input_tensor_id`, and `input_dtypes`; the latter two are somewhat
  // redundant but taken as arguments to avoid repeatedly fetching these values
  // between calls to ShouldRecord and Accumulator), and its outputs
  // (`output_tensors`).
  //

Niedermeier, Junpeng Lao, Kai Sasaki, @Kankroc, Karl Lessard, Kyle Bostelmann,
@Lezcano, Li Yi, Luo Yun, @lurker, Mahmoud-Abuzaina, Mandeep Singh, Marek
Kolodziej, Mark Szepieniec, Martial Hue, Medhat Omr, Memo Akten, Michael Gharbi,
MichaëL Defferrard, Milan Straka, @MircoT, @mlucool, Muammar Ibn Faisal, Nayana
Thorat, @nghiattran, Nicholas Connor, Nikolaas Steenbergen, Niraj Patel,

        //
        // TODO(allenl): There may be smarter ways to do this copy in some
        // cases, i.e. with a collective broadcast. We'll need to be careful
        // about things that are taken as inputs on the host or on their
        // existing device (for multi-device functions).
        std::unique_ptr<ParallelTensor> parallel_tensor(
            parallel_device.CopyToParallelDevice(