1. SC forward pass
  2. TC forward/backward pass
  3. SC backward pass
  4. non-TPU loop counter updates
These 4 functions are then staggered so as to enable parallel execution.

In pseudocode, the algorithm is as follows:

// Start step 0
C_0 = cond(args_0)
N_0 = non_tpu(args_0)
if (C_0) {
   F_0 = forward(args_0, N_0)
   T_0 = core_tpu(args_0, N_0, F_0)
   // B_0 = backward() is not evaluated here.