seen couple of use cases where the model usage pattern is not as
            expected and would not work with this change.
    *   output tensors of one layer is used in the constructor of another.
    *   symbolic tensors outside the scope of the model are used in custom loss
        functions. The flag can be disabled for these cases and ideally the
        usage pattern will need to be fixed.