Support Convolution&Batchnorm Fusing for Optimized Inference Mode

[DonghakPark]

One of the ways to accelerate running the NNTrainer in inference mode is to fuse operations.
We are currently using this fusion when exporting to TensorFlow Lite.
By applying it to the current NNTrainer, we can improve the speed during inference.

Many Deeplearning Model using Batchnorm after Conv layer

and when inference we can fusing ops to below

image ref

[taos-ci]

cibot: Thank you for posting issue #2646. The person in charge will reply soon.

[lhs8928]

Fusing the operation on infererence means that this graph is little bit different from the graph used to train the model.
To support fusing operation we might need to make 2 graphs for 1 model. (One for training and the other one for inference)

[DonghakPark]

Fusing the operation on infererence means that this graph is little bit different from the graph used to train the model. To support fusing operation we might need to make 2 graphs for 1 model. (One for training and the other one for inference)

That's right !! when we save()&load() model we check [INFERENCE|TRAIN] Mode and make graph