How to fix the CUDA out of memory error when using Multi-Step Loss Optimization (MSL)?

[machanic]

Dear author:
when I ran your code I found that
https://github.com/AntreasAntoniou/HowToTrainYourMAMLPytorch/blob/master/few_shot_learning_system.py#L196
This line will cause the OOM error because each time the gradient of loss backprop w.r.t. the whole network's parameters will be recorded. And such gradient will not be freed util all the mini-batch's multi-tasks loop over.

I use 11Gb 1080Ti to run CIFAR-10 dataset, but it report CUDA out of memory error.
How to deal and fix it?

[pandeydeep9]

Maybe this could help: reduce the batch size or the number of inner loop updates from 5 to say 2 or 3.