Can you publish a example use gpu+cpu+nvme to train big model ？ #1593
dy2009
started this conversation in
Community | General
Replies: 1 comment
-
|
For NVME offload, all you need to do is to replace the optimizer with HybridAdam. For more details, see https://colossalai.readthedocs.io/en/latest/colossalai/colossalai.nn.optimizer.hybrid_adam.html#colossalai.nn.optimizer.hybrid_adam.HybridAdam |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Can you publish a example use gpu+cpu+nvme to train big model use 1 gpu ?
I have tryed ColossalAI-Examples , but no success.
also RuntimeError: CUDA out of memory.
Beta Was this translation helpful? Give feedback.
All reactions