For non low_resource llama loading code, I added an argument low_cpu_…

…mem_usage=True, drastically speeding up load time (cuts more than a minute off of load)
Vision-CAIR · Oct 11, 2023 · 5ab62fe · 5ab62fe
1 parent ef1ac08
commit 5ab62fe
Showing 1 changed file with 1 addition and 0 deletions.
diff --git a/minigpt4/models/mini_gpt4.py b/minigpt4/models/mini_gpt4.py
@@ -118,6 +118,7 @@ def __init__(
             self.llama_model = LlamaForCausalLM.from_pretrained(
                 llama_model,
                 torch_dtype=torch.float16,
+                low_cpu_mem_usage=True,
             )
 
         if lora_r > 0: