Adding Torchtune recipe for fine-tuning

[wukaixingxp]

What does this PR do?

Adding Torchtune recipe for fine-tuning so that the users can now use torchtune to fine-tune Llama models

Feature/Issue validation/testing

• Followed this document to use torchtune.

~/work/torchtune/recipes/configs (main)]$ tune run lora_finetune_single_device --config llama3/8B_qlora_single_device
INFO:torchtune.utils.logging:Running LoRAFinetuneRecipeSingleDevice with resolved config:

batch_size: 2
checkpointer:
  _component_: torchtune.utils.FullModelMetaCheckpointer
  checkpoint_dir: /tmp/Meta-Llama-3-8B/original/
  checkpoint_files:
  - consolidated.00.pth
  model_type: LLAMA3
  output_dir: /tmp/Meta-Llama-3-8B/
  recipe_checkpoint: null
compile: false
dataset:
  _component_: torchtune.datasets.alpaca_cleaned_dataset
  train_on_input: true
device: cuda
dtype: bf16
enable_activation_checkpointing: true
epochs: 1
gradient_accumulation_steps: 16
log_every_n_steps: 1
loss:
  _component_: torch.nn.CrossEntropyLoss
lr_scheduler:
  _component_: torchtune.modules.get_cosine_schedule_with_warmup
  num_warmup_steps: 100
max_steps_per_epoch: null
metric_logger:
  _component_: torchtune.utils.metric_logging.DiskLogger
  log_dir: /tmp/qlora_finetune_output/
model:
  _component_: torchtune.models.llama3.qlora_llama3_8b
  apply_lora_to_mlp: true
  apply_lora_to_output: false
  lora_alpha: 16
  lora_attn_modules:
  - q_proj
  - v_proj
  - k_proj
  - output_proj
  lora_rank: 8
optimizer:
  _component_: torch.optim.AdamW
  lr: 0.0003
  weight_decay: 0.01
output_dir: /tmp/qlora_finetune_output/
profiler:
  _component_: torchtune.utils.profiler
  enabled: false
resume_from_checkpoint: false
seed: null
shuffle: true
tokenizer:
  _component_: torchtune.models.llama3.llama3_tokenizer
  path: /tmp/Meta-Llama-3-8B/original/tokenizer.model

DEBUG:torchtune.utils.logging:Setting manual seed to local seed 4023430205. Local seed is seed + rank = 4023430205 + 0
Writing logs to /tmp/qlora_finetune_output/log_1717628124.txt
INFO:torchtune.utils.logging:Model is initialized with precision torch.bfloat16.
INFO:torchtune.utils.logging:Memory Stats after model init:
{'peak_memory_active': 10.126081536, 'peak_memory_alloc': 10.126081536, 'peak_memory_reserved': 12.639535104}
INFO:torchtune.utils.logging:Tokenizer is initialized from file.
INFO:torchtune.utils.logging:Optimizer and loss are initialized.
INFO:torchtune.utils.logging:Loss is initialized.
Downloading readme: 100%|██████████████████████████████████████████████████████████████| 11.6k/11.6k [00:00<00:00, 56.7MB/s]
Downloading data: 100%|████████████████████████████████████████████████████████████████| 44.3M/44.3M [00:00<00:00, 88.3MB/s]
Generating train split: 51760 examples [00:00, 214648.47 examples/s]
INFO:torchtune.utils.logging:Dataset and Sampler are initialized.
INFO:torchtune.utils.logging:Learning rate scheduler is initialized.
1|33|Loss: 1.6673557758331299:   0%|                                                   | 32/25880 [00:24<5:37:10,  1.28it/s]1|33|Loss: 1.6673557758331299:   0%|                                                   | 32/25880 [00:25<5:38:58,  1.27it/s]

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[HamidShojanazeri]

@wukaixingxp can you pls move it under recipes/finetuning/torchtune

[wukaixingxp]

sure, moved to recipes/finetuning/torchtune.md

[HamidShojanazeri]

@wukaixingxp sorry I mean to have a torchtune folder under finetuning, it would be recipes/finetuning/torchtune/torchtune.md