Fine-tuning an LLM using distributed training and multiple GPUs

This project is part of the Deep Learning course for the MSDS program at the University of San Francisco.

Team Members

Submission-1: Notebook

The notebook can be found here

The final video recording resulted in a file of size ~ 500 MB. Hence we have uploaded the video on Youtube as an unlisted video.

The youtube video can be found here

Name		Name	Last commit message	Last commit date
Latest commit History 18 Commits
.gitignore		.gitignore
Decoderarchitecture.png		Decoderarchitecture.png
Finetuning a LLM using LoRA Code Walkthrough .ipynb		Finetuning a LLM using LoRA Code Walkthrough .ipynb
GPTarchitecture.png		GPTarchitecture.png
README.md		README.md
decoder.png		decoder.png
llama.png		llama.png
loss_results.png		loss_results.png
swishrelu.png		swishrelu.png