Explore the intricacies of fine-tuning large language models under resource constraints in this live session from the NeurIPS Hacker Cup AI competition. Delve into optimization techniques like gradient accumulation, activation checkpointing, and LoRa as Joe Cunnings from Meta's torchtune team shares strategies for maximizing model performance with limited hardware. Learn the importance of high-quality datasets and gain practical advice for working within a 40 GB VRAM GPU environment. Perfect for developers seeking to enhance their skills in efficient model fine-tuning and competition-ready AI development. Access the torchtune GitHub repository at https://github.com/pytorch/torchtune for additional resources and tools.
Fine-Tuning Large Language Models with Limited Resources - NeurIPS Hacker Cup AI
Overview
Syllabus
NeurIPS Hacker Cup AI: FineTuning
Taught by
Weights & Biases
Related Courses
-
Fine-Tuning LLM Models - Generative AI Course
-
H2O ai Large Language Models (LLMs) - Level 2
-
Generative AI Advance Fine-Tuning for LLMs
-
Full Fine-tuning LLMs with Lower VRAM: Optimizers, GaLore, and Advanced Techniques
-
Master AI Efficiency with LoRA - Optimize Fine-Tuning for Large Language Models
-
Creating Your Own Flux AI Model with LoRA Fine-Tuning
